In Activator message-handling, sequential delivery is the ability to deliver messages in the order they were originally consumed on a specific pickup, and also filtered for delivery by the resolved destination application or partner delivery. This is sometimes referred to as first-in-first-out (FIFO) behavior.
An Activator internal sequence manager ensures that messages are delivered in the order in which they are consumed, per pickup and per resolved destination partner or application pickup delivery.
You can currently configure sequential message delivery on the following application and partner pickups:
For FTP and SFTP embedded servers:
There is no sequential message handling during the processing phase. The sequential IDs that are assigned on consumption (pickup) are only taken into account at production (delivery).
In general, sequential delivery can only work if one and only one output message is generated for each input message.
For MQ, SMTP, FTP and SFTP clients:
For the FTP and SFTP embedded servers:
Activator reorders messages according to the order of message assigned by the pickup process for all deliveries.
When Activator assigns a new sequence ID and number (derived from consummation order and destination delivery exchange) it adds the previous SequenceId and SequenceCount to the metadata (PreviousSequenceId and PreviousSequenceNumber respectively). You can then use this metadata in Message Tracker searches to track messages in their previous sequence.
In the case where Activator allocates a sequence ID/Number but fails to produce the message to the integration engine, it produces an empty message. This can occur when Activator starts to consume a payload, assigns the sequence ID/Number, but then payload transfer fails. In this case, the allocated sequence ID/Number is canceled in Activator and an empty message is sent to the integration engine indicating that no message will be received for that ID/Number.
Normal sequence:
| Sequence ID | Sequence number |
|---|---|
| 1 | 1 |
| 1 | 2 |
| 1 | 3 |
| 1 | 4 |
Sequence with failed message number 3.
| Sequence ID | Sequence number |
|---|---|
| 1 | 1 |
| 1 | 2 |
| 1 | 3 (failed) |
| 1 | 4 |
Sequence with dummy message replacing failed message number 3.
| Sequence ID | Sequence number |
|---|---|
| 1 | 1 |
| 1 | 2 |
| 1 | 3 (dummy/empty) |
| 1 | 4 |
In the case of multiple out of sequence instances, the sequence is not guaranteed.
For example:
| Message number | Status | Timeout status (seconds) |
|---|---|---|
| 1 | Available | – |
| 2 | Not available | – |
| 3 | Available | 60 |
| 4 | Not available | 5 |
| 5 | Available | 0 |
In this example, messages 3, 4 and 5 wait until 1 and 2 are available. When 5 expires, Activator checks the availability of 4. If 4 is available, Activator ignores the timer status. This behavior cascades upstream to generate the new status:
| Message number | Status | Timeout status (seconds) |
|---|---|---|
| 1 | Not available | – |
| 2 | Not available | – |
| 3 | Available | 55 |
| 4 | Available | – |
| 5 | Available | – |
If message number 3 arrives at the time-out limit and messages 1 and 2 are still not available, Activator sends messages 3, 4 and 5 in sequence.
In the case of multiple out of sequence gaps, the situation is different. Consider the following example:
| Message number | Status | Timeout status (seconds) |
|---|---|---|
| 1 | Not available | – |
| 2 | Available | 60 |
| 3 | Not available | – |
| 4 | Available | 55 |
| 5 | Available | 50 |
If message 5 times out first, Activator checks the availability of message 4. Because message number 4 is available, Activator ignores the timer.
| Message number | Status | Timeout status (seconds) |
|---|---|---|
| 1 | Not available | – |
| 2 | Available | 5 |
| 3 | Not available | – |
| 4 | Available | 0 |
| 5 | Available | – |
When message number 4 times out, Activator detects that message 3 has not yet been received. Activator sends message 4, despite the fact that number 2 did not time out yet. Activator sends messages number 4 and 5 before message number 2. The order between 4 and 5 is maintained.
In the case of a larger number of queued messages and multiple sequence gaps between messages, this logic causes out of sequence message delivery: Activator sets timers on the various messages. When the time-outs expire there will likely be a gap between the “expired” entry and the last processed entry. This causes the expired entry, and everything after it, to be delivered out of sequence.
For example:
Messages 1, 2 and 3 are ready to be sent. Message 2 is in the actual process of being sent. Message 2 and 3 are in the queue. Then the system stops abruptly. On system restart, messages 1, 2 and 3 are all pushed back to the queue. Their sequence is maintained and their timers are reset (in this example to default = 60 seconds).
Before restart:
| Message number | Status | Timeout status (seconds) |
|---|---|---|
| 1 | In transfer | – |
| 2 | Available | 5 |
| 3 | Available | 10 |
After restart:
| Message number | Status | Timeout status (seconds) |
|---|---|---|
| 1 | In transfer (reintroduced) | – |
| 2 | Available | 60 |
| 3 | Available | 60 |
A side effect of this pattern is that any message that timed out will get a new opportunity for delivery after the restart. For example, Message 1 is missing and 2 and 3 are still waiting for this message. When the system restarts, message 2 and 3 are reintroduced by the fail-over coordinator and the timers for message 2 and 3 are re-started., so message 1 is allotted an additional X seconds to arrive.
| Message number | Status | Timeout status (seconds) |
|---|---|---|
| 1 | Not available | – |
| 2 | Available | 60 |
| 3 | Available | 60 |
After a shut down, the new messages that Activator picks up from either the application side or the trading side, end up back in the list of to-do work. As reload tasks are limited to 50% of the capacity, it can take a while achieve full recovery.
Activator does not reintroduce work in sequential order, so the likelihood of time-outs and gaps occurring is much higher when there are a lot of messages being processed at shutdown. For example, if the trading engine has 3000 sequenced messages to restart, it could end up re-introducing the messages in random sequence order (299, 1, 1967, 556, etc). Since this occurs at 50% of processing capacity, timeouts are likely.
With larger numbers and gaps between messages, this logic will cause out of sequence message delivery. For example, if we have a large quantity of messages in the backlog and new messages are consumed for processing, Activator sets timers for this “new” work. After the time-out expires, there will most likely be a gap between the “expired” entry and the last processed entry. The result is that particular entry (and everything after it) is delivered out of sequence.
The failover coordinator stops injecting old workload if Activator throttle is invoked due to new work.
When Activator rejects messages because the processing failed, it is the responsibility of the sending user to resubmit messages in the proper sequenced order.
If sequencing is activated on the corresponding pickup, Activator guarantees that the order in which messages are re-submitted is kept.
The following metadata attributes are now added to messages that are handled in Activator:
Together these attributes provide a unique sequence identity for messages consumed by Activator pickups, when sequential messaging is activated for the pickup.