How do you persist state of Sagas?

Just a quick question on saga persistence - how do you persist saga state and dispatch messages while avoiding transactions and 2PC? 

Long story: I'm trying to reason out the logic behind sagas, in order to understand everything better (and map concepts back to the reactive programming) 

Basically a saga is an entity, that is used to coordinate some long- running process. It can subscribe to events (UserAccountCreated), keep track of time (i.e.: user should activate his account within 24 hours) and send commands (CancelUserRegistration). 

Additionally, since saga is an entity and could be addressed in the scalable world, we can send command directly to the saga (StopRegistrationProcess). Sagas can be modeled and perceived as finite state machines. So far - so good and rather straightforward. 

However just a quick question: how do you persist saga state and send messages out of it? 

Logically, in order to avoid 2PC and transactions you would need to join state transition and publication in one atomic operation (just like with the aggregate roots and event sourcing) and reuse message dispatching mechanism that catches up with the history (append-only persistence scales much better anyway) 

This feels like more sensible and simple operation, than introducing relational DBs or any kind of transactions into the system. However, as I recall, I've never heard of using event sourcing for the saga state persistence. Is there a reason for this? How do you implement your sagas and persist their state? 

All feedback would be appreciated! 

Best regards, 
Rinat Abdullin

 

Rinat, 

There are a few options two avoid 2PC.  One of the easiest ways is to simply have the saga entity store a list of all command IDs internally.  Rarely will you have sagas that exist beyond even several dozen commands/events.  That being the case, you can effectively treat the saga as a kind of aggregate root using event sourcing. (More on this in a minute.) 

By storing the command IDs internal to the saga, you can avoid 2PC by having two completely separate transactions--an outer as well as an inner transaction.  The inner transaction is related to committing the saga "aggregate" to the event store.  The outer transaction is related to removing the message from the message queue.  If the message queue doesn't support TransactionScope, it's not a big deal--it will attempt to deliver the message at least once and you can easily detect it as a duplicate and drop it because it's already been handled. 

Let the event store do the publishing for you asynchronously. 

I've outlined a few of these concepts in some blog posts I wrote a few 
months back (one of which you commented on): 
http://jonathan-oliver.blogspot.com/2010/04/extending-nservicebus-avoiding-two.html 
http://jonathan-oliver.blogspot.com/2010/04/idempotency-patterns.html 
http://jonathan-oliver.blogspot.com/2010/04/message-idempotency-patterns-and-disk.html 

The other part of your question is how to leverage event sourcing to take care of sagas.  It's not unlike your typically aggregate root. Some kind of stimulus comes in (either a command or event), you transition the state (this being the part that's distinct from DDD aggregates), which results in a message being "raised".  Then, you commit the new state to the event store and let it perform the message dispatch asynchronously. 

Jonathan Oliver

The only thing that I would add is that sagas should be more like a state machine which is about *process*, whereas our aggregates are more about *logic* (if statements and flow control).

Jonathan Oliver 

Ah, thanks a lot guys.

So basically for the saga persistence we can have either event sourcing (command is saved along with the events in the transaction) or simple state storage (command is saved along with the latest state and possible outgoing events). Dispatcher could dispatch in async later in both cases.

Once we have command info persisted atomically with the resulting changes, we can have all the idempotence we need (still staying away from the 2PC). Consistency is 100% even if process dies between the commit and ACK.

So technically sagas are just like the aggregates (they are entities), and the primary difference is in the intent (similar to the differencebetween commands and events) and life span expectations.

This way everything that happens in saga between the handler and message dispatch is rather straightforward, reliable and simple (and
similar to the aggregate behavior).

Thanks again for helping to think though the logic of this part of CQRS!

Best regards,
Rinat

I agree. Sagas and aggregates have different intent plus resulting differences in behavior, life cycle and persistence. Ignoring this  in
the project might kick in the natural selection process for it.

However, implementation logic of command handlers outside of these "inner" specifics seem to be similar for both cases (i.e.: questions of reliability, 2PC, transactions and message dispatch). Don't you think?

Best regards,
Rinat

 

hi Rinat,

I'm using Esper for my "sagas" and currently I can rebuild it's state by replaying events at startup.

Esper allows one send timetick events to control the flow of time when replaying in isolation, and it's pretty awesome!

Pedro H S Teixeira

 

Hi,

Can anybody provide with a pseudo code for saga?

That would make things more clear.

Bhoomi Kakaiya

 

 

Bhoomi,

I've published an article that goes into some deeper on Sagas (as per discussions in this thread and outside of it).

Although there is still no source code, but it might help to understand everything.

http://abdullin.com/journal/2010/9/26/theory-of-cqrs-command-handlers-sagas-ars-and-event-subscrip.html

Just a caveat: I'm sorry for going into deep details about the partitioning logic (this was needed by the specifics). In practice implementations will probably skip this part completely in 95% of cases (and go lightly on a few other explicit constraints as well).

Best regards,
Rinat