Application servers are dead. Apparently.

OK so you may be wondering what the title has to do with transactions. Well it's really because of this article I came across recently. It's slightly old and I'm not sure how I missed it the first time round (maybe I had better things to do at the time!) Anyway, I'm not going to argue about whether or not application servers are dead: you can find enough from me on that topic by searching on JBoss.org or elsewhere.

What I did want to focus on was the transactions references in the article. Basically the author looks at some of the capabilities that an application server provides and tries to show how the same things can be done outside of an application server or with other frameworks/stacks. Well you can probably guess how the comments on transactions went down with me! Let's start by deconstructing the paragraph - yes, it only deserved a paragraph on something I think is so fundamental to enterprise applications! But hey, maybe I'm just strange (don't go there!)

"2PC is used to coordinate multiple transaction resource that should take part in a single transaction. Whether this feature makes sense could be the subject of an article in its own right. " Now of course you could interpret the last sentence in a number of ways, but I'll give the author the benefit of the doubt. Yes, transactions aren't always the right thing to use and in some cases they've definitely been misused.

Then we have: "If 2PC is used to coordinate two databases the architecture might be wrong. Why is data distributed across two databases if it has such a close relation that it should be changed as part of a single transaction?" Erm, exsqueeze me?! Have you heard of separation of concerns? Or maybe different organisations? Or perhaps difference companies? Or what about different levels of security/encryption, so a company stores and updates data in different repositories/data stores with different levels? And of course there's that other example ... replication!

The next bit made be laugh and then cry: "If JMS and access to a database are done in the same transaction synchronization might be an alternative. It offers almost the same guarantees but is a lot less complex and resource consuming." I can almost imagine the conversation: "Yes sir, we lost your money transfer because the message got lost but we were assured the chances of it happening were so small we really didn't need transactions and could use something that was almost exactly the same thing!" Look, a Citroen Belingo is almost the same thing as a Ferrari but I know which one I'd prefer!

"And remember: Also 2 PC can fail – like any IT system." Yes, it can fail. But guess what? A good transaction system implementation will have something called crash recovery (or failure recovery) which will attempt to resolve things for you automatically so you, the developer or sys admin, don't have to. Catastrophic failures may be unrecoverable, but then that would be the case without transactions anyway.

Then it wraps up with: "In modern Systems that use technologies such as REST or NoSQL 2PC cannot be used anyway. These technologies do not support 2PC. So more often than not 2PC is not that useful." OK so what about things like REST Transactions (the clue's in the name!) and the work we've been doing on compensation transactions and NoSQL?

Look, in general the recommendation(s) to stay away from transactions is bogus and ill-informed. If you don't want to use an application server that doesn't mean you should throw away everything that you get from it such as transactions. For instance, you don't need to use Narayana in the scope of an application server - in fact it began life years before there was ever such a thing!

2PC or 3PC

"2PC or not 2PC, that is the question". Perhaps if Shakespeare were alive today Hamlet would be asking one of the most popular questions in distributed systems for at least the past three decades. I'm not going to revisit the reasons why we use a consensus protocol in ACID transactions, because you can find enough articles on the subject if you read this blog. However, it seems that the distinction between different consensus protocols, specifically two-phase commit (2PC) and three-phase commit (3PC) is not necessarily as widely understood, or rather why vendor transaction managers use 2PC. Yes, there are other consensus protocols, such as Paxos, but let's just focus on 2PC or 3PC today. Before we start looking at why 2PC is more popular, if you haven't read about the FischerLynchPaterson (FLP) results which show that you can't reach agreement in an asynchronous environment if even one failure is allowed, without augmenting the system with, say, failure detection mechanisms.

Strict 2PC is a blocking protocol: for instance, if the coordinator crashes during the second (commit) phase, then the participants must remain in their indeterminate state until it recovers, which could be forever if recovery never happens. Now of course using a strict 2PC protocol in the real world would immediately present some problems: as long as participants remain blocked, the resources that they represent and hold, e.g., locks, are maintained, thus possibly preventing further work on behalf of others. If failures never happen then it's not an issue, but unfortunately the 2nd law of thermodynamics will always get you in the end: entropy increases and failures do happen, no matter how improbable.

This is why heuristics were introduced as a way of allowing 2PC to operate in a more pragmatic manner: any participant that has got past the first (prepare) phase and does not receive a response from the coordinator in a timely manner can unilaterally decide to commit or abort its portion of the transaction. If it gets the decision right (same as the one the coordinator made) then everything is fine. If it gets the decision wrong, then we have a non-atomic outcome, or a heuristic outcome. Resolving this typically requires outside help, e.g., a system administrator. However, with the assumption that failures are relatively rare coupled with the fact that a failure would have to happen after the prepare phase to potentially cause a heuristic, this is not such a bad compromise. Yes heuristics happen, but they will typically be rare. And of course let's not forget about those other optimisations we typically use with 2PC, such as presumed abort, which help to reduce performance overhead as well as allowing safe autonomous choices to be made which do not result in heuristics.

But what about 3PC? You can find good descriptions of the protocol in the literature and elsewhere, so I won't go into details here. Suffice it to say that 3PC removes the blocking nature by disseminating the decision from the coordinator about whether to commit or abort the transaction amongst all of the participants. This means that if the coordinator does crash, the participants can still move forward to complete the transaction. In theory this is a good thing. However, an additional phase, which includes information about the transaction outcome and participants, introduces an overhead in every transaction and this overhead is only useful if the coordinator fails.

As such, 3PC is good for environments where failures are common (more probable) and where heuristics aren't allowed. This additional overhead is not something which the majority of environments, users, use cases etc. are prepared to accept and prefer to use 2PC, along with the other optimisations and trade-offs I mentioned earlier. We optimise for the failure-free environment, which is still the most probable situation for the majority of transaction use cases. That's why all major transaction implementations today use 2PC and not 3PC. Now that doesn't mean it will continue to be the case: if environments or scenarios change in such a way that blocking or heuristics are no longer applicable (maybe mobile) then we may see a change in direction.