Saturday, May 16, 2015

XA and microservices

I had a very interesting series of discussions this week with a few friends/colleagues about whether or not transactions are applicable in a microservices architecture. I've already covered these points in an earlier entry so I won't repeat them here. However, several things did come up that are worthy of trying to address, including the assumption that they're too hard to use and too hard to test your application for correctness when using them. Now as I've mentioned in several presentations there are valid concerns like these around using or not using transactions in general, i.e., irrespective of microservices.

These points that came up recently are not just to be dismissed out of hand. Having worked in transactions for almost 30 years I've spent a long time hearing about these and other concerns and trying to address them within our industry with many others from a range of companies. For instance, if you look at what we've achieved within Java EE with CDI annotations for transactions, or within Narayana for similar approaches towards compensations, I think we've come a long way to simplifying how developers can use transactions. The STM work that we've done is also another example of putting transactions into the hands of developers and not requiring them to have to worry even about demarcation or state management.

Testing applications and their use of transactions is obviously important, in just the same way as when adding any component or capability, such as Hibernate, Zookeeper or Map/Reduce. However, once again we've made a lot of progress over the years. Within Narayana we have 1000s of unit and QA tests which not only help to prove the correctness of the underlying implementation but also feed into best practices etc. for developers. Then there's transaction extensions to Arquillian which also help to make testing much easier. Now in terms of proving the correctness of transaction systems we're also pretty safe there (no pun intended). Since transactions have been used in mission critical environments for over five decades there's a large body of associated work to prove the various aspects are correct for implementations and applications. Overall I'm fairly sure that these concerns have been addressed sufficiently well to not be a meaningful reason to avoid transactions, in general or with microservices. As an industry could we do better? Of course. We should always be striving to improve the overall developer experience and how transactions fit in.

However, one other thing which did come up during the conversations I referred to initially is that there's often a concern that transactions equals XA, or put another way, that the problems perceived to be present with transactions are the result of design issues inherent within the original XA protocol and associated implementations. As I keep saying, 2PC and transactions existed before XA and many transaction system implementations aren't tied to only supporting the XA protocol. All of the transaction standards have their quirks, just as most standards do no matter their domain. But XA has a few more than its fair share, often due to the fact it's been around for so long that the ways in which we tend to develop applications have changed radically since it was defined. Yet we're tied to it due to the fact that relational databases still represent the majority way in which we store data. Of course we've developed new transaction standards which don't rely upon XA, such as OTS or WS-AT, but most developers still see and use XA and hence why there may be concerns about transactions which are really more to do with XA.

Back to the topic: I'm not suggesting that XA doesn't have a role to play within microservices and between microservices. At least in the short term it most definitely does for some (small) set of applications. But really when I'm suggesting transactions have a role I'm looking well beyond XA. I'm still considering ACID as well as compensation transactions though because as I've said, some implementations aren't tied to XA and yet provide the optimisations you expect from XA (one-phase commit, read-only optimisation) as well as going beyond (interposition, nested transactions). So whilst I definitely understand concerns about transactions, I still believe they either have been addressed and we need to do a better job of developer education, or we can address them; the benefits they bring are worth it.

Thursday, May 7, 2015

Application servers are dead. Apparently.

OK so you may be wondering what the title has to do with transactions. Well it's really because of this article I came across recently. It's slightly old and I'm not sure how I missed it the first time round (maybe I had better things to do at the time!) Anyway, I'm not going to argue about whether or not application servers are dead: you can find enough from me on that topic by searching on or elsewhere.

What I did want to focus on was the transactions references in the article. Basically the author looks at some of the capabilities that an application server provides and tries to show how the same things can be done outside of an application server or with other frameworks/stacks. Well you can probably guess how the comments on transactions went down with me! Let's start by deconstructing the paragraph - yes, it only deserved a paragraph on something I think is so fundamental to enterprise applications! But hey, maybe I'm just strange (don't go there!)

"2PC is used to coordinate multiple transaction resource that should take part in a single transaction. Whether this feature makes sense could be the subject of an article in its own right. " Now of course you could interpret the last sentence in a number of ways, but I'll give the author the benefit of the doubt. Yes, transactions aren't always the right thing to use and in some cases they've definitely been misused.

Then we have: "If 2PC is used to coordinate two databases the architecture might be wrong. Why is data distributed across two databases if it has such a close relation that it should be changed as part of a single transaction?" Erm, exsqueeze me?! Have you heard of separation of concerns? Or maybe different organisations? Or perhaps difference companies? Or what about different levels of security/encryption, so a company stores and updates data in different repositories/data stores with different levels? And of course there's that other example ... replication!

The next bit made be laugh and then cry: "If JMS and access to a database are done in the same transaction synchronization might be an alternative. It offers almost the same guarantees but is a lot less complex and resource consuming." I can almost imagine the conversation: "Yes sir, we lost your money transfer because the message got lost but we were assured the chances of it happening were so small we really didn't need transactions and could use something that was almost exactly the same thing!" Look, a Citroen Belingo is almost the same thing as a Ferrari but I know which one I'd prefer!

"And remember: Also 2 PC can fail – like any IT system." Yes, it can fail. But guess what? A good transaction system implementation will have something called crash recovery (or failure recovery) which will attempt to resolve things for you automatically so you, the developer or sys admin, don't have to. Catastrophic failures may be unrecoverable, but then that would be the case without transactions anyway.

Then it wraps up with: "In modern Systems that use technologies such as REST or NoSQL 2PC cannot be used anyway. These technologies do not support 2PC. So more often than not 2PC is not that useful." OK so what about things like REST Transactions (the clue's in the name!) and the work we've been doing on compensation transactions and NoSQL?

Look, in general the recommendation(s) to stay away from transactions is bogus and ill-informed. If you don't want to use an application server that doesn't mean you should throw away everything that you get from it such as transactions. For instance, you don't need to use Narayana in the scope of an application server - in fact it began life years before there was ever such a thing!