Update to STM API

A few weeks ago we saw details of the new optimistic concurrency control (MVCC) for the STM implementation. At the time I mentioned that the syntax for using pessimistic and optimistic was different and also slightly more verbose than it really needed to be. So there have been a few changes recently and although it's not quite the way I want it, it's worth discussing now to at least give you an idea of where we are going with things. Especially important if you're trying to develop something using the STM and the previous examples as a template, since although all of those original classes and methods continue to exist, they may have moved packages!

I'll assume that you can compare and contrast by looking at the original posting, so here's how you create and use objects now:

Container theContainer = new Container();

Sample1 obj1 = theContainer.create(new SampleImple(10));

Sample1 obj2 = theContainer.create(new SampleImple(10));

Sample1 obj3 = theContainer.clone(new SampleImple(), obj1);

Sample1 obj4 = theContainer.clone(new SampleImple(), obj2);

It no longer matters whether they are @Pessimistic or @Optimistic (the default) - the syntax remains the same. Here we have two objects, obj1 and obj2 and two clones (copies) of them, obj3 and obj4. They're all recoverable (state will not survive a machine crash), since that's the default behaviour for Container, but that can be changed via the constructor which allows for PERSISTENT or RECOVERABLE to be specified at creation time.

You can probably guess where further improvements are going to happen. For a start, the fact that the clone method takes a "blank" instance to copy the state into is necessary, but it would be better if it could be inferred. In addition, there are some state management aspects of the implementation that need tidying up because they currently escape through into the application. However, these are both fairly straightforward aspects to fix and the plan is to get them finalised soon. But for now, these changes should be sufficient to try things out. Enjoy!

The Future of XTS

Last September I took over the great work that Andrew Dinn has been doing leading the XTS (Web service transactions) project. Since then I've been having a think about where we should be taking the project over the next few years. In this post I want to present our ideas to the community. Hopefully this will give you an idea of where we are going with XTS and give you an opportunity to provide feedback. I'll also refer to work planned for the Narayana project (AKA JBossTS), where it also applies to XTS.

About Me
First of all I'll take this opportunity to introduce myself. I've worked in the field of transactions and Java middleware for over 10 years. I started out in 2001 as a software engineer on the XTS project, recently after it was founded by Hewlette Packard. Here I helped to develop the industry's first Web service transaction implementation. In those days we were implementing the BTP standard. Since then I headed up the consultancy arm of Arjuna Technologies, helping high-profile customers deploy JBoss Transactions, JBossWS and also Arjuna's Cloud Computing product. In late summer of 2011 I made the jump to Red Hat to lead the XTS project. I have also received a PhD in middleware and non-repudiation from the University of Newcastle upon Tyne, where I now hold the role of Visiting Research Fellow. In this role I teach JEE and distributed transactions as part of a masters course.

After reviewing where the XTS project currently stands we've had a good think about what areas of improvement we should be focusing on over the coming months and years. We've found that these improvements tend to fall into one of four 'themes'; usability, modernization, streamlining and consistency.

Usability
Improving usability is our number one priority. In particular we will be introducing quickstarts (JBTM-870) and making significant improvements to the XTS java API.

Currently we have a fantastic demonstrator that shows the developer how to use, pretty much, every feature of XTS. The problem is that it is not an easy place to start for those new to XTS. Recently we added a number of basic quickstarts to the JBossAS 7 quickstart project. These quickstarts are designed as a place to start when beginning to use XTS. We also have plans to provide additional quickstarts as part of the Narayana quickstart project (JBTM-870). These quickstarts will focus on more advanced features, such as recovery. These quickstarts will adhere to the same strict guidelines imposed by the JBossAS team. This will bring a high level of quality as well as familiarity. We also intend to provide unit tests with every quickstart; and those in the Narayana quickstart project will undergo continuous integration on our Hudson servers (JBTM-1068). This should ensure our quickstarts always work in the intended environment. We have an idea of what quickstarts we should initially provide, but we feel it would be best to be community driven in the future. So if you have an idea for a quickstart, or maybe you have one to contribute, then please get in touch on the forum.

The biggest changes around usability are coming as part of our new sub-project; currently named the 'TXFramework'. The TXFramework is a new and novel framework for writing portable ACID and NoACID transactional applications. It provides an abstract annotation based API for specifying transactional requirements. This decouples the application logic from the actual transaction model used (WS-AT, REST_AT, JTA, etc). It also removes a lot of boilerplate code that's currently needed by WS-AT and WS-BA services. The TXFramework also provides seamless bridging between different transaction protocols. This subject could cover many blog postings, so I won't discuss it in any more detail here. Stay tuned for more posts on this subject in the coming months. I'll also be presenting the TXFramework at JUDCon2012:Boston on the 25th June. Hope to see you there!

Modernisation
The next focus is around modernization. There's been a lot of exciting new technology emerging over the last few years and we are making many improvements to take advantage of it. In particular, Arquillian and Git.

The XTS functional tests need to run in a container. Before JBoss developed the Arquillian project, we were forced to create an in-container test-runner to run our functional tests. By migrating to Arquillian we are now able to remove this test-runner code and take advantage of additional benefits that Arquillian brings. In particular we can run any/all of the XTS tests from with the IDE as regular JUnit tests. Arquillian also manages the setup and teardown of the test environment, making it a lot easier to run the tests. This improvement is going to remove a lot of code from the XTS codebase and bring great simplification to the way we write and run tests. JBTM-954, JBTM-955, JBTM-956

Narayana and Blacktie were recently migrated from SVN to Git and Github (See here). Git has been gaining a lot of traction recently in the world of open source and most JBoss projects have now made the leap. After reviewing the benefits of moving, we decided now was the time to make the jump. In particular it was the improvements to collaboration provided by Git and GitHub that made us feel that this was the right thing to do. If you are a contributor, you're going to find it a lot easier to get involved with XTS and Narayana.

We've also moved the entire Narayana project to maven, which is making it a lot easier for our contributors to get up and running quickly and is making it easier for us to maintain our build system. We're also beginning to use JaCoCo to check our test coverage JBTM-952. This ensures that XTS is fully tested and helps find redundant code.


Streamlining
We are doing some streamlining too to the XTS code base. We are taking two approaches here; as mentioned above we are using JaCoCo to help find redundant code. This should cut down the amount of code we need to maintain and help contributors get up to speed more quickly as they don't need to sift through unused code (JBTM-741). The second approach we are taking is to replace code with third party libraries where we can be sure that they will provide an equal or better implementation. For example, as stated above we are in the process of replacing our test-harness code with Arquillian (JBTM-953).


Consistency
Being productive and helping our contributors be productive is very important to us. Having to switch between and understand different practices when working with our projects does not help with this goal. Therefore, we are aiming to bring as much consistency as we can between XTS and Narayana and to some extent between Narayana and the other JBoss projects. A move to improved consistency also brings with it an opportunity to re-use ideas that have been proven elsewhere. In particular, our quickstarts are now in the process of being moved to a single repository and follow the strict guidlines set out by the JBossAS team. The XTS codebase is being re-structured to follow the default maven layout (JBTM-968). This should make it easier for new contributors to follow our layout.

Getting Involved
Of course, we would love the community to get involved with XTS and the Narayana project. I mentioned many pieces of work in this post; you should follow the Jira issue links and vote up the ones that matter to you. If you have an idea for something you would like to be included, please start a discussion on the Narayana forum. You can also fork our GitHub projects if you'd like to get involved with the coding side.

Transactions and NoSQL - not reinventing the wheel!

It's nice to see some NoSQL solutions recognising that they need transactions (or extended transactions) and rediscovering the work that the transactions community (ourselves included) have been working on for a few decades! This presentation is interesting, but frustrating in a few places, e.g., the discussion around the viability of nested transactions for long duration interactions, in general the discussion around Sagas, and the Q&A at the end where the glossing over of technical aspects during the talk really lead to confusion.

However, that aside, it's not a bad overview of the needs for long duration, compensation based transactions and references a few of the important papers and research in this space going back several decades. Of course if you've been reading this blog long enough or watching the work we've been doing for a while, you'll know about others, including:

• the compensation REST transactions work;
• a history of extended transactions;
• a comparison of two of the original standards efforts in the area of compensation transactions;
• a couple of presentations on early standards effort around Web Services and transactions;
• an overview of the wider expanse of work that has been done around extended transactions, workflow, messaging, choreography etc.

The reference to the Sagas pattern that is mentioned, but forgotten, in the presentation I mentioned originally can be found here. It's nice to see that at least in the case of this NoSQL implementation, the engineers are willing to see what's gone before and learn from it. And if you want to look at implementations (and frameworks) that do much of this already, then take a look at JBossTS - without you having to write your own do/undo log!