A transactional Android example

OK, so as I announced a couple of days ago, the initial code that allows transactional applications to be written on Android is available. There's even an example demo. Therefore, I thought I'd go through the application and its associated configuration in case people wanted to give it a try in its current state. Also rather than try to address all of the idiosyncrasies of the various Android handsets out there, we'll focus on using the Eclipse emulator this time around.

OK, so let's get started. I'll assume that you've checked out the source for the example (it comes with everything you need, so although you can also check out the source for JBossTS, it's not strictly necessary). As a result, you should have something that looks like:

Now before we take a look at the application code, it's important to remember that we're trying to add transactions to these applications. I won't reiterate all of what this means (I'll leave it as an exercise to the reader to either look through the other articles in this blog or check out transactional resources elsewhere). However, for the purposes of this entry it's important to remember/realise that a transaction needs a durable location where it can store the log to be used in the event of a failure and recovery. In the case of Android, we'll assume this is on an SD card somewhere. In terms of the application and the Android emulator, this has an impact. First, for the emulator we need to create an Android Virtual Device (AVD) and ensure it has an emulated SD card. You can do this easily using the AVD creator:

As you can see, in this case we've selected 512 Meg for the storage. That should be plenty of room for quite a few transactions, especially as most of the time the logs will be created and almost immediately deleted. So you don't have to go overboard with the amount of storage allocated.

Now for the application we need to ensure that it is allowed to access the SD card (external storage). We do that by editing the AndroidManifest.xml:

In this configuration we also need to allow the application to access the internet. This is because JBossTS needs to access the IP address of the device in order to create unique transaction identifiers. We'll look at whether this is something we can change later, but it shouldn't be an issue at this stage.

We're now ready to look at the application. If you've read this blog for a while then you'll know about ArjunaCore and the transactional toolkit within. Not only is it the basis for the work we're doing on STM, but we're using it here to create our application. I won't go into details, because the transactional object is your standard transactional int AtomicObject derived from LockManager and providing operations to read and write the state. It uses nested transactions as well!

So how do we use instances of this transactional int?

Pretty simple. As you can see above, we simply create a new transaction via AtomicAction; create an instance of the AtomicObject; start the transaction and modify the state of the object before committing the transaction. If all goes according to plan, when you deploy this application and run it, you should first see something like:

And then see output similar to below (your transaction id will be different):

That's it! Fairly simple and straightforward. At some point in the hopefully not-too-distant future, I'll get the JBossTS code changes into the mainline code in github and look at a more complex application. This one doesn't cover recovery for instance, and I'd like to more tests. But for now it shows what's possible and the direction in which we're heading. Enjoy!

Transactional android applications

OK, so it's taken me longer than I expected, but I finally found some time to commit the transactional Android code that I mentioned back in January. Thanks to my JBossWorld keynote preparation, a lot of EAP 6 work and too many meetings to count, I still haven't had enough time to actually commit these changes to the trunk of JBossTS (Narayana). However, rather than continue to delay things, I've decided to check in my edited source to the JBossTS svn repository along with a quick example.

I'll post another article soon to walkthrough the example and its configuration, but if you're interested in checking out the JBossTS source then it's available via svn at https://svn.jboss.org/repos/labs/labs/jbosstm/workspace/mlittle/android and the example at https://svn.jboss.org/repos/labs/labs/jbosstm/workspace/mlittle/TxAndroid. Of course you can build the JBossTS source if you want, but for the sakes of convenience I've included the necessary jars in the example directory.

Running a WS-AtomicTransaction Enabled Web Service on Openshift

We recently published a video showing you how to deploy a WS-AT enabled Web service on Openshift. The focus of this video is to provide a very simple example that introduces the WS-AT technology and the basics of deploying it to Openshift. The video is based on the wsat-simple quickstart  which ships with the JBossAS7 quickstart project. This quickstart and the accompanying video are a great place to start when learning WS-AT. This video also shows you how to enable Web services and WS-AT in Openshift.



In order to keep this quickstart simple, I have focused on demonstrating how the WS-AT protocol works and the Java API to it. As a result, I needed to omit several features that you would expect in a real application. The XTS demonstrator provides a much fuller example of how to use WS-AT. Although more complete, it has a steeper learning curve for those new to WS-AT.

The limitations are as follows:

1. No backend resource. In a real application, it is likely that your Web service will be manipulating some transactional state. For example, it could be updating a database or sending a JMS message. This quickstart uses a mock resource that is neither transactional nor persistent. For an example of how to write and interface with transactional resources, you can take a look at the XTS demonstrator.

2. No WS-AT to JTA bridging. Using the JBossTS TXBridge technology, WS-AT transactions can be seamlessly bridged onto backend JTA transactions (and vice versa). This allows you to use WS-AT as in integration technology for creating a distributed transaction. WS-AT has proven interoperability with many vendors and in some situations it is the only way to create a distributed transaction in a heterogeneous vendor environment. WS-AT also uses SOAP over HTTP as the transport, which can overcome some integration problems that, say a binary protocol, cannot. The TXBridge is currently a technology preview and will not be supported in JBoss EAP 6.0. Therefore it could not be used in this quickstart. The TXBridge does ship with both JBossAS 7.x and EAP 6.0 so you can try it out. For an example, please see the TXBridge quickstart.

3. No recovery. The quickstart does not implement the required hooks to support crash recovery. This is a complex subject and one that is covered by the XTS demonstrator.

4. Multiple Services. WS-AT is a consensus protocol which implies that it is designed to support multiple parties. This quickstart only uses a single service, but could quite easily be modified to use multiple. Again the XTS demonstrator provides an example of this in action.

Transactions and parallelism and actors, oh my!

I wanted to cross-post an article I just wrote on the benefits of transactions in a massively parallel environment (not just a distributed system), not just because some may find it interesting in itself, but also because it references some of the work we've been doing in the project over the past decades. I'm hoping that as we do more work on STM and mobile transactions, to name but two areas of work, that we'll bring some of the techniques that were written about 20+ years ago to a new audience and maybe out of the realm of research. We have got a great transaction system and team here that is second to none. I'm convinced that just as Arjuna pushed the boundaries of standards and adoption of transactions for the past decades, so too can JBossTS/Narayana into the future.

Update to STM API

A few weeks ago we saw details of the new optimistic concurrency control (MVCC) for the STM implementation. At the time I mentioned that the syntax for using pessimistic and optimistic was different and also slightly more verbose than it really needed to be. So there have been a few changes recently and although it's not quite the way I want it, it's worth discussing now to at least give you an idea of where we are going with things. Especially important if you're trying to develop something using the STM and the previous examples as a template, since although all of those original classes and methods continue to exist, they may have moved packages!

I'll assume that you can compare and contrast by looking at the original posting, so here's how you create and use objects now:

Container theContainer = new Container();

Sample1 obj1 = theContainer.create(new SampleImple(10));

Sample1 obj2 = theContainer.create(new SampleImple(10));

Sample1 obj3 = theContainer.clone(new SampleImple(), obj1);

Sample1 obj4 = theContainer.clone(new SampleImple(), obj2);

It no longer matters whether they are @Pessimistic or @Optimistic (the default) - the syntax remains the same. Here we have two objects, obj1 and obj2 and two clones (copies) of them, obj3 and obj4. They're all recoverable (state will not survive a machine crash), since that's the default behaviour for Container, but that can be changed via the constructor which allows for PERSISTENT or RECOVERABLE to be specified at creation time.

You can probably guess where further improvements are going to happen. For a start, the fact that the clone method takes a "blank" instance to copy the state into is necessary, but it would be better if it could be inferred. In addition, there are some state management aspects of the implementation that need tidying up because they currently escape through into the application. However, these are both fairly straightforward aspects to fix and the plan is to get them finalised soon. But for now, these changes should be sufficient to try things out. Enjoy!

The Future of XTS

Last September I took over the great work that Andrew Dinn has been doing leading the XTS (Web service transactions) project. Since then I've been having a think about where we should be taking the project over the next few years. In this post I want to present our ideas to the community. Hopefully this will give you an idea of where we are going with XTS and give you an opportunity to provide feedback. I'll also refer to work planned for the Narayana project (AKA JBossTS), where it also applies to XTS.

About Me
First of all I'll take this opportunity to introduce myself. I've worked in the field of transactions and Java middleware for over 10 years. I started out in 2001 as a software engineer on the XTS project, recently after it was founded by Hewlette Packard. Here I helped to develop the industry's first Web service transaction implementation. In those days we were implementing the BTP standard. Since then I headed up the consultancy arm of Arjuna Technologies, helping high-profile customers deploy JBoss Transactions, JBossWS and also Arjuna's Cloud Computing product. In late summer of 2011 I made the jump to Red Hat to lead the XTS project. I have also received a PhD in middleware and non-repudiation from the University of Newcastle upon Tyne, where I now hold the role of Visiting Research Fellow. In this role I teach JEE and distributed transactions as part of a masters course.

After reviewing where the XTS project currently stands we've had a good think about what areas of improvement we should be focusing on over the coming months and years. We've found that these improvements tend to fall into one of four 'themes'; usability, modernization, streamlining and consistency.

Usability
Improving usability is our number one priority. In particular we will be introducing quickstarts (JBTM-870) and making significant improvements to the XTS java API.

Currently we have a fantastic demonstrator that shows the developer how to use, pretty much, every feature of XTS. The problem is that it is not an easy place to start for those new to XTS. Recently we added a number of basic quickstarts to the JBossAS 7 quickstart project. These quickstarts are designed as a place to start when beginning to use XTS. We also have plans to provide additional quickstarts as part of the Narayana quickstart project (JBTM-870). These quickstarts will focus on more advanced features, such as recovery. These quickstarts will adhere to the same strict guidelines imposed by the JBossAS team. This will bring a high level of quality as well as familiarity. We also intend to provide unit tests with every quickstart; and those in the Narayana quickstart project will undergo continuous integration on our Hudson servers (JBTM-1068). This should ensure our quickstarts always work in the intended environment. We have an idea of what quickstarts we should initially provide, but we feel it would be best to be community driven in the future. So if you have an idea for a quickstart, or maybe you have one to contribute, then please get in touch on the forum.

The biggest changes around usability are coming as part of our new sub-project; currently named the 'TXFramework'. The TXFramework is a new and novel framework for writing portable ACID and NoACID transactional applications. It provides an abstract annotation based API for specifying transactional requirements. This decouples the application logic from the actual transaction model used (WS-AT, REST_AT, JTA, etc). It also removes a lot of boilerplate code that's currently needed by WS-AT and WS-BA services. The TXFramework also provides seamless bridging between different transaction protocols. This subject could cover many blog postings, so I won't discuss it in any more detail here. Stay tuned for more posts on this subject in the coming months. I'll also be presenting the TXFramework at JUDCon2012:Boston on the 25th June. Hope to see you there!

Modernisation
The next focus is around modernization. There's been a lot of exciting new technology emerging over the last few years and we are making many improvements to take advantage of it. In particular, Arquillian and Git.

The XTS functional tests need to run in a container. Before JBoss developed the Arquillian project, we were forced to create an in-container test-runner to run our functional tests. By migrating to Arquillian we are now able to remove this test-runner code and take advantage of additional benefits that Arquillian brings. In particular we can run any/all of the XTS tests from with the IDE as regular JUnit tests. Arquillian also manages the setup and teardown of the test environment, making it a lot easier to run the tests. This improvement is going to remove a lot of code from the XTS codebase and bring great simplification to the way we write and run tests. JBTM-954, JBTM-955, JBTM-956

Narayana and Blacktie were recently migrated from SVN to Git and Github (See here). Git has been gaining a lot of traction recently in the world of open source and most JBoss projects have now made the leap. After reviewing the benefits of moving, we decided now was the time to make the jump. In particular it was the improvements to collaboration provided by Git and GitHub that made us feel that this was the right thing to do. If you are a contributor, you're going to find it a lot easier to get involved with XTS and Narayana.

We've also moved the entire Narayana project to maven, which is making it a lot easier for our contributors to get up and running quickly and is making it easier for us to maintain our build system. We're also beginning to use JaCoCo to check our test coverage JBTM-952. This ensures that XTS is fully tested and helps find redundant code.


Streamlining
We are doing some streamlining too to the XTS code base. We are taking two approaches here; as mentioned above we are using JaCoCo to help find redundant code. This should cut down the amount of code we need to maintain and help contributors get up to speed more quickly as they don't need to sift through unused code (JBTM-741). The second approach we are taking is to replace code with third party libraries where we can be sure that they will provide an equal or better implementation. For example, as stated above we are in the process of replacing our test-harness code with Arquillian (JBTM-953).


Consistency
Being productive and helping our contributors be productive is very important to us. Having to switch between and understand different practices when working with our projects does not help with this goal. Therefore, we are aiming to bring as much consistency as we can between XTS and Narayana and to some extent between Narayana and the other JBoss projects. A move to improved consistency also brings with it an opportunity to re-use ideas that have been proven elsewhere. In particular, our quickstarts are now in the process of being moved to a single repository and follow the strict guidlines set out by the JBossAS team. The XTS codebase is being re-structured to follow the default maven layout (JBTM-968). This should make it easier for new contributors to follow our layout.

Getting Involved
Of course, we would love the community to get involved with XTS and the Narayana project. I mentioned many pieces of work in this post; you should follow the Jira issue links and vote up the ones that matter to you. If you have an idea for something you would like to be included, please start a discussion on the Narayana forum. You can also fork our GitHub projects if you'd like to get involved with the coding side.

Transactions and NoSQL - not reinventing the wheel!

It's nice to see some NoSQL solutions recognising that they need transactions (or extended transactions) and rediscovering the work that the transactions community (ourselves included) have been working on for a few decades! This presentation is interesting, but frustrating in a few places, e.g., the discussion around the viability of nested transactions for long duration interactions, in general the discussion around Sagas, and the Q&A at the end where the glossing over of technical aspects during the talk really lead to confusion.

However, that aside, it's not a bad overview of the needs for long duration, compensation based transactions and references a few of the important papers and research in this space going back several decades. Of course if you've been reading this blog long enough or watching the work we've been doing for a while, you'll know about others, including:

• the compensation REST transactions work;
• a history of extended transactions;
• a comparison of two of the original standards efforts in the area of compensation transactions;
• a couple of presentations on early standards effort around Web Services and transactions;
• an overview of the wider expanse of work that has been done around extended transactions, workflow, messaging, choreography etc.

The reference to the Sagas pattern that is mentioned, but forgotten, in the presentation I mentioned originally can be found here. It's nice to see that at least in the case of this NoSQL implementation, the engineers are willing to see what's gone before and learn from it. And if you want to look at implementations (and frameworks) that do much of this already, then take a look at JBossTS - without you having to write your own do/undo log!