JBossTS team blog

Transactional android applications

2012-05-27T22:43:00.000+01:00

OK, so it's taken me longer than I expected, but I finally found some time to commit the transactional Android code that I mentioned back in January. Thanks to my JBossWorld keynote preparation, a lot of EAP 6 work and too many meetings to count, I still haven't had enough time to actually commit these changes to the trunk of JBossTS (Narayana). However, rather than continue to delay things, I've decided to check in my edited source to the JBossTS svn repository along with a quick example.

I'll post another article soon to walkthrough the example and its configuration, but if you're interested in checking out the JBossTS source then it's available via svn at https://svn.jboss.org/repos/labs/labs/jbosstm/workspace/mlittle/android and the example at https://svn.jboss.org/repos/labs/labs/jbosstm/workspace/mlittle/TxAndroid. Of course you can build the JBossTS source if you want, but for the sakes of convenience I've included the necessary jars in the example directory.

Running a WS-AtomicTransaction Enabled Web Service on Openshift

2012-05-18T11:00:00.000+01:00

We recently published a video showing you how to deploy a WS-AT enabled Web service on Openshift. The focus of this video is to provide a very simple example that introduces the WS-AT technology and the basics of deploying it to Openshift. The video is based on the wsat-simple quickstart which ships with the JBossAS7 quickstart project. This quickstart and the accompanying video are a great place to start when learning WS-AT. This video also shows you how to enable Web services and WS-AT in Openshift.

In order to keep this quickstart simple, I have focused on demonstrating how the WS-AT protocol works and the Java API to it. As a result, I needed to omit several features that you would expect in a real application. The XTS demonstrator provides a much fuller example of how to use WS-AT. Although more complete, it has a steeper learning curve for those new to WS-AT.

The limitations are as follows:

1. No backend resource. In a real application, it is likely that your Web service will be manipulating some transactional state. For example, it could be updating a database or sending a JMS message. This quickstart uses a mock resource that is neither transactional nor persistent. For an example of how to write and interface with transactional resources, you can take a look at the XTS demonstrator.

2. No WS-AT to JTA bridging. Using the JBossTS TXBridge technology, WS-AT transactions can be seamlessly bridged onto backend JTA transactions (and vice versa). This allows you to use WS-AT as in integration technology for creating a distributed transaction. WS-AT has proven interoperability with many vendors and in some situations it is the only way to create a distributed transaction in a heterogeneous vendor environment. WS-AT also uses SOAP over HTTP as the transport, which can overcome some integration problems that, say a binary protocol, cannot. The TXBridge is currently a technology preview and will not be supported in JBoss EAP 6.0. Therefore it could not be used in this quickstart. The TXBridge does ship with both JBossAS 7.x and EAP 6.0 so you can try it out. For an example, please see the TXBridge quickstart.

3. No recovery. The quickstart does not implement the required hooks to support crash recovery. This is a complex subject and one that is covered by the XTS demonstrator.

4. Multiple Services. WS-AT is a consensus protocol which implies that it is designed to support multiple parties. This quickstart only uses a single service, but could quite easily be modified to use multiple. Again the XTS demonstrator provides an example of this in action.

Transactions and parallelism and actors, oh my!

2012-04-06T21:56:00.002+01:00

I wanted to cross-post an article I just wrote on the benefits of transactions in a massively parallel environment (not just a distributed system), not just because some may find it interesting in itself, but also because it references some of the work we've been doing in the project over the past decades. I'm hoping that as we do more work on STM and mobile transactions, to name but two areas of work, that we'll bring some of the techniques that were written about 20+ years ago to a new audience and maybe out of the realm of research. We have got a great transaction system and team here that is second to none. I'm convinced that just as Arjuna pushed the boundaries of standards and adoption of transactions for the past decades, so too can JBossTS/Narayana into the future.

Update to STM API

2012-03-24T21:37:00.003Z

A few weeks ago we saw details of the new optimistic concurrency control (MVCC) for the STM implementation. At the time I mentioned that the syntax for using pessimistic and optimistic was different and also slightly more verbose than it really needed to be. So there have been a few changes recently and although it's not quite the way I want it, it's worth discussing now to at least give you an idea of where we are going with things. Especially important if you're trying to develop something using the STM and the previous examples as a template, since although all of those original classes and methods continue to exist, they may have moved packages!

I'll assume that you can compare and contrast by looking at the original posting, so here's how you create and use objects now:

Container theContainer = new Container();

Sample1 obj1 = theContainer.create(new SampleImple(10));

Sample1 obj2 = theContainer.create(new SampleImple(10));

Sample1 obj3 = theContainer.clone(new SampleImple(), obj1);

Sample1 obj4 = theContainer.clone(new SampleImple(), obj2);

It no longer matters whether they are @Pessimistic or @Optimistic (the default) - the syntax remains the same. Here we have two objects, obj1 and obj2 and two clones (copies) of them, obj3 and obj4. They're all recoverable (state will not survive a machine crash), since that's the default behaviour for Container, but that can be changed via the constructor which allows for PERSISTENT or RECOVERABLE to be specified at creation time.

You can probably guess where further improvements are going to happen. For a start, the fact that the clone method takes a "blank" instance to copy the state into is necessary, but it would be better if it could be inferred. In addition, there are some state management aspects of the implementation that need tidying up because they currently escape through into the application. However, these are both fairly straightforward aspects to fix and the plan is to get them finalised soon. But for now, these changes should be sufficient to try things out. Enjoy!

The Future of XTS

2012-03-16T14:22:00.004Z

Last September I took over the great work that Andrew Dinn has been doing leading the XTS (Web service transactions) project. Since then I've been having a think about where we should be taking the project over the next few years. In this post I want to present our ideas to the community. Hopefully this will give you an idea of where we are going with XTS and give you an opportunity to provide feedback. I'll also refer to work planned for the Narayana project (AKA JBossTS), where it also applies to XTS.

About Me
First of all I'll take this opportunity to introduce myself. I've worked in the field of transactions and Java middleware for over 10 years. I started out in 2001 as a software engineer on the XTS project, recently after it was founded by Hewlette Packard. Here I helped to develop the industry's first Web service transaction implementation. In those days we were implementing the BTP standard. Since then I headed up the consultancy arm of Arjuna Technologies, helping high-profile customers deploy JBoss Transactions, JBossWS and also Arjuna's Cloud Computing product. In late summer of 2011 I made the jump to Red Hat to lead the XTS project. I have also received a PhD in middleware and non-repudiation from the University of Newcastle upon Tyne, where I now hold the role of Visiting Research Fellow. In this role I teach JEE and distributed transactions as part of a masters course.

After reviewing where the XTS project currently stands we've had a good think about what areas of improvement we should be focusing on over the coming months and years. We've found that these improvements tend to fall into one of four 'themes'; usability, modernization, streamlining and consistency.

Usability
Improving usability is our number one priority. In particular we will be introducing quickstarts (JBTM-870) and making significant improvements to the XTS java API.

Currently we have a fantastic demonstrator that shows the developer how to use, pretty much, every feature of XTS. The problem is that it is not an easy place to start for those new to XTS. Recently we added a number of basic quickstarts to the JBossAS 7 quickstart project. These quickstarts are designed as a place to start when beginning to use XTS. We also have plans to provide additional quickstarts as part of the Narayana quickstart project (JBTM-870). These quickstarts will focus on more advanced features, such as recovery. These quickstarts will adhere to the same strict guidelines imposed by the JBossAS team. This will bring a high level of quality as well as familiarity. We also intend to provide unit tests with every quickstart; and those in the Narayana quickstart project will undergo continuous integration on our Hudson servers (JBTM-1068). This should ensure our quickstarts always work in the intended environment. We have an idea of what quickstarts we should initially provide, but we feel it would be best to be community driven in the future. So if you have an idea for a quickstart, or maybe you have one to contribute, then please get in touch on the forum.

The biggest changes around usability are coming as part of our new sub-project; currently named the 'TXFramework'. The TXFramework is a new and novel framework for writing portable ACID and NoACID transactional applications. It provides an abstract annotation based API for specifying transactional requirements. This decouples the application logic from the actual transaction model used (WS-AT, REST_AT, JTA, etc). It also removes a lot of boilerplate code that's currently needed by WS-AT and WS-BA services. The TXFramework also provides seamless bridging between different transaction protocols. This subject could cover many blog postings, so I won't discuss it in any more detail here. Stay tuned for more posts on this subject in the coming months. I'll also be presenting the TXFramework at JUDCon2012:Boston on the 25th June. Hope to see you there!

Modernisation
The next focus is around modernization. There's been a lot of exciting new technology emerging over the last few years and we are making many improvements to take advantage of it. In particular, Arquillian and Git.

The XTS functional tests need to run in a container. Before JBoss developed the Arquillian project, we were forced to create an in-container test-runner to run our functional tests. By migrating to Arquillian we are now able to remove this test-runner code and take advantage of additional benefits that Arquillian brings. In particular we can run any/all of the XTS tests from with the IDE as regular JUnit tests. Arquillian also manages the setup and teardown of the test environment, making it a lot easier to run the tests. This improvement is going to remove a lot of code from the XTS codebase and bring great simplification to the way we write and run tests. JBTM-954, JBTM-955, JBTM-956

Narayana and Blacktie were recently migrated from SVN to Git and Github (See here). Git has been gaining a lot of traction recently in the world of open source and most JBoss projects have now made the leap. After reviewing the benefits of moving, we decided now was the time to make the jump. In particular it was the improvements to collaboration provided by Git and GitHub that made us feel that this was the right thing to do. If you are a contributor, you're going to find it a lot easier to get involved with XTS and Narayana.

We've also moved the entire Narayana project to maven, which is making it a lot easier for our contributors to get up and running quickly and is making it easier for us to maintain our build system. We're also beginning to use JaCoCo to check our test coverage JBTM-952. This ensures that XTS is fully tested and helps find redundant code.

Streamlining
We are doing some streamlining too to the XTS code base. We are taking two approaches here; as mentioned above we are using JaCoCo to help find redundant code. This should cut down the amount of code we need to maintain and help contributors get up to speed more quickly as they don't need to sift through unused code (JBTM-741). The second approach we are taking is to replace code with third party libraries where we can be sure that they will provide an equal or better implementation. For example, as stated above we are in the process of replacing our test-harness code with Arquillian (JBTM-953).

Consistency
Being productive and helping our contributors be productive is very important to us. Having to switch between and understand different practices when working with our projects does not help with this goal. Therefore, we are aiming to bring as much consistency as we can between XTS and Narayana and to some extent between Narayana and the other JBoss projects. A move to improved consistency also brings with it an opportunity to re-use ideas that have been proven elsewhere. In particular, our quickstarts are now in the process of being moved to a single repository and follow the strict guidlines set out by the JBossAS team. The XTS codebase is being re-structured to follow the default maven layout (JBTM-968). This should make it easier for new contributors to follow our layout.

Getting Involved
Of course, we would love the community to get involved with XTS and the Narayana project. I mentioned many pieces of work in this post; you should follow the Jira issue links and vote up the ones that matter to you. If you have an idea for something you would like to be included, please start a discussion on the Narayana forum. You can also fork our GitHub projects if you'd like to get involved with the coding side.

Transactions and NoSQL - not reinventing the wheel!

2012-03-11T17:12:00.004Z

It's nice to see some NoSQL solutions recognising that they need transactions (or extended transactions) and rediscovering the work that the transactions community (ourselves included) have been working on for a few decades! This presentation is interesting, but frustrating in a few places, e.g., the discussion around the viability of nested transactions for long duration interactions, in general the discussion around Sagas, and the Q&A at the end where the glossing over of technical aspects during the talk really lead to confusion.

However, that aside, it's not a bad overview of the needs for long duration, compensation based transactions and references a few of the important papers and research in this space going back several decades. Of course if you've been reading this blog long enough or watching the work we've been doing for a while, you'll know about others, including:

the compensation REST transactions work;
a history of extended transactions;
a comparison of two of the original standards efforts in the area of compensation transactions;
a couple of presentations on early standards effort around Web Services and transactions;
an overview of the wider expanse of work that has been done around extended transactions, workflow, messaging, choreography etc.

The reference to the Sagas pattern that is mentioned, but forgotten, in the presentation I mentioned originally can be found here. It's nice to see that at least in the case of this NoSQL implementation, the engineers are willing to see what's gone before and learn from it. And if you want to look at implementations (and frameworks) that do much of this already, then take a look at JBossTS - without you having to write your own do/undo log!

Optimistic STM

2012-02-19T16:49:00.003Z

It's been a while since I last posted anything on the Software Transactional Memory work that's been going on in JBossTS (aka Narayana). At that time we had everything in place for you to write transactional applications that have all of the ACID properties or, more common for STM, without the D. Per object concurrency control is done through locks and type specific concurrency control is available. You can define locks on a per object and per method basis, and combined with nested transactions this provides for a flexible way of structuring applications that would typically not block threads unless there is really high contention:

@Transactional

public class SampleLockable implements Sample

{

public SampleLockable (int init)

{

_isState = init;

}

@ReadLock

public int value ()

{

return _isState;

}

@WriteLock

public void increment ()

{

_isState++;

}

@WriteLock

public void decrement ()

{

_isState--;

}

@State

private int _isState;

}

If you recall from previous articles, all but the @Transactional annotation are optional, with sensible defaults taken for everything else including locks and state. And you create and use instances fairly simply:

RecoverableContainer theContainer = new RecoverableContainer();

Sample obj1 = theContainer.enlist(new SampleLockable(10));

However, the locking strategy we had originally was pessimistic. As I mentioned separately:

"Most transaction systems utilize what is commonly referred to as pessimistic concurrency control mechanisms: in essence, whenever a data structure or other transactional resource is accessed, a lock is obtained on it as described earlier. This lock will remain held on that resource for the duration of the transaction and the benefit of this is that other users will not be able to modify (and possibly not even observe) the resource until the holding transaction has terminated. There are a number of disadvantages of this style: (i) the overhead of acquiring and maintaining concurrency control information in an environment where conflict or data sharing is not high, (ii) deadlocks may occur, where one user waits for another to release a lock not realizing that that user is waiting for the release of a lock held by the first."

And the obvious alternative to this approach is optimistic:

"Therefore, optimistic concurrency control assumes that conflicts are not high and tries to ensure locks are held only for brief periods of time: essentially locks are only acquired at the end of the transaction when it is about to terminate. This kind of concurrency control requires a means to detect if an update to a resource does conflict with any updates that may have occurred in the interim and how to recover from such conflicts. Typically detection will happen using timestamps, whereby the system takes a snapshot of the timestamps associated with resources it is about to use or modify and compares them with the timestamps available when the transaction commits."

Therefore, you can probably guess what's just been added to our STM implementation? Yes, that's right ... optimistic concurrency control! In line with the annotation based approach we've used so far, there are now two new annotations: @Optimistic and @Pessimistic, with Pessimistic being the default. These are defined on a per interface basis and define the type of concurrency control implementation that is used whenever locks are needed:

@Transactional

@Optimistic

public class SampleLockable implements Sample

{

public SampleLockable (int init)

{

_isState = init;

}

@ReadLock

public int value ()

{

return _isState;

}

@WriteLock

public void increment ()

{

_isState++;

}

@WriteLock

public void decrement ()

{

_isState--;

}

@State

private int _isState;

}

And that's it. No other changes are needed to the interface or to the implementation. However, at present there is a subtle change in the way in which you create your objects. Recall how that was done previously and then compare it with the style necessary when using optimistic concurrency control:

RecoverableContainer theContainer = new RecoverableContainer();

Sample obj1 = theContainer.enlist(new SampleLockable(10));

Sample obj2 = theContainer.enlist(new SampleLockable(10),

theContainer.getUidForHandle(obj1));

In the original pessimistic approach the instance obj1 can be shared between any number of threads and the STM implementation, along with JBossTS, will ensure that the state is manipulated consistently and safely. However, with optimistic concurrency we need to have one instance of the state per thread. So in the above code we first create the object (obj1) and then we create a copy of it (obj2), passing a reference to the original to the container.

It is likely that the API for this will change soon in order to unify optimistic and pessimistic: the aim is to make it as opaque as possible to the application so you only have to modify an annotation when you want to change the implementation. One other thing that we're considering is changing the implementation dynamically, perhaps based on monitored metrics for contention. But for now that's it and it's all available in the Narayana git repository.

connecting the dots

2012-01-12T10:36:00.002Z

Hmm, looks like it's 2012 already, which means we're overdue for another rant. Today's lecture topic is the perils of trying to replace or do without bits of code that you don't understand.

Programming in the large is all about abstraction, loose coupling and separation of concerns. Java EE promotes the ability to use complex functionality without understanding how it is implemented. As long as you follow the instructions this works surprisingly well. With an app server at your disposal you require only the sketchiest understanding of distributed transactions to write code that consumes a message and updates a database in an ACID fashion. Which on the whole is a good thing, since not all that many people actually want to understand all the plumbing that makes it work. There is a snag though: this apparent simplicity can cause some users to think the Java EE container is not doing all that much work and can easily be replaced or done without. Likewise it makes it difficult to tell the difference between a full fledged container that will support the functionality you need and a cut down framework or lightweight container that may not.

Let's debunk a few myths...

Spring is not a JTA.

It's an abstraction layer on top of a transaction manager, not an implementation of one. For transactions involving only a single resource, it simply delegates the hard bits to that resource manager - so called 'native transactions'. Simple, fast and mostly adequate if you need only a single resource. You don't get transaction lifecycle events, but you're probably using an ORM that provides the equivalent of the beforeCompletion hook for its cache anyhow.

For transactions with more than one resource, you need to wire in a real transaction manager to Spring. You can do this with bitronix, atomikos or JBossTS, usually just by specifying the right TransactionManager and UserTransaction implementation bean classes. But your problems don't end there, because...

Spring is not a JCA.

The roles, responsibilities and relationship between the JTA and JCA components of an app server are critical considerations when you're trying to do without one. The JTA manages transaction lifecycle - begin/commit/rollback. Most importantly, it make appropriate calls on any enlisted XAResources as the transaction progresses. But here is the bit that most users don't pay attention to: A JTA does not magically know what resources you want to participate in the transaction. Telling it that is the JCA's job.

In a full on app server, the JCA manages connections to resource managers such as databases and message queues. If you deploy those drivers/connectors in a manner that identifies them as XA enabled, the JCA ensures that they are correctly associated with the transaction. Application code simply e.g. looks up the JNDI name for a connection pool and calls getConnection(). The JCA intercepts the call, get the XAResource for the connection and passes it to the transaction manager.

In some cases you don't need a full JCA. You can often make do with a transaction manager aware XA connection pool, which is essentially a subset of the JCA functionality. But you can't get away with only an XA aware driver or a non-XA connection pool. Trying to do that leads to some interesting behaviour: your app will deploy and run, but you have a transaction and a connection that know nothing about one another. Committing or rolling back the transaction won't commit or rollback the work in the database. oops.

So, you also need to wire in a JCA or suitable connection pooling implementation. Most 'standalone' JTA implementations ship with a simple connection management solution that is suitable for light use. The one in JBossTS is called the TransactionalDriver. For serious deployments you want IronJacamar or some other JCA that has robust and fast connection management.

So now you have wired up your JTA and JCA in Spring, but you are still not done because...

The standard contract between JTA and JCA does not include recovery management setup. Wiring up resources for crash recovery requires a proprietary solution that differs for each transaction manager. The connection manager that ships with the transaction manager may do this more or less automatically, but third party JCAs or XA aware connection pools probably won't. So, go read the transaction manager documentation and write a few test cases.

phew, that was a lot of work, wasn't it? Spring is good at what it does, but its not an out of box replacement for all the transactional plumbing in a Java EE app server. Nor is tomcat - you'll have much the same steps to perform there. In both cases it is possible, but not as simple as unzipping and starting JBossAS. Do you really want to make your life harder than it has to be?

Transactional Android coming soon!

2012-01-01T18:14:00.003Z

I spent some time this Christmas porting JBossTS to run on Android. It's pretty much done, with the exception of a few workarounds that I need to fix properly over the next few weeks, when I find the time. But once I check this code into the repository, you'll be able to write your own transactional Android applications. Unfortunately I can't guarantee which version this will be in at the moment, but if it is going to take me too long to do the merges then I may create a branch in svn that interested people can just pull from directly, with the usual caveats. If time allows, I may say something about this at JUDCon India too!

Transactions making a comeback? They were never away!

2011-12-23T19:06:00.003Z

Over the years that I've been involved with transaction processing theory and practice (way too many years for me to admit!) I've seen transactions used, abused and ignored in a wide variety of applications and situations. I've discussed many of these scars in articles here and elsewhere for decades (ouch! that pains me to even use such a timeframe!) But every new software wave, whether it's Web Services, object-orientation, and even the Web itself, has come to the inevitable conclusion that transactions are needed in some way and in one form or another. Maybe not ACID transactions; maybe it's compensation based, or some other variation on the extended transaction theme.

So it is nice to see that being repeated in the past couple of years with Cloud and NoSQL. If you read this blog enough you'll have heard from myself and others on the fact that although some believe that you need to ditch transactions in order to achieve scalability, others aren't quite convinced that it is always necessary to do so - and I count us amongst the latter group. Complete reliance on transactions is sometimes too much. However, completely ignoring them and pushing consistency and recovery up to the application is sometimes too much as well. This is something that appears to be dawning on a number of transaction-less implementations and hopefully it's a trend that will continue.

Anyone who knows me or has followed things I've written or presented over the years will also know that I think transactions are a great structuring technique for concurrent programming. Of course in distributed systems we tend to take this for granted: you've inherently got multiple clients/users manipulating your data, typically at the same time, so transactions with their ACID properties allow developers to concentrate on the functional aspects of the application or service, whilst letting the transaction system do the hard work on ensuring isolation and consistency in the presence of these concurrent users (and possibly failures).

But distribution isn't the only place where you get concurrency, and particularly these days with multi-core processors. Back in the last century (ouch!) some of us in academia and industry, had access to multi-processor machines which weren't as common as you might expect (they were expensive!) But when you had them it quickly became apparent that transactions could help with developing applications that had no distribution in them but were (massively) parallel. From this early work techniques such as software transactional memory were born. Back then it was more of an edge case and people found it hard to understand why you'd need transactions without distribution or even a database involved. Well obviously the advances in hardware have silenced most of those critics and we're seeing more and more vendors, open source projects, languages etc. looking at transactions are a fundamental component and not just an add-on for some scenarios.

So what does all of this mean? Well first of all I think it's great to see transactions continuing to have an impact in these new waves. Fundamental requirements like fault tolerance, concurrency control, security etc. are fundamental for a reason. Secondly I think it's fair to say that as with previous waves, we'll see transaction theory and implementations adapt and change to better address some of the new concerns and requirements that are bound to arise. I'm excited by all of this, as I am whenever there's something new to apply transactions. I'm also excited by the fact that JBossTS (implementation and team) is well placed to help drive some of this as we've done for ... well ... let's just say for quite a long time and leave it at that!

my last commit

2011-12-19T14:00:00.002Z

Today I created the tag for the JBossTS 4.16.0.Final release, thereby making my last commit to the JBossTS repository as team lead.

It is the end of an era in more ways than one, as 4.16 is planned to be the last feature release for the 4.x line and the last for the JBossTS brand.

Starting in the new year, the project gets a new name, Narayana, a new major version, 5.0, and a new development team lead, Tom Jenkinson. So, exciting times ahead for the project in 2012 and beyond, with fresh blood and fresh ideas.

As for myself, I start 2012 with a shiny new JBoss role looking at Big Data and noSQL. Who knows, there may even be a new blog for you to subscribe to. But first there is the small matter of a seasonal vacation...

Merry Christmas

Jonathan.

HPTS 2011

2011-11-11T15:53:00.002Z

I've uploaded most of the presentations and posters sessions for HPTS 2011 to the website now. If you interested in transactions, NoSQL, eventual consistency, data provenance and a whole raft of topical subjects, then you really should check them out!

unnecessary

2011-11-10T13:44:00.003Z

Hot on the heals of thinking about information quality, I came across this little gem regarding Spring and JPA configuration:

"Using multiple session factories in a logical transaction imposes challenging issues concerning transaction management... We quickly abandoned [JTA transaction management] due to the potential cost and complexity of an XA protocol with two-phase commit. Since most of the modules of an application share the same data source, the imposed cost and complexity are definitively unnecessary."

The authors then go on to describe how they built a custom solution that causes the same database connection to be used by the modules, removing the need for transaction coordination across multiple connections.

"Extending Spring’s transaction management with SessionFactory swapping and the Shared Transaction Resource pattern was a very challenging task."

That last bit should probably have read "challenging and unnecessary task".

A good JCA will automatically track connections enlisted with a JTA transaction and will reuse the already enlisted connection to satisfy a further dataSource.getConnection() request. Further, even if it does enlist multiple connections, the JTA will use isSameRM to detect that they relate to the same resource manager and thus still maintain the one phase commit optimisation. All of these challenging tasks are taken care of for you by the application server.

You probably should not bother to invent a better mousetrap until you've determined that current mousetraps don't catch your mice. The imposed cost and complexity are definitively unnecessary.

musings on peer review

2011-11-10T12:36:00.004Z

I've been reading up in a new field of study recently and as a result have been thinking about information provenance, reputation and collaboration.

Once upon a time, getting up to speed on a new topic in computing science required finding a good library and wading through a stack of conference proceedings and journals. Now it requires an online search and, critically, the ability to evaluate the quality of the massive quantity of material that comes back.

Formal peer review of publications is gone, unable to scale to online use. Meanwhile online systems for review, comment and reputation management are still in their infancy. The best we have right now is an ad-hoc stew of brands, social graphs and distributed conversations.

Those with enough time to invest can build a mental model of a new field that, after the initial investment in learning the landscape, allows them to maintain an ongoing overview of developments with relatively little effort. Those who's work only tangentially involves a deeply technical topic don't have this luxury. They typically perform searches not to learn about the new field in general, but to get specific solutions to a problem outside of their core field of expertise, after which they move on. Such users vastly outnumber the domain experts for niche topics like transactions.

What implications does this new model of communication and information dissemination have for the behaviour of professed experts in technical fields? Clearly our obligation to ensure that information we provide is accurate remains unchanged, and is in any case in our own best interest. Should we consider there to be an implied professional obligation to publish information only in a context that allows feedback to be attached directly to it e.g. blog with comments enabled? Or even allow collaborative editing e.g. wiki? How about taking time out to correct misinformation where we find it - is that now part of our social contract?

Question for the audience: Where do you get your information on transactions, and how do you assess its quality?

pen or pencil for writing logs?

2011-10-20T10:15:00.004+01:00

Those old enough to remember the days of manned space flight in the western world will recall that NASA expended considerable resources to come up with a pen that would work in space. Meanwhile, half a world away, the Russians just used a bunch of pencils.

I can't help feeling that story is going to have a new counterpart in the history books. Whilst HP are working on memristor based persistent RAM, someone else just grafted a bunch of flash and a super-capacitor onto a regular DIMM instead. Now I just need the linux guys to come up with a nice API...

New RAM shunts data into flash in power cuts

nested transactions 101

2011-10-18T17:52:00.004+01:00

You wait ages for an interesting technical problem, then you get the same one twice in as many weeks. If you are a programmer, you now write a script so that you don't have to do any manual work the third time you encounter the problem. If you are a good programmer, you just use the script you wrote the previous week.

When applied to technical support questions, this approach results in the incremental creation of a FAQ. My last such update was way back in April, on the topic of handling XA commit race conditions between db updates and JMS messages. Like I said, you wait ages for an interesting problem, but another has finally come along. So, this FAQ update is all about nested transactions.

Mark has posted about nested transaction before, back in 2010 and 2009. They have of course been around even longer than that and JBossTS/ArjunaTS has support for them that actually pre-dates Java - it was ported over from C++. So you'd think people would have gotten the hang of it by now, but not so. Nested transactions are still barely understood and even less used.

Let's deal with the understanding bit first. Many people use the term 'nested transactions' to mean different things. A true nested transaction is used mainly for fault isolation of specific tasks within a wider transaction.


tm.begin();
doStuffWithOuterTransaction();
tm.begin();
try {
  doStuffWithInnerTransaction();
  tm.commit();
} catch(Exception e) {
  handleFailureOfInnerTransaction();
}
doMoreStuffWithOuterTransaction();
tm.commit();

This construct is useful where we have some alternative way to achieve the work done by the inner transaction and can call it from the exception handler. Let's try a concrete example:


tm.begin(); // outer tx
bookTheatreTickets();
tm.begin(); // inner tx
try {
  bookWithFavoriteTaxiCompany();
  tm.commit(); // inner tx
} catch(Exception e) {
  tm.begin(); // inner tx
  bookWithRivalTaxiFirm();
  tm.commit(); // inner tx
}
bookRestaurantTable();
tm.commit(); // outer tx

So, when everything goes smoothly you have behaviour equivalent to a normal flat transaction. But when there is minor trouble in a non essential part of the process, you can shrug it off and make forward progress without having to start over and risk losing your precious theatre seats.

As it turns out there are a number of reasons this a not widely used.

Firstly, it's not all that common to have a viable alternative method available for the inner update in system level transactions. It's more common for business process type long running transactions, where ACID is frequently less attractive than an extended tx model such as WS-BA anyhow. What about the case where you have no alternative method, don't care if the inner tx fails, but must not commit its work unless the outer transaction succeeds? That's what afterCompletion() is for.

Secondly, but often of greater practical importance, nested transactions are not supported by any of the widely deployed databases, message queuing products or other resource managers. That severely limits what you can do in the inner transaction. You're basically limited to using the TxOJ resource manager bundled with JBossTS, as described in Mark's posts. Give up any thought of updating your database conditionally - it just won't work. JDBC savepoints provide somewhat nested transaction like behaviour for non-XA situations, but they don't work in XA situations. Nor does the XA protocol, foundation of the interoperability between transaction managers and resource managers, provide any alternative. That said, it's theoretically possible to fudge things a bit. Let's look at that example again in XA terms.


tm.begin(); // outer tx
bookTheatreTickets(); // enlist db-A.
tm.begin(); // inner tx
try {
  bookWithFavoriteTaxiCompany(); // enlist db-B.
  tm.commit(); // inner tx - prepare db-B. Don't commit it though. Don't touch db-A.
} catch(Exception e) {
  // oh dear, the prepare on db-B failed. roll it back. Don't rollback db-A though.
  tm.begin(); // inner tx
  bookWithRivalTaxiFirm(); // enlist db-C
  tm.commit(); // inner tx - prepare db-C but don't commit it or touch db-A
}
bookRestaurantTable(); // enlist db-D
tm.commit(); // outer tx - prepare db-A and db-D. Commit db-A, db-C and db-D.

This essentially fakes a nested transaction by manipulating the list of resource managers in a single flat transaction - we cheated a bit by removing db-B part way through, so the tx is not true ACID across all the four participants, only three. JBossTS does not support this, because it's written by purists who think you should use an extended transaction model instead. Also, we don't want to deal with irate users whose database throughput has plummeted because of the length of time that locks are being held on db-B and db-C.

Fortunately, you may not actually need true nested transactions anyhow. There is another sort of nested transaction, properly known as nested top-level, which not only works with pretty much any environment, but is also handy for many common use cases.

The distinction is founded on the asymmetry of the relationship between the outer and inner transactions. For true nested transactions, failure of the inner tx need not impact the outcome of the outer tx, whilst failure of the outer tx will ensure the inner tx rolls back. For nested top-level, the situation is reversed: failure of the outer transaction won't undo the inner tx, but failure of the inner tx may prevent the outer one from committing. Sound familiar? The most widely deployed use case for nested top-level is ensuring that an audit log entry of the processing attempt is made, regardless of the outcome of the business activity.


tm.begin();
doUnauditedStuff();
writeAuditLogForProcessingAttempt();
doSecureBusinessSystemUpdate();
tm.commit();

The ACID properties of the flat tx don't achieve what we want here - the audit log entry must be created regardless of the success or failure of the business system update, whereas we have it being committed only if the business system update also commits. Let's try that again:


tm.begin(); // tx-A
doUnauditedStuff();
Transaction txA = tm.suspend();
tm.begin(); // new top level tx-B
try {
  writeAuditLogForProcessingAttempt();
  tm.commit(); //  tx-B
} catch(Exception e) {
  tm.resume(txA);
  tm.rollback(); // tx-A
  return;
}
tm.resume(txA);
doSecureBusinessSystemUpdate();
tm.commit(); // tx-A

Well, that's a little better - we'll not attempt the business logic processing unless we have first successfully written the audit log, so we're guaranteed to always have a log of any update that does take place. But there is a snag: the audit log will only show the attempt, not the success/failure outcome of it. What if that's not good enough? Let's steal a leaf from the transaction optimization handbook: presumed abort.


tm.begin(); // tx-A
doUnauditedStuff();
Transaction txA = tm.suspend();
tm.begin(); // new top level tx-B
try {
  writeAuditLogForProcessingAttempt("attempting update, assume it failed");
  tm.commit(); //  tx-B
} catch(Exception e) {
  tm.resume(txA);
  tm.rollback(); // tx-A
  return;
}
tm.resume(txA);
doSecureBusinessSystemUpdate();
writeAuditLogForProcessingAttempt("processing attempt completed successfully");
tm.commit(); // tx-A

So now we have an audit log will always show an entry and always show if it succeeded or not. Also, I'll hopefully never have to answer another nested transaction question from scratch. Success all round I'd say.

memristor based logs

2011-10-18T16:49:00.002+01:00

Long time readers will recall that I've been tinkering with shiny toys in the form of SSDs, trying to assess how changes in storage technology cause changes in the way transaction logging should be designed. SSDs are here now, getting cheaper all the time and therefore becoming more 'meh' by the minute. So, I need something even newer and shinier to drool over...

Enter memristors, arguably the coolest tech to emerge from HP since the last Total-e-Server release. Initially intended to complete with flash, memristor technology also has the longer term potential to give us persistent RAM. A server with a power-loss tolerant storage mechamism that runs at approximately main memory speed will fundamentally change the way we think about storage hierarchies, process state and fault tolerance.

Until now the on-chip cache hierarchy and off-chip RAM have both come under the heading of 'volatile', whilst disk has been considered persistent, give or take a bit of RAID controller caching.

Volatile storage is managed by the memory subsystem, with the cache hierarchy within that tier largely transparent to the O/S much less the apps. Yes, performance gurus will take issue with that - understanding cache coherency models is vital to getting the best out of multi-core chips and multi-socket servers. But by and large we don't control it directly - MESI is hardcoded in the CPU and we only influence it with simple primitives - memory fencing, thread to core pinning and such.

Persistent storage meanwhile is managed by the file system stack - the O/S block cache, disk drivers, RAID controllers, on-device and on-controller caches etc. As more of it is in software we have a little more control over the cache model, by O_DIRECT, fsync, firmware config tweaking and such. Most critically, we can divide the persistent storage into different pools with different properties. The best known example is the age old configuration suggestion for high performance transaction processing: put the log storage on a dedicated device.

So what will the programming model look like when we have hardware that offers persistent RAM, either for all the main memory or, more likely in the medium term, for some subset of it? Will the entire process state survive a power loss at all cache tiers from the on-CPU registers to the disk platters, or will we need fine grained cache control to say 'synchronously flush from volatile RAM to persistent RAM', much as we currently force a sync to disk? How will we resume execution after a power loss? Will we need to explicitly reattach to our persistent RAM and rebuild our transient data structures from its contents, or will the O/S magically handle it all for us? Do we explicitly serialize data moving between volatile and non-volatile RAM, as we currently do with RAM to disk transfers, or is it automatic as with cache to RAM movements? What does this mean for us in terms of new language constructs, libraries and design patterns?

Many, many interesting questions, the answers to which will dictate the programming landscape for a new generation of middleware and the applications that use it. The shift from HDD to SSD may seem minor in comparison. Will everything we know about the arcane niche of logging and crash recovery become obsolete, or become even more fundamental and mainstream? Job prospects aside, on this occasion I'm leaning rather in favour of obsolete.

Workshop on the Theory of Transactional Memory

2011-09-25T20:58:00.003+01:00

I'm just back from the WTTM in Rome. It was good to go to the workshop and listen to presentations from the likes of Maurice Herlihy on advances in both hardware and software transactional memory. I was there representing the Cloud-TM work that we're doing in collaboration with others and specifically around Infinispan and JBossTS.

One of my favourite presentations of the event was on the use of pessimistic concurrency control in STM. Most STM approaches use an optimistic approach, which obviously works well in cases where contention is limited. The problem with pessimistic is retaining locks for the duration even when updates are infrequent. However, as the presenter showed, with suitable a priori ordering, pessimistic can be more efficient even in low contention environments. This was good because we tend to use pessimistic concurrency control in JBossTS a lot, and as was shown in the presentation, it's also a lot easier for developers to understand and reason about.

Unfortunately I had to leave the workshop before the last session, so I didn't get to listen to Torvald who represents Red Hat on the transactional C++ working group. That's a shame, because I've been following this effort for several years and know that we may be seeing this in gcc sometime soon. And who knows, maybe EE8 will see some STM eventually.

RESTful Transactions in the Cloud: Now they're for real

2011-08-12T16:28:00.000+01:00

A few months back Mark posted on the subject of REST, transactions and the cloud. Well now you can test the validity of his claims for yourselves. RedHat recently announced their OpenShift initiative for deploying applications into the cloud and we have put together a demonstration to show how easy it is to perform end to end transactional requests whilst remaining RESTful/RESTlike throughout and using little more than a JAX-RS container, an HTTP client library and a transaction coordinator.

As our starting point we use an implementation of the draft specification for a RESTful interface to 2-Phase-Commit transactions which we refer to as REST-AT.

The demonstration starts out by showing local clients (written in various languages such as javascript and Ruby) and local (JAX-RS) web services interacting transactionally using HTTP for all communications. The transaction coordinator that implements REST-AT is written using JAX-RS and also runs locally.

The push to cloud based deployments demands that providers support piecemeal migration of applications into the cloud infrastructure. REST-AT together with OpenShift makes the realisation of this goal, at least for transactional services, a relatively painless exercise. The demonstration code and accompanying video shows how to migrate the transaction coordinator component leaving clients and services outside of the infrastructure. We achieve this by using the OpenShift JBoss AS7 cartridge and deploy REST-AT as a war into the application server that this cartridge provides. In a later post we will show how to migrate Java and Ruby services into the OpenShift infrastructure using the AS7 and Ruby cartridges, respectively.

To begin using OpenShift a good starting point would be to watch the various getting started videos (look for the Zero to Cloud in 30 minutes track or go directly to the demo video). To start using REST-AT check out the quickstarts and the sources for the demo.

Using JBossTS in OpenShift Express

2011-08-12T16:26:00.002+01:00

Hi,

I just wanted to let you know that you can use transactions in Red Hat's new OpenShift Express tool. Having had some hand's on time with the system I can safely say its one of the most exciting new projects I have seen in a while.

It makes deploy applications into the cloud so easy, plus as it supports deploying applications onto AS7 you can benefit from all the cool features that have been added there.

As AS7 features transactions (naturally!) I decided to kick the tyres of Openshift Express a little harder by not just using JPA, EJB, JTA and JSF, but also utilise one of the JBoss Transactions extension features - TXOJ. You can watch my video of this here, and the source code is available here.

Safe to say it was extremely simple to get this up and running, if you follow the video you can too!

For those who don't like videos though, here's a little cheat sheet I put together to get it running:

1. You need an openshift express domain, go to: http://openshift.redhat.com to get one
2. rhc-create-app -a txapp -t jbossas-7.0
3. cd txapp
4. svn export https://anonsvn.jboss.org/repos/labs/labs/jbosstm/trunk/ArjunaJTA/quickstarts/jee_transactional_app
5. mv jee_transactional_app/* .
6. rmdir jee_transactional_app
7. Add an openshift profile to your pom.xml:

   <profile>
    <id>openshift</id>
    <build>
       <plugins>
         <plugin>
           <artifactId>maven-war-plugin</artifactId>
           <configuration>
             <outputDirectory>deployments</outputDirectory>
             <warName>ROOT</warName>
           </configuration>
         </plugin>
       </plugins>
     </build>
   </profile>

8. git add src pom.xml
9. git commit -a -m “jee_transactional_app” && git push
10. Open up firefox and go to http://txapp-<YOUR_DOMAIN>.openshift.redhat.com/jee_transactional_app/

I hope you enjoy using Openshift Express and also can add a few Transactional Objects to your applications!

Tom

norecoveryxa

2011-07-22T13:36:00.000+01:00

"Infamy! Infamy! They've all got it in for me!"

Of all the error and warning messages in JBossTS, none is more infamous than norecoveryxa. Pretty much every JBossTS user has an error log containing this little gem. Usually more than once. A lot more. But not for much longer. It has incurred my wrath and its days are numbered. I've got it in for that annoying little beast.


[com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa]
Could not find new XAResource to use for recovering non-serializable XAResource

Transaction logs preserve information needed to complete interrupted transactions after a crash. Boiled down to pure essence, a transaction log is a list of the Xids involved in a transaction. An Xid is simply a bunch of bytes, beginning with a unique tx id and ending with a branch id that is (usually) different for each resource manager in the transaction. During recovery, these Xids are used to tell the resource managers to commit the in-doubt transactions. All of which is fine in theory, but an utter pain in practice.

The core problem is that an Xid is not a lot of use unless you can actually reconnect to the resource manager and get an XAResource object from its driver, as it is the methods on that object to which you pass the Xid. So you need some way of storing connection information also. In rare cases the RM's driver provides a Serializable XAResource implementation, in which case the simplest (although not necessarily fastest) thing to do is serialize it into the tx log file. Recovery is then easy - deserialize the XAResource and invoke commit on it. Well, except for the whole multiple classloaders thing, but that's another story. Besides, hardly any drivers are actually so accommodating. So, we need another approach.

To deal with non-serializable XAResources, the recovery system allows registration of plugins, one per resource manager, that provide a new XAResource instance on demand. In olden times configuring these was a manual task, frequently overlooked. As a result the recovery system found itself unable to get a new XAResource instance to handle the Xid from the transaction logs. Hence norecoveryxa. That problem was solved, at least for xa-datasource use within JBossAS, by having the datasource deployment handler automatically register a plugin with the recovery system. Bye-bye AppServerJDBCXARecovery, you will not be missed.

That leaves cases where the resource manager is something other than an XA aware JDBC datasource. To help diagnose such cases it would be helpful to have something more than a hex representation of the Xid. Whilst the standard XAResource API does not allow for such metadata, we now have an extension that provides the transaction manager with RM type, version and JNDI binding information. This gets written to the tx logs and, in the case of the JNDI name, encoded into the Xid.

As a side benefit, this information allows for much more helpful debug messages and hopefully means I spend less time on support cases. Didn't really think I was doing all this just for your benefit did you?

before:


TRACE: XAResourceRecord.topLevelPrepare for < formatId=131076,
gtrid_length=29, bqual_length=28, tx_uid=0:ffff7f000001:ccd6:4e0da81f:2,
node_name=1, branch_uid=0:ffff7f000001:ccd6:4e0da81f:3>

after:

TRACE: XAResourceRecord.topLevelPrepare for < formatId=131076,
gtrid_length=29, bqual_length=28, tx_uid=0:ffff7f000001:ccd6:4e0da81f:2,
node_name=1, branch_uid=0:ffff7f000001:ccd6:4e0da81f:3>,
product: OracleDB/11g, jndiName: java:/jdbc/TestDB >

The main reason we want the RM metadata though is so that when recovery tries and fails to find an XAResource instance to match the Xid, it can now report the details of the RM to which that Xid belongs. This makes it more obvious which RM is unavailable e.g. because it's not registered for recovery or it is temporarily down.


WARN: ARJUNA16037: Could not find new XAResource to use for recovering
non-serializable XAResource XAResourceRecord < resource:null,
txid:< formatId=131076, gtrid_length=29, bqual_length=41,
tx_uid=0:ffff7f000001:e437:4e294bb6:6, node_name=1,
branch_uid=0:ffff7f000001:e437:4e294bb6:7, eis_name=java:/jdbc/DummyDB >,
heuristic: TwoPhaseOutcome.FINISH_OK,
product: DummyProductName/DummyProductVersion,
jndiName: java:/jdbc/DummyDB>

Despite the name change (part of the new i18n framework), that's still our old nemesis norecoveryxa. It's made of stern stuff and, although now a little more useful, is not so easily vanquished entirely.

Even with all the resource managers correctly registered, it's still possible to be unable to find the Xid during a recovery scan. That's because of a timing window in the XA protocol, where a crash occurs after the RM has committed and forgotten the branch but before the TM has disposed of its no longer needed log file. In such cases no RM will claim ownership of the branch during replay, leaving the TM to assume that the owning RM is unavailable for some reason. With the name of the owning RM to hand, we now have the possibility to match against the name of the scanned RMs, conclude the branch belongs to one of the scanned RMs even though it pleaded ignorance, and hence dispose of it as a safely committed branch.

All of which should ensure that norecoveryxa's days in the spotlight are severely numbered.

rethinking log storage

2011-07-08T14:00:00.000+01:00

Transaction coordinators like JBossTS must write log files to fault tolerant storage in order to be able to guarantee correct completion of transactions in the event of a crash. Historically this has meant using files hosted on RAIDed hard disks.

The workload is characterised by near 100% writes, reads for recovery only, a large number of small (sub 1 KB) operations and, critically, the need to guarantee that the data has been forced through assorted cache layers to non-volatile media before continuing.

Unfortunately this combination of requirements has a tendency to negate many of the performance optimizations provided in storage systems, making log writing a common bottleneck for high performance transaction systems. Addressing this can often force users into unwelcome architectural decisions, particularly in cloud environments.

So naturally transaction system developers like the JBossTS team spend a lot of time thinking about how to mange log I/O better. Those who have read or watched my JUDCon presentation will already have some hints of the ideas we're currently looking at, including using cluster based in-memory replication instead of disk storage.

More recently I've been playing with SSD based storage some more, following up on earlier work to better understand some of the issues that arise as users transition to a new generation of disk technology with new performance characteristics.

As you'll recall, we recently took the high speed journalling code from HornetQ and used it as the basis for a transaction log. Relatively speaking, the results were a pretty spectacular improvement over our classic logging code. In absolute terms however we still weren't getting the best out the hardware.

A few weeks back I got my grubby paws on one of the latest SSDs. On paper its next generation controller and faster interface should have provided a substantial improvement over its predecessor. Not so for my transaction logging tests though - in keeping with tradition it utterly failed to outperform the older generation of technology. On further investigation the reasons for this become clear.

Traditional journalling solutions are based on a) aggregating a large number of small I/Os into a much smaller number of larger I/Os so that the drive can keep up with the load and b) serializing these writes to a single append-only file in order to avoid expensive head seeks.

With SSDs the first of those ideas is still sound, but the number of I/O events the drive can deal with is substantially higher. This requires re-tuning the journal parameters. For some usage it even becomes undesirable to batch the I/Os - until the drive is saturated with requests it's just pointless overhead that delays threads unnecessarily. As those threads (transactions) may have locks on other data any additional latency is undesirable. Also, unlike some other systems, a transaction manager has one thread per tx, as it's the one executing the business logic. It can't proceed until the log write completes, so write batching solutions involve significant thread management and scheduling overhead and often have a large number of threads parked waiting on I/O.

There is another snag though: even where the journalling code can use async I/O to dispatch multiple writes to the O/S in parallel, the filesystem still runs them sequentially as they contend on the inode semaphore for the log file. Thus writes for unrelated transactions must wait unnecessarily, much like the situation that arises in business applications which uses too coarse-grained locking. Also, the nice ncq for the drive remains largely unused, limiting the ability of the drive controller to execute writes in parallel.

The serialization of writes to the hardware, whilst useful for head positioning on HDDs, is a painful mistake for SSDs. These devices, like a modern CPU, require high concurrency in the workload to perform at their best. So, just as we go looking for parallelization opportunities in our apps, so we must look for them in the design of the I/O logging.

The most obvious solution is to load balance the logging over several journal files when running on an SSD. It's not quite that simple though - care must be taken to avoid a filesystem journal write as that will trigger a buffer flush for the entire filesystem, not just the individual file. Not to mention contention on the filesystem journal. For optimal performance it may pay to put each log file on its own small filesystem. But I'm getting ahead of myself. First there is the minor matter of actually writing a prototype load balancer to test the ideas. Any volunteers?

STM Arjuna

2011-06-26T23:04:00.000+01:00

I'd forgotten how long ago I'd promised to talk about some of the STM work we've been doing. Well I haven't been able to do much more on it for a while, but I do have time to at least outline what's possible at the moment. So let's just remember a bit about the current way in which you can use JBossTS to build transactional POJOs without the need for a database; we'll use an example to illustrate:

  public class AtomicObject extends LockManager
 {
 public AtomicObject()
 {
     super();

     state = 0;

     AtomicAction A = new AtomicAction();

     A.begin();

     if (setlock(new Lock(LockMode.WRITE), 0) == LockResult.GRANTED)
     {
         if (A.commit() == ActionStatus.COMMITTED)
             System.out.println("Created persistent object " + get_uid());
         else
             System.out.println("Action.commit error.");
     }
     else
     {
         A.abort();

         System.out.println("setlock error.");
     }
 }

 public void incr (int value) throws Exception
 {
     AtomicAction A = new AtomicAction();

     A.begin();

     if (setlock(new Lock(LockMode.WRITE), 0) == LockResult.GRANTED)
     {
         state += value;

         if (A.commit() != ActionStatus.COMMITTED)
             throw new Exception("Action commit error.");
         else
             return;
     }

     A.abort();

     throw new Exception("Write lock error.");
 }

 public void set (int value) throws Exception
 {
     AtomicAction A = new AtomicAction();

     A.begin();

     if (setlock(new Lock(LockMode.WRITE), 0) == LockResult.GRANTED)
     {
         state = value;

         if (A.commit() != ActionStatus.COMMITTED)
             throw new Exception("Action commit error.");
         else
             return;
     }

     A.abort();

     throw new Exception("Write lock error.");
 }

 public int get () throws Exception
 {
     AtomicAction A = new AtomicAction();
     int value = -1;

     A.begin();

     if (setlock(new Lock(LockMode.READ), 0) == LockResult.GRANTED)
     {
         value = state;

         if (A.commit() == ActionStatus.COMMITTED)
             return value;
         else
             throw new Exception("Action commit error.");
     }

     A.abort();

     throw new Exception("Read lock error.");
 }

 public boolean save_state (OutputObjectState os, int ot)
 {
     boolean result = super.save_state(os, ot);

     if (!result)
         return false;

     try
     {
         os.packInt(state);
     }
     catch (IOException e)
     {
         result = false;
     }

     return result;
 }

 public boolean restore_state (InputObjectState os, int ot)
 {
     boolean result = super.restore_state(os, ot);

     if (!result)
         return false;

     try
     {
         state = os.unpackInt();
     }
     catch (IOException e)
     {
         result = false;
     }

     return result;
 }

 public String type ()
 {
     return "/StateManager/LockManager/AtomicObject";
 }

 private int state;
}

Yes, quite a bit of code for a transactional integer. But if you consider what's going on here and why, such as setting locks and creating potentially nested transactions within each method, it starts to make sense. So how would we use this class? We'll let's just take a look at a unit test to see:

     AtomicObject obj = new AtomicObject();
    AtomicAction a = new AtomicAction();

    a.begin();

    obj.set(1234);

    a.commit();

    assertEquals(obj.get(), 1234);

    a = new AtomicAction();

    a.begin();

    obj.incr(1);

    a.abort();

    assertEquals(obj.get(), 1234);

But we can do a lot better in terms of ease of use, especially if you consider what's behind STM: where objects are volatile (don't survive machine crashes). And this is where our approach comes in. Let's look at the example above and create an interface for the AtomicObject:

public interface Atomic
{
   public void incr (int value) throws Exception;

   public void set (int value) throws Exception;

   public int get () throws Exception;
}

And now we can create an implementation of this:

@Transactional
public class ExampleSTM implements Atomic
{
   @ReadLock
   public int get () throws Exception
   {
       return state;
   }

   @WriteLock
   public void set (int value) throws Exception
   {
       state = value;
   }

   @WriteLock
   public void incr (int value) throws Exception
   {
       state += value;
   }

   @TransactionalState
   private int state;
}

Here we now simply use annotations to specify what we want. The @Transactional is needed to indicate that this will be a transactional object. Then we use @ReadLock or @WriteLock to indicate the types of locks that we need on a per method basis. (If you don't define these then the default is to assume @WriteLock). And we use @TransactionalState to indicate to the STM implementation which state we want to operate on and have the isolation and atomicity properties (remember, with STM there's no requirement for the D in ACID). If there's more state in the implementation than we want to recover (e.g., it can be recomputed on each method invocation) then we don't have to annotate it.

But that's it: the AtomicObject and ExampleSTM classes are identical. So let's take a look at another unit test:

      RecoverableContainer theContainer = new RecoverableContainer();
     ExampleSTM basic = new ExampleSTM();
     Atomic obj = obj = theContainer.enlist(basic);    
     AtomicAction a = new AtomicAction();
  
     a.begin();
  
     obj.set(1234);
  
     a.commit();

     assertEquals(obj.get(), 1234);
  
     a = new AtomicAction();

     a.begin();

     obj.incr(1);
  
     a.abort();

     assertEquals(obj.get(), 1234);

Think of the RecoverableContainer as the unit of software transactional memory, within which we can place objects that it will manage. In this case ExampleSTM. Once placed, we get back a reference to the instance within the unit of memory, and as long as we use this reference from that point forward we will have the A, C and I properties that we want.

Ever wondered about transactions and threads?

2011-06-14T15:25:00.000+01:00

No? Well you really should! When transaction systems were first developed they were single-threaded (where a thread is defined to be an entity which performs work, e.g., a lightweight process, or an operating-system process.) Executing multiple threads within a single process was a novelty! In such an environment the thread terminating the transaction is, by definition, the thread that performed the work. Therefore, the termination of a transaction is implicitly synchronized with the completion of the transactional work: there can be no outstanding work still going on when the transaction starts to finish.

With the increased availability of both software and hardware multi-threading, transaction services are now being required to allow multiple threads to be active within a transaction (though it’s still not mandated anywhere, so if this is something you want then you may still have to look around the various implementations). In such systems it is important to guarantee that all of these threads have completed when a transaction is terminated, otherwise some work may not be performed transactionally.

Although protocols exist for enforcing thread and transaction synchronization in local and distributed environments (commonly referred to as checked transactions), they assume that communication between threads is synchronous (e.g., via remote procedure call). A thread making a synchronous call will block until the call returns, signifying that any threads created have terminated. However, a range of distributed applications exists (and yours may be one of them) which require extensive use of concurrency in order to meet real-time performance requirements and utilize asynchronous message passing for communication. In such environments it is difficult to guarantee synchronization between threads, since the application may not communicate the completion of work to a sender, as is done implicitly with synchronous invocations.

As we’ve just seen, applications that do not create new threads and only use synchronous invocations within transactions implicitly exhibit checked behavior. That is, it is guaranteed that whenever the transaction ends there can be no thread active within the transaction which has not completed its processing. This is illustrated below, in which vertical lines indicate the execution of object methods, horizontal lines message exchange, and the boxes represent objects.

The figure illustrates a client who starts a transaction by invoking a synchronous ‘begin’ upon a transaction manager. The client later performs a synchronous invocation upon object a that in turn invokes object b. Each of these objects is registered as being involved in the transaction with the manager. Whenever the client invokes the transaction ‘end’ upon the manager, the manager is then able to enter into the commit protocol (of which only the final phase is shown here) with the registered objects before returning control to the client.

However, when asynchronous invocation is allowed, explicit synchronization is required between threads and transactions in order to guarantee checked (safe) behavior. The next figure illustrates the possible consequences of using asynchronous invocation without such synchronization. In this example a client starts a transaction and then invokes an asynchronous operation upon object a that registers itself within the transaction as before. a then invokes an asynchronous operation upon object b. Now, depending upon the order in which the threads are scheduled, it’s possible that the client might call for the transaction to terminate. At this point the transaction coordinator knows only of a’s involvement within the transaction so enters into the commit protocol, with a committing as a consequence. Then b attempts to register itself within the transaction, and is unable to do so. If the application intended the work performed by the invocations upon a and b to be performed within the same transaction, this may result in application-level inconsistencies. This is what checked transactions are supposed to prevent.

Some transaction service implementations will enforce checked behavior for the transactions they support, to provide an extra level of transaction integrity. The purpose of the checks is to ensure that all transactional requests made by the application have completed their processing before the transaction is committed. A checked transaction service guarantees that commit will not succeed unless all transactional objects involved in the transaction have completed the processing of their transactional requests. If the transaction is rolled back then a check is not required, since all outstanding transactional activities will eventually rollback if they are not told to commit.

As a result, most (though not all) modern transaction systems provide automatic mechanisms for imposing checked transactions on both synchronous and asynchronous invocations. In essence, transactions must keep track of the threads and invocations (both synchronous and asynchronous) that are executing within them and whenever a transaction is terminated, the system must ensure that all active invocations return before the termination can occur and that all active threads are informed of the termination. This may sound simple, but believe us when we say that it isn’t!

Unfortunately this is another aspect of transaction processing that many implementations ignore. As with things like interposition (for performance) and failure recovery, it is an essential aspect that you really cannot do without. Not providing checked transactions is different from allowing checking to be disabled, which most commercial implementations support. In this case you typically have the ability to turn checked transactions off when you know it is safe, to help improve performance. If you think there is the slightest possibility you’ll be using multiple threads within the scope of a single transaction or may make asynchronous transactional requests, then you’d better find out whether your transaction implementation it up to the job. I'm not even going to bother about the almost obligatory plug for JBossTS here ;-)

WTF is Interposition?

2011-06-02T18:44:00.000+01:00

Consider the situation depicted below, where there is a transaction coordinator and three participants. For this sake of this example, let us assume that each of these participants is on a different machine to the transaction coordinator and each other. Therefore, each of the lines not only represents participation within the transaction, but also remote invocations from the transaction coordinator to the participants and vice versa.

In a distributed system there’s always an overhead incurred when making remote invocations compared to making a purely local (within the same VM) invocation. Now the overhead involved in making these distributed invocations will depend upon a number of factors, including how congested the network is, the load on the respective machines, the number of transactions being executed etc. Some applications may be able to tolerate this overhead, whereas others may not. As the number of participants increase, so does the overhead for fairly obvious reasons.

A common approach to reduce this overhead is to realize that as far as a coordinator is concerned, it does not matter what the participant implementation does. For example, although one participant may interact with a database to commit the transaction, another may just as readily be responsible for interacting with a number of databases: essentially acting as a coordinator itself, as shown below:

In this case, the participant is acting like a proxy for the transaction coordinator (the root coordinator): it is responsible for interacting with the two participants when it receives an invocation from the coordinator and collating their responses (and it’s own) for the coordinator. As far as the participants are concerned, a coordinator is invoking them, whereas as far as the root coordinator is concerned it only sees participants.

This technique of using proxy coordinators (or subordinate coordinators) is known as interposition. Each domain (machine) that imports a transaction context may create a subordinate coordinator that enrolls with the imported coordinator as though it were a participant. Any participants that are required to enroll in the transaction within this domain actually enroll with the subordinate coordinator. In a large distributed application, a tree of coordinators and participants may be created.

A subordinate coordinator must obviously execute the two-phase commit protocol on its enlisted participants. Thus, it must have its own transaction log and corresponding failure recovery subsystem. It must record sufficient recovery information for any work it may do as a participant and additional recovery information for its role as a coordinator. Therefore, it is impossible for a normal participant to simply be a sub-coordinator because the roles are distinctly different; sub-coordinators are tightly coupled with the transaction system.

So the question then becomes when and why does interposition occur?

Performance: if a number of participants reside on the same node, or are located physically close to one another (e.g., reside in the same LAN domain) then it can improve performance for a remote coordinator to send a single message to a sub-coordinator that is co-located with those participants and for that sub-coordinator to disseminate the message locally, rather than for it to send each participant the same message.
Security and trust: a coordinator may not trust indirect participants and neither may indirect participants trust a remote coordinator. This makes direct registration impossible. Concentrating security and trust at coordinators can make it easier to reason about such issues in a large scale, loosely coupled environment.
Connectivity: some participants may not have direct connectivity with a specific coordinator, requiring a level of indirection.
Separation of concerns: many domains and services may simply not want to export (possibly sensitive) information about their implementations to the outside world.

You'll find interposition used in a number of transaction systems. Within JBossTS it's used by the JTS and XTS components.