Nested transactions

Here's another periodic entry on our road to discussing the STM work we're doing around JBossTS. We're going to build on the Transactional Objects for Java toolkit that I described before. But before doing so it's worth stressing that everything I'm talking about here is based on the ArjunaCore module, so it doesn't need an ORB or SOAP stack. In fact it doesn't actually need anything from an application server, whether or not that is J(2)EE based. So yes, you could take that module and embed it within your application, framework or whatever and gain full ACID transaction capabilities.

Hopefully if you're reading these entries then you already understand the concept of traditional transactions, which we'll refer to as top-level transactions here. If not then there are enough articles, presentations and books to get you there, and the JBossTS documentation will help too. But suffice it to say that with a top-level transaction, you get the all-or-nothing (Atomic) affect: no matter how many resources are updated within the scope of that transaction, if one of them commits the work then all of them will and likewise, if one of them rolls back the work then all of the others will also roll back their work.

Now this is a great capability to have, especially if you have to do very little as a user to get there. However, for some applications and objects it can be restrictive. For instance, if your transactions last for a long period of time (say minutes), updating different resources throughout, and just at the very end there's a failure, then everything has to be done again. This and other limitations with top-level transactions lead us and others to look at various extended transaction models, and one of the earliest approaches was nested transactions from 1981, also known as subtransactions (not to be confused with subordinate coordinators).

Transactions which are contained within the other transaction are said to be nested (or subtransactions), and the resulting transaction is referred to as the enclosing transaction. The enclosing transaction is sometimes referred to as the parent of a nested (or child) transaction. A hierarchical transaction structure can thus result, with the root of the hierarchy being referred to as the top-level transaction. An important difference exists between nested and top-level transactions: the effect of a nested transaction is provisional upon the commit/roll back of its enclosing transaction(s), i.e., the effects will be recovered if the enclosing transaction aborts, even if the nested transaction has committed.

Subtransactions are a useful mechanism for two reasons:

• fault-isolation: if subtransaction rolls back (e.g., because an object it was using fails) then this does not require the enclosing transaction to rollback, thus undoing all of the work performed so far.


• modularity: if there is already a transaction associated with a call when a new transaction is begun, then the transaction will be nested within it. Therefore, a programmer who knows that an object require transactions can use them within the object: if the object’s methods are invoked without a client transaction, then the object’s transactions will simply be top-level; otherwise, they will be nested within the scope of the client’s transactions. Likewise, a client need not know that the object is transactional, and can begin its own transaction.

Imagine what you could do with nested transactions, particularly if you get to control them programmatically. You can start transactions within your application or objects without having to be concerned about whether or not there are transactions already in scope. If you do know and care, then you can utilize them for application needs. For instance, nested transactions give you the ability to try alternate paths in a decision tree without having to worry about the impact on transactional resources; you try each option in a separate nested transaction and roll back the paths you don't chose, relying on the transaction system to undo that work automatically, and then you simply commit the selected path (nested transaction).

So how do you use them? Well in JBossTS there are no special constructs for nesting of transactions: if a transaction is begun while another is running on the thread then it is automatically nested. This allows for a modular structure to applications, whereby objects can be implemented using transactions within their operations without the application programmer having to worry about the applications which use them, i.e., whether or not the applications will use transactions as well. Thus, in some applications those transactions may be top-level, whereas in others they may be nested. Objects written in this way can then be shared between applications and JBossTS will guarantee their consistency. If a nested transaction is rolled back then all of its work will be undone, although strict two-phase locking means that any locks it may have obtained will be retained until the top-level transaction commits or rolls back. As mentioned before, if a nested transaction commits then the work it has performed will only be committed by the system if the top-level transaction commits; if the top-level transaction aborts then all of the work will be undone.

What all of the above means is that if you want to use nested transactions in your applications then all you need to is start them programmatically as we saw earlier. So for example, if we wanted to use the AtomicObject we defined earlier within a top-level transaction, the code could look like:


AtomicAction A = new AtomicAction();
AtomicObject obj = new AtomicObject();

A.begin();

obj.incr(1);

A.commit();


The fact that the AtomicObject incr method uses a transaction internally is hidden from the invoker. In this case it'll be nested. If the code looked like ...


AtomicObject obj = new AtomicObject();

obj.incr(1);


... then the transaction would be top-level. Hopefully the differences in behaviour are pretty obvious by now, but just in case: in the former case we get a chance to undo the increment if we were to roll back the enclosing transaction, whereas in the latter, assuming the incr method didn't fail, then there is no opportunity to roll back; we would have to do an application driven undo which isn't guaranteed to be equivalent from an ACID perspective (e.g., another user could have seen the state between the increment and the decrement). And of course that leads us towards compensating transactions, but they are definitely out of scope for this article.

So I hope I've given you a flavour to nested transactions and some of the power they could bring to applications and objects. Why not give them a go?

Thank you HAL Corporation for FUD spreading

I mentioned a few months back how a certain group of people at a certain company that was conjectured to be the basis for HAL 9000 in 2001: A Space Odyssey were spreading FUD about JBossTS. Now not doing your homework is bad enough, but then building a whole marketing exercise around it is even worse! To say it's like building a house of cards is an understatement. It only takes someone to start questioning, probing and pushing for the whole thing to start to come tumbling down and the sheer stupidity of the original exercise comes to the fore.

Now comparing and contrasting projects, products or pretty much anything, should really be done on a scientific basis. (OK there are some things in life where that might be overkill or inappropriate, but not here.) It should be testable and reproduceable, ideally by anyone independent. If you're going to make statements about the pros and cons of this or that, then in some ways your reputation is being put on the line. Of course if you don't value your reputation then this is all moot. However, for JBoss and Red Hat, reputation and integrity are very important, so we're not going to say something bad or potentially maligning unless we have all of the facts, and even then we'll typically do it in a very scientific manner once again. This means that if we're wrong and it's because we made a mistake, e.g., had overlooked something, then we'll admit it.

Another thing we won't do is go around prospective customers spreading FUD, demonstrating FUD and making statements about "fundamental issues" with a competitor's product/project that mean it can never do X, Y or Z, unless there's proof, and in which case it's not really FUD. But of course our principles aren't shared by everyone. But when I heard that that was precisely what was happening it made their previous FUD attempts pale into relative insignificance. Despite my previous blog post, lots of real world deployments showing that recovery works, a history dating back to the 1980's which included being part of HP's foray into the middleware space, it wasn't enough for some who had been bedazzled by the IBM Roadshow (and to a degree you can't really blame them, since this is Big Blue after all.)

So we decided that enough was enough. Our honour had been besmirched (insert smiley). Fortunately recovery is so critical to a transaction system that we have lots of demonstrators and QA tests that cover pretty much every situation you can consider. And most of them are publicly available in our repository. Wow: who knew?! (Insert tongue firmly in cheek). So our teams (TS and QA) got together and in less than a day had produced something that duplicated the scenario, showed that recovery works, and therefore showed what a load of rubbish was being spouted.

Now of course it can be argued that we knew all of this to start with so we just wasted time and effort proving it yet again. I agree in part, and when this impacts on my holidays, as it did, it does make me a little bit mad. But I don't blame anyone other than the original mud slingers. Hopefully they'll learn something from this exercise. Hopefully they'll learn what open source is about and how to use it as well as contribute towards it. Unfortunately I have my doubts. So until the next time I'll leave you with this thought: what's that they say about glass houses?

Building transactional applications with JBossTS

Towards the end of last year I mentioned that we've been doing some work on STM around JBossTS. I said I'd give some hints at what we've been doing, but in order to do so we need to cover some background on how you can construct transactional applications with the existing code using Transactional Objects for Java (TXOJ). You'll find a lot more information about what I'm going to summarize in the documentation and various papers.

In order for your applications to possess all of the necessary ACID properties when using transactions, the objects manipulated by those transactions need to be both durable and isolated from conflicts. If your application objects aren't backed by a database, then you have the headache of providing these capabilities yourself, unless you have your friendly neighborhood transactional framework like TXOJ. Without going into all of the details (which really are in the documentation) probably the most important class is StateManager which provides all of the basic support mechanisms required by an object for state management purposes. As with concurrency control, which we'll come on to later, durability is obtained through inheritance.

Objects in TXOJ are assumed to be of three possible basic flavours. They may simply be recoverable, in which case StateManager will attempt to generate and maintain appropriate recovery information for the object. Such objects have lifetimes that do not exceed the application program that creates them. Objects may be recoverable and persistent, in which case the lifetime of the object is assumed to be greater than that of the creating or accessing application so that in addition to maintaining recovery information StateManager will attempt to automatically load (unload) any existing persistent state for the object at appropriate times. Finally, objects may possess none of these capabilities in which case no recovery information is ever kept nor is object activation/deactivation ever automatically attempted.

There are quite a few StateManager methods that your application could utilize directly, but for now the only three that are of immediate importance are save_state, restore_state and type. The first two methods are responsible for saving (or restoring) the state of the object, whereas the last is used when positioning the state in the transactional object store. Now although the store may also be the location of the transaction log, the application object states aren't necessarily maintained within the log. (Yes, I'm glossing over subjects such as activating and passivating objects, but we may come back to those in a later posting.)

With these three StateManager methods defined you could manage the state of your object's within the scope of a transaction. But typically you'll also want isolation, so let's move on to LockManager, which inherits from StateManager. So your application objects can become isolated and durable by having their corresponding classes inherit from LockManager. Although you have to be concerned about the three StateManager methods mentioned earlier, there's nothing else here that you need worry about in terms of overriding or defining. But of course you do need to tell the system when and how to isolate your objects. The primary programmer interface to the concurrency controller is via the setlock operation. By default the runtime enforces strict two-phase locking following a multiple reader, single writer policy on a per object basis. Although lock acquisition is under programmer control, lock release is normally under control of the system and requires no further intervention by the programmer. This ensures that the two-phase property can be correctly maintained. (Again, I'll ignore subjects like type specific concurrency control for this posting.)

But sometimes a picture paints a thousand words, so here's an example:

public class AtomicObject extends LockManager
{
    public AtomicObject ()
    {
        super(ObjectType.ANDPERSISTENT);

        state = 0;

        AtomicAction A = new AtomicAction();

        A.begin();

        if (setlock(new Lock(LockMode.WRITE), 0) == LockResult.GRANTED)
        {
            if (A.commit() == ActionStatus.COMMITTED)
                System.out.println("Created persistent object " + get_uid());
            else
                System.out.println("Action.commit error.");
        }
        else
        {
            A.abort();

            System.out.println("setlock error.");
        }
    }

    public AtomicObject (Uid u)
    {
        super(u, ObjectType.ANDPERSISTENT);

        state = -1;

        AtomicAction A = new AtomicAction();

        A.begin();

        if (setlock(new Lock(LockMode.READ), 0) == LockResult.GRANTED)
        {
            System.out.println("Recreated object " + u);
            A.commit();
        }
        else
        {
            System.out.println("Error recreating object " + u);
            A.abort();
        }
    }
    
    public void incr (int value) throws TestException
    {
        AtomicAction A = new AtomicAction();

        A.begin();

        if (setlock(new Lock(LockMode.WRITE), retry) == LockResult.GRANTED)
        {
            state += value;

            if (A.commit() != ActionStatus.COMMITTED)
                throw new TestException("Action commit error.");
            else
                  return;
        }

        A.abort();

        throw new TestException("Write lock error.");
    }

    public void set (int value) throws TestException
    {
        AtomicAction A = new AtomicAction();

        A.begin();

        if (setlock(new Lock(LockMode.WRITE), retry) == LockResult.GRANTED)
        {
            state = value;

            if (A.commit() != ActionStatus.COMMITTED)
                throw new TestException("Action commit error.");
            else
                return;
        }

        A.abort();

        throw new TestException("Write lock error.");
    }

    public int get () throws TestException
    {
        AtomicAction A = new AtomicAction();
        int value = -1;

        A.begin();

        if (setlock(new Lock(LockMode.READ), retry) == LockResult.GRANTED)
        {
            value = state;

            if (A.commit() == ActionStatus.COMMITTED)
                return value;
            else
                throw new TestException("Action commit error.");
        }

        A.abort();

        throw new TestException("Read lock error.");
    }

    public boolean save_state (OutputObjectState os, int ot)
    {
        boolean result = super.save_state(os, ot);

        if (!result)
            return false;

        try
        {
            os.packInt(state);
        }
        catch (IOException e)
        {
            result = false;
        }

        return result;
    }

    public boolean restore_state (InputObjectState os, int ot)
    {
        boolean result = super.restore_state(os, ot);

        if (!result)
            return false;

        try
        {
            state = os.unpackInt();
        }
        catch (IOException e)
        {
            result = false;
        }

        return result;
    }

    public String type ()
    {
        return "/StateManager/LockManager/AtomicObject";
    }

    private int state;
}

What we have here is a durable and isolatable integer. It may look like quite a bit of code, but there's a lot of defensive programming here to illustrate explicitly some of the problems that could occur. It's left as an exercise for the reader to optimize the code.

However, what you can see here is the use of AtomicAction for creating transactions (which will be automatically nested if there's already a transaction associated with the thread of control) and then the acquiring of the right types of Lock within the application object code (READ or WRITE to signify whether or not the state of the object may be modified). If you consider what's needed to ensure that the object is transactional, including full recovery, there isn't a lot for the developer to do. OK, it's a bit more than you might expect if you're used to EJB3s, for example, but then there's a bit more going on here, and some of it could well be automated (future posting hint). But when you consider that an entire database system was built on this basis back in the early 1990's (OK, the C++ version of the transaction system and not Java, but very little difference), it helps to illustrate the power of the abstractions as well as the implementation.

OK, that's enough for now. In a later post we may go into more detail on some of the concepts mentioned here (or glossed over so far) before heading off into STM land. If there's anything specific you'd like concentrating on then just suggest via the comments.