HAL FUD rebuttal

A while ago I got rather annoyed at the shoddy work that IBM did at FUD around JBossTS. I later said that we were going to produce a response (as if one was really needed after 25 years of work, but hey, this is the Youtube generation!) Well it's taken a little longer than I had expected, but it's now available. And unlike the original IBM mess, this demo includes a reference to the source code so you can download it and try it for yourself. I'd recommend to IBM that they give it a go and perhaps re-examine their own tests as a result, but I doubt they're interested in facts and the truth.

Thanks the the entire team for pulling this together! It was a great JBoss effort.

JavaOne presentations

Catch various ones here, including a one of transactions.

Standalone XTS and lightweight clients

I've recently been working with a community user to help restore the ability to deploy the Web Services Transactions (XTS) code outside of JBoss AS. XTS is a layer built on top of the JBoss Transactions core which allows you to implement transactional web services and transactional web service clients. Those long enough in the tooth to know about the previous incarnation of the XTS code when it was part of the Arjuna codebase will recall that in the good old, bad old days XTS had to be deployed bundled in with the web service client and/or the web service itself. When we re-implemented XTS to conform to the WSTX 1.1 standard we rebuilt it so that it deployed as a service inside JBoss AS. We also relaid the internal message transport on new foundations. Communications between the client or web service participants and the transaction coordinator were rebuilt over the JaxWS implementation provided by JBossWS-Native. More recently we upgraded XTS so that it also works with JBossWS-CXF.

Although this decision appears to tie XTS tightly into the JBoss AS product it was actually motivated by the desire to render the code independent of a specific transport layer. In theory JaxWS provides an abstraction that is available in many implementation flavours and, in theory, this means that it should be possible to deploy the 1.1 implementation in any container that

• provides support for deployment of JaxWS endpoints on the server side and
• allows for creation of JaxWS service proxies on the client side

That's pretty much all that is needed to allow XTS to run in any web service container, except ... it ignores a significant lacuna in the JaxWS specifications.

XTS requires the use of Web Service Addressing (WSA) headers on the protocol messages sent between the client and transaction coordinator or between the participant web service and the coordinator. The original draft JaxWS-WSA specification defined an API which enabled the two sides of a JaxWS exchange to access and manipulate WSA headers. However, this feature got shunted off into the long grass and the final version only provided a minimal WSA capability which made it impossible to implement the WSTX spec using only JaxWS standard functionality.

Luckily for me, JBossWS Native and the CXF project have both always provided an implementation of the original draft WSA feature and, modulo a few annoyingly variations (that's the problem with not standardising), this means that both these WS stacks include all that is needed to deploy the XTS service. JBossWS now includes a small library containing a simple abstraction layer which hides the minor differences in the WSA implementations. That is all that is needed for XTS to deploy inside JBoss AS over either JBossWS-CXF or JBossWS-Native. However, this also re-opens the possibility of bundling the XTS code with the JBossTS code and CXF as a standalone deployment.

Obviously, there is not a great deal of benefit in deploying a transactional web service outside of an application server. It's very likely that you will want such a web service to run inside a container located in a managed environment with 24/7 monitoring, support and so on. A web service is likely to need to employ a database and require configuration of data backups, crash recovery, centralised logging, IT staff on hand to deal with catastrophic failures etc.

Deployment of a lightweight transactional web service client is a much more interesting proposition. If you can bundle the transactional web service client software in a simple desktop Java application then this provides an easy management path for exposing reliable distributed service applications to users. The desktop client negotiate with a networked coordinator service when it creates a transaction. It can use normal web service invocations to request service from transactional web services located in the intranet or even on the internet. The web services negotiate with the same coordinator to ensure that they either all commit their changes or all revert to their original state. The more heavy-weight apparatus needed on the coordinator or web service hosts to ensure reliability, availability, security etc are not actually needed on the client node.

In theory all the transactional client needs to be able to do in order to participate in a Web Service Transaction is talk to the coordinator and the web services using JaxWS client capabilities built over an HTTP client library. There is no reason for the client to require all the capabilities of a full web service container and if definitely does not need to impose the management overheads that running a full container would mandate. The client only requires minimal installation, configuration and maintenance to operate as an HTTP client. It does not need to be involved in automatic or manual crash recovery procedures. It does not need to take part in centralised logging in order to allow monitoring/supervision by IT support staff. Nor does the client need to expose service endpoints on a public network interface and, in consequence, open holes in the client machine's firewall. Bundling the CXF client libraries with the XTS client code in a standalone Java app seems like a very attractive proposition.

In practice things are not quite so simple as that which is why I have been busy for the last month or two. The practical problems were twofold. The first problem was that XTS was built to be deployed as a monolith. When you install the XTS service inside JBoss AS you actually install the code required by all 3 of the agents involved in a transaction: the client, the participant web service and the transaction coordinator. The XTS code actually comprises about half a dozen main software services with their associated web services. These software services are layered so as to support the functions defined by the WSTX specifications. Unfortunately, the needs of the three agents, client, web service, coordinator, cut across these layers making it hard to decouple them.

So, when it comes to deploying a transactional application you might decide that you want to deploy your client application to one JBoss AS instance on Host1 and your web services on two other JBoss AS instances running on Host2 and Host3. For better reliability you might want to the coordinator services to operate in a separate JBoss AS instance on Host4. Unfortunately, up until a few weeks ago you would have to deploy, load and initialise the XTS services for all three types of agent on all of these hosts.

Well, not any more. There is now a simple configuration mechanism which allows you to enable or disable the client, participant web service or coordinator capabilities in a given XTS deployment. There is still a single XTS service archive to drop into the JBoss AS deploy directory. However, you can easily configure the deployment using either dependency injection or file-based property configuration to ensure that only the code and services you want get loaded and initialised. So, you can configure Host1 only to initialise the client capability, and so on.

If you want to deploy to a non-JBoss web container it is now also straightforward to isolate the configuration elements which control deployment of the JaxWS endpoints used internally by XTS. The XTS service archive contains several web archives which do nothing more than map endpoints exposed by the 3 XTS agents. These can easily be unbundled and the mapping can be redefined using an alternative deployment framework like, say, Spring. All you need to do to deploy XTS outside JBoss AS is to bundle CXF, the XTS and JBossTS libraries and the JBossWS WSA compatibility library with your deployment, configure which XTS agents you want to initialise then call the JBossTS and XTS start and stop methods during deployment and undeployment of your application.

This is all you need if you want to deploy to a web container. Unfortunately there is still a second barrier to deploying a simple, lightweight transactional client. This is a harder problem because it lies within the WSTX standard itself. Unfortunately, the WSTX specification, or at least the WSAT component (which specifies the Atomic Transaction protocol), requires a transactional client to expose a service endpoint. WSTX bypasses this issue as far as the WSBA (Business Activity) protocol specification is concerned, but only because it leaves the corresponding part of the WSBA specification undefined! In practice the same requirement applies for WSBA.

This may seem a bit bizarre but it's a rather unfortunate artefact of how WSTX was defined. I guess that at the time WSTX was defined it was felt that anyone implementing a transactional client would only want to deploy it inside an application server. So, this was not considered to be an issue. In order to explain why the spec is like this and what is required to fix it I'll need to go into some of the details of what the spec actually requires a client implementation to do.

The problem does not arise when an application client starts a transaction, either by calling UserTransaction.begin() or UserBusinessActivity.begin(). Under the hood these two calls use JaxWS to talk to an XTS service called the Activation Coordinator. This service employs a request-response message exchange pattern (MEP). The client opens an HTTP connection and sends a create request. The service creates a transaction and uses the HTTP back channel to hand back a transaction context containing a unique identifier for the transaction and the address of another coordinator service called the Registration Coordinator. This is the same information which gets smuggled into client requests to the transactional web services, allowing them to talk to the Registration Coordinator and register as a participant in the WSAT or WSBA transaction (although, confusingly, the standard term used for this operation is enlist).

The application client does not just stash away the context returned by the Activation Coordinator and then return fro the begin() call. It also registers (or enlists) as a participant in the transaction. Although it uses the same operation as the web services it does so for a different purpose. A transactional web service has to enlist so that it can be told when to prepare and commit (or, possibly, abort) any changes made during the transaction. The client has to enlist in order to be able to complete the transaction. In a WSAT transaction the client sends a message to the Registration Coordinator to enlist with the WSAT Completion Coordinator service. This allows it to send a termination request, either Commit or Rollback, later on. Once again the enlist operation uses a request-response MEP but this time the response on the back channel includes the address of the Completion Coordinator service. Likewise, a WSBA client enlists with the Termination Coordinator service and gets back that service's address.

The problem arises when the application client code calls UserTransaction.commit() or UserBusinessActivity.close() to terminate the transaction. The XTS code uses JaxWS to send, respectively, a Commit or Close request to the Completion or Termination Coordinator service. Alternatively, if the application client code calls UserTransaction.rollback() or UserBusinessActivity.cancel() the XTS code uses JaxWS to send, respectively, an Abort or Cancel request. Unfortunately, in all of these cases the WSAT specification mandates that this request empoloys one-way MEP. That means that the client sends the request then closes the connection without waiting for a response. So, how is it supposed to find out when the request has been handled and what happened?

Well, that is why the client needs to expose a JaxWS endpoint. The WSAT client has to implement a Completion Initiator service which will accept incoming messages using a one-way MEP, either a Committed or an Aborted message. After posting , say, a Commit request the XTS code places the calling thread in a wait until one or other messages is received by the Completion Initiator service. This service wakes up the waiting thread and notifies it of the outcome. So, what is essentially a request-response exchange is achieved using two independent messages on two independent HTTP connections one initiated from the application client to the coordinator host and one initiated from the coordinator host back to the client. The WSBA specification does not actually specify a termination protocol but the old Arjuna implementation adopted a similar model based on two one-way messages thus requiring a WSBA client also to expose an end point, in this case for the Termination Participant service.

So, how does the Completion or Termination Coordinator get hold of the address for the client endpoint? Well, the answer to that question really explains how and why this problem comes about. The enlist request supported by the Registration Service expects to be passed three values, a unique identifier for the enlisting party derived from the transaction id, the name of the service the caller wants to enlist for and, finally, a URL providing the missing address. Once the registration request has completed both sides have each others addresses. The protocol expects them to conduct any further communications using a sequence of one way messages labelled with the unique identifier supplied in the enlist call.

One way messages are not a bad idea for communications between the coordinator and the participant web services. Using a request-response MEP requires holding open a connection while waiting for the response. For, say, a Prepare-Prepared exchange this might take a long time, especially if a web service has made a lot of state changes which have to be logged to disk. Holding open lots of connections in a busy web server might cause a resource bottleneck. Also, request-response is only useful if the exchange is always a simple two-way exchange. WSBA sometimes requires longer message sequences not easily mapped onto the request-reponse model. For example, if a WSBA web service is told to complete its changes and finds that it cannot do so the 3-way message exchange, Complete, CannotComplete, NotCompleted is required.

At the client the first of these concerns is less important (there is normally only one connection per client) and the second does not apply. If the completion requests could be implemented using a simple request-response exchange then this would remove the need to expose service endpoints and, hence, allow deployment of a really lightweight client. So, the solution we have adopted is to augment the standard and provide an alternative version of the Completion and Termination Coordinator services which support this type of MEP. Our coordinator service implementations exposes two new endpoints alongside the original completion services, the Completion Coordinator RPC service and the Termination Coordinator RPC service. These services have no corresponding Initiator service. Our client implementation can be configured to use either the normal service or the RPC vwersion. So, if you deploy a client to a web container you can configure it to use the standard one-way MEP requests to the standard services. If you deploy a client as a standalone Java app you can configure it to use request-response MEPs to the extended completion services.

Of course, you can only use this lightweight client if you point it at a JBoss XTS coordinator. If you tried to use another vendor's implementation of WSTX then you would get an error when you begin your transaction because the 3rd party coordinator service would not recognise enlist requests for our extended completion services. This is not quite such a leap into the unknown as it appears. WSBA does not specify completion protocol. So, if you try to use another vendor's coordinator service with our WSBA client library it will fail whichever type of client you deploy. That's a very good reason why the WSTX specification should be extended to include a WSBA completion protocol and to add the request-response MEP variants of the completion protocols so they sit alongside the one-way MEP versions.

Transactions and volatile state

So far we've looked at some of the basic of writing transactional applications using just JBossTS and TXOJ, as well as the importance of nested transactions. In this entry we'll look at another way of relaxing some of the traditional ACID properties, namely that of durability (persistence).

Hopefully you'll realize that normally when using transactions the Durability aspect kicks in with the transaction log, which maintains a record of the participants and the state of the transaction, as well as transactional participants (e.g., business objects), which have to record any state changes that are made in the context of the transaction. Both of these are needed to ensure atomicity in the presence of failures. But although disk speeds have improved significantly over the years, compared to improvements in processor speeds the disk is still the bottleneck. But it needn't be if all you want is ACI for your objects and applications. Such objects are often referred to as volatile or recoverable in that they have state that cannot survive machine crashes but which is still subject to atomic updates, i.e., previous states can be recovered if the transaction rolls back as long as there are no crashes.

But of course it's no good to simply stop your business objects from writing state, assuming you are even aware that they are doing so within a transactional application; you must have similar recognition by the transaction manager so that it doesn't write the log on behalf of volatile objects. Obviously if there are only volatile objects in the transaction then there's no need for a lot at all and performance can improve significantly. But of course you need to understand the trade-offs that using volatile objects give, which can essentially be summarized as performance vs crash recovery.

Fortunately JBossTS and TXOJ support volatile objects, including recognition by the coordinator. You don't need to do anything special when creating or starting your transactions: the coordinator will only create a log during commit if it detects more than one non-volatile object within its scope. And for your TXOJ objects you have to do very little, as illustrated by the modified AtomicObject code below from one of the previous articles:

public class AtomicObject extends LockManager
{
    public AtomicObject ()
    {
        state = 0;
    }
    
    public void incr (int value) throws TestException
    {
        AtomicAction A = new AtomicAction();

        A.begin();

        if (setlock(new Lock(LockMode.WRITE), retry) == LockResult.GRANTED)
        {
            state += value;

            if (A.commit() != ActionStatus.COMMITTED)
                throw new TestException("Action commit error.");
            else
                  return;
        }

        A.abort();

        throw new TestException("Write lock error.");
    }

    public void set (int value) throws TestException
    {
        AtomicAction A = new AtomicAction();

        A.begin();

        if (setlock(new Lock(LockMode.WRITE), retry) == LockResult.GRANTED)
        {
            state = value;

            if (A.commit() != ActionStatus.COMMITTED)
                throw new TestException("Action commit error.");
            else
                return;
        }

        A.abort();

        throw new TestException("Write lock error.");
    }

    public int get () throws TestException
    {
        AtomicAction A = new AtomicAction();
        int value = -1;

        A.begin();

        if (setlock(new Lock(LockMode.READ), retry) == LockResult.GRANTED)
        {
            value = state;

            if (A.commit() == ActionStatus.COMMITTED)
                return value;
            else
                throw new TestException("Action commit error.");
        }

        A.abort();

        throw new TestException("Read lock error.");
    }

    public boolean save_state (OutputObjectState os, int ot)
    {
        boolean result = super.save_state(os, ot);

        if (!result)
            return false;

        try
        {
            os.packInt(state);
        }
        catch (IOException e)
        {
            result = false;
        }

        return result;
    }

    public boolean restore_state (InputObjectState os, int ot)
    {
        boolean result = super.restore_state(os, ot);

        if (!result)
            return false;

        try
        {
            state = os.unpackInt();
        }
        catch (IOException e)
        {
            result = false;
        }

        return result;
    }

    public String type ()
    {
        return "/StateManager/LockManager/AtomicObject";
    }

    private int state;
}

The only real differences here are that we don't need the constructor call to LockManager with the ANDPERSISTENT flag (the default for all TXOJ objects is that they are recoverable) and it doesn't make sense to have a Uid constructor because there will never be any persistent state for this object, so it can never be recreated later. If you know that the object is only ever going to be recoverable then you may also be able to play tricks with state manipulation during save_state and restore_state calls, which is why there's a second parameter to them, which is either ANDPERSISTENT or RECOVERABLE.

But that's it. With these very minor changes to the code you can create atomic and recoverable objects that have a lot of the benefit of being used within a transaction, including transactional locking, but lacking the overhead of logging. And of course it's possible to mix recoverable and persistent objects in the same transaction seamlessly. But again, there's a price to be paid for turning off persistence. However, for some types of object and application this is a price worth paying, and one of those areas is STM. But it's not alone and you may find the idea of volatile transactional objects appealing for your own needs. In which case give it a go.

Nested transactions

Here's another periodic entry on our road to discussing the STM work we're doing around JBossTS. We're going to build on the Transactional Objects for Java toolkit that I described before. But before doing so it's worth stressing that everything I'm talking about here is based on the ArjunaCore module, so it doesn't need an ORB or SOAP stack. In fact it doesn't actually need anything from an application server, whether or not that is J(2)EE based. So yes, you could take that module and embed it within your application, framework or whatever and gain full ACID transaction capabilities.

Hopefully if you're reading these entries then you already understand the concept of traditional transactions, which we'll refer to as top-level transactions here. If not then there are enough articles, presentations and books to get you there, and the JBossTS documentation will help too. But suffice it to say that with a top-level transaction, you get the all-or-nothing (Atomic) affect: no matter how many resources are updated within the scope of that transaction, if one of them commits the work then all of them will and likewise, if one of them rolls back the work then all of the others will also roll back their work.

Now this is a great capability to have, especially if you have to do very little as a user to get there. However, for some applications and objects it can be restrictive. For instance, if your transactions last for a long period of time (say minutes), updating different resources throughout, and just at the very end there's a failure, then everything has to be done again. This and other limitations with top-level transactions lead us and others to look at various extended transaction models, and one of the earliest approaches was nested transactions from 1981, also known as subtransactions (not to be confused with subordinate coordinators).

Transactions which are contained within the other transaction are said to be nested (or subtransactions), and the resulting transaction is referred to as the enclosing transaction. The enclosing transaction is sometimes referred to as the parent of a nested (or child) transaction. A hierarchical transaction structure can thus result, with the root of the hierarchy being referred to as the top-level transaction. An important difference exists between nested and top-level transactions: the effect of a nested transaction is provisional upon the commit/roll back of its enclosing transaction(s), i.e., the effects will be recovered if the enclosing transaction aborts, even if the nested transaction has committed.

Subtransactions are a useful mechanism for two reasons:

• fault-isolation: if subtransaction rolls back (e.g., because an object it was using fails) then this does not require the enclosing transaction to rollback, thus undoing all of the work performed so far.


• modularity: if there is already a transaction associated with a call when a new transaction is begun, then the transaction will be nested within it. Therefore, a programmer who knows that an object require transactions can use them within the object: if the object’s methods are invoked without a client transaction, then the object’s transactions will simply be top-level; otherwise, they will be nested within the scope of the client’s transactions. Likewise, a client need not know that the object is transactional, and can begin its own transaction.

Imagine what you could do with nested transactions, particularly if you get to control them programmatically. You can start transactions within your application or objects without having to be concerned about whether or not there are transactions already in scope. If you do know and care, then you can utilize them for application needs. For instance, nested transactions give you the ability to try alternate paths in a decision tree without having to worry about the impact on transactional resources; you try each option in a separate nested transaction and roll back the paths you don't chose, relying on the transaction system to undo that work automatically, and then you simply commit the selected path (nested transaction).

So how do you use them? Well in JBossTS there are no special constructs for nesting of transactions: if a transaction is begun while another is running on the thread then it is automatically nested. This allows for a modular structure to applications, whereby objects can be implemented using transactions within their operations without the application programmer having to worry about the applications which use them, i.e., whether or not the applications will use transactions as well. Thus, in some applications those transactions may be top-level, whereas in others they may be nested. Objects written in this way can then be shared between applications and JBossTS will guarantee their consistency. If a nested transaction is rolled back then all of its work will be undone, although strict two-phase locking means that any locks it may have obtained will be retained until the top-level transaction commits or rolls back. As mentioned before, if a nested transaction commits then the work it has performed will only be committed by the system if the top-level transaction commits; if the top-level transaction aborts then all of the work will be undone.

What all of the above means is that if you want to use nested transactions in your applications then all you need to is start them programmatically as we saw earlier. So for example, if we wanted to use the AtomicObject we defined earlier within a top-level transaction, the code could look like:


AtomicAction A = new AtomicAction();
AtomicObject obj = new AtomicObject();

A.begin();

obj.incr(1);

A.commit();


The fact that the AtomicObject incr method uses a transaction internally is hidden from the invoker. In this case it'll be nested. If the code looked like ...


AtomicObject obj = new AtomicObject();

obj.incr(1);


... then the transaction would be top-level. Hopefully the differences in behaviour are pretty obvious by now, but just in case: in the former case we get a chance to undo the increment if we were to roll back the enclosing transaction, whereas in the latter, assuming the incr method didn't fail, then there is no opportunity to roll back; we would have to do an application driven undo which isn't guaranteed to be equivalent from an ACID perspective (e.g., another user could have seen the state between the increment and the decrement). And of course that leads us towards compensating transactions, but they are definitely out of scope for this article.

So I hope I've given you a flavour to nested transactions and some of the power they could bring to applications and objects. Why not give them a go?

Thank you HAL Corporation for FUD spreading

I mentioned a few months back how a certain group of people at a certain company that was conjectured to be the basis for HAL 9000 in 2001: A Space Odyssey were spreading FUD about JBossTS. Now not doing your homework is bad enough, but then building a whole marketing exercise around it is even worse! To say it's like building a house of cards is an understatement. It only takes someone to start questioning, probing and pushing for the whole thing to start to come tumbling down and the sheer stupidity of the original exercise comes to the fore.

Now comparing and contrasting projects, products or pretty much anything, should really be done on a scientific basis. (OK there are some things in life where that might be overkill or inappropriate, but not here.) It should be testable and reproduceable, ideally by anyone independent. If you're going to make statements about the pros and cons of this or that, then in some ways your reputation is being put on the line. Of course if you don't value your reputation then this is all moot. However, for JBoss and Red Hat, reputation and integrity are very important, so we're not going to say something bad or potentially maligning unless we have all of the facts, and even then we'll typically do it in a very scientific manner once again. This means that if we're wrong and it's because we made a mistake, e.g., had overlooked something, then we'll admit it.

Another thing we won't do is go around prospective customers spreading FUD, demonstrating FUD and making statements about "fundamental issues" with a competitor's product/project that mean it can never do X, Y or Z, unless there's proof, and in which case it's not really FUD. But of course our principles aren't shared by everyone. But when I heard that that was precisely what was happening it made their previous FUD attempts pale into relative insignificance. Despite my previous blog post, lots of real world deployments showing that recovery works, a history dating back to the 1980's which included being part of HP's foray into the middleware space, it wasn't enough for some who had been bedazzled by the IBM Roadshow (and to a degree you can't really blame them, since this is Big Blue after all.)

So we decided that enough was enough. Our honour had been besmirched (insert smiley). Fortunately recovery is so critical to a transaction system that we have lots of demonstrators and QA tests that cover pretty much every situation you can consider. And most of them are publicly available in our repository. Wow: who knew?! (Insert tongue firmly in cheek). So our teams (TS and QA) got together and in less than a day had produced something that duplicated the scenario, showed that recovery works, and therefore showed what a load of rubbish was being spouted.

Now of course it can be argued that we knew all of this to start with so we just wasted time and effort proving it yet again. I agree in part, and when this impacts on my holidays, as it did, it does make me a little bit mad. But I don't blame anyone other than the original mud slingers. Hopefully they'll learn something from this exercise. Hopefully they'll learn what open source is about and how to use it as well as contribute towards it. Unfortunately I have my doubts. So until the next time I'll leave you with this thought: what's that they say about glass houses?

Building transactional applications with JBossTS

Towards the end of last year I mentioned that we've been doing some work on STM around JBossTS. I said I'd give some hints at what we've been doing, but in order to do so we need to cover some background on how you can construct transactional applications with the existing code using Transactional Objects for Java (TXOJ). You'll find a lot more information about what I'm going to summarize in the documentation and various papers.

In order for your applications to possess all of the necessary ACID properties when using transactions, the objects manipulated by those transactions need to be both durable and isolated from conflicts. If your application objects aren't backed by a database, then you have the headache of providing these capabilities yourself, unless you have your friendly neighborhood transactional framework like TXOJ. Without going into all of the details (which really are in the documentation) probably the most important class is StateManager which provides all of the basic support mechanisms required by an object for state management purposes. As with concurrency control, which we'll come on to later, durability is obtained through inheritance.

Objects in TXOJ are assumed to be of three possible basic flavours. They may simply be recoverable, in which case StateManager will attempt to generate and maintain appropriate recovery information for the object. Such objects have lifetimes that do not exceed the application program that creates them. Objects may be recoverable and persistent, in which case the lifetime of the object is assumed to be greater than that of the creating or accessing application so that in addition to maintaining recovery information StateManager will attempt to automatically load (unload) any existing persistent state for the object at appropriate times. Finally, objects may possess none of these capabilities in which case no recovery information is ever kept nor is object activation/deactivation ever automatically attempted.

There are quite a few StateManager methods that your application could utilize directly, but for now the only three that are of immediate importance are save_state, restore_state and type. The first two methods are responsible for saving (or restoring) the state of the object, whereas the last is used when positioning the state in the transactional object store. Now although the store may also be the location of the transaction log, the application object states aren't necessarily maintained within the log. (Yes, I'm glossing over subjects such as activating and passivating objects, but we may come back to those in a later posting.)

With these three StateManager methods defined you could manage the state of your object's within the scope of a transaction. But typically you'll also want isolation, so let's move on to LockManager, which inherits from StateManager. So your application objects can become isolated and durable by having their corresponding classes inherit from LockManager. Although you have to be concerned about the three StateManager methods mentioned earlier, there's nothing else here that you need worry about in terms of overriding or defining. But of course you do need to tell the system when and how to isolate your objects. The primary programmer interface to the concurrency controller is via the setlock operation. By default the runtime enforces strict two-phase locking following a multiple reader, single writer policy on a per object basis. Although lock acquisition is under programmer control, lock release is normally under control of the system and requires no further intervention by the programmer. This ensures that the two-phase property can be correctly maintained. (Again, I'll ignore subjects like type specific concurrency control for this posting.)

But sometimes a picture paints a thousand words, so here's an example:

public class AtomicObject extends LockManager
{
    public AtomicObject ()
    {
        super(ObjectType.ANDPERSISTENT);

        state = 0;

        AtomicAction A = new AtomicAction();

        A.begin();

        if (setlock(new Lock(LockMode.WRITE), 0) == LockResult.GRANTED)
        {
            if (A.commit() == ActionStatus.COMMITTED)
                System.out.println("Created persistent object " + get_uid());
            else
                System.out.println("Action.commit error.");
        }
        else
        {
            A.abort();

            System.out.println("setlock error.");
        }
    }

    public AtomicObject (Uid u)
    {
        super(u, ObjectType.ANDPERSISTENT);

        state = -1;

        AtomicAction A = new AtomicAction();

        A.begin();

        if (setlock(new Lock(LockMode.READ), 0) == LockResult.GRANTED)
        {
            System.out.println("Recreated object " + u);
            A.commit();
        }
        else
        {
            System.out.println("Error recreating object " + u);
            A.abort();
        }
    }
    
    public void incr (int value) throws TestException
    {
        AtomicAction A = new AtomicAction();

        A.begin();

        if (setlock(new Lock(LockMode.WRITE), retry) == LockResult.GRANTED)
        {
            state += value;

            if (A.commit() != ActionStatus.COMMITTED)
                throw new TestException("Action commit error.");
            else
                  return;
        }

        A.abort();

        throw new TestException("Write lock error.");
    }

    public void set (int value) throws TestException
    {
        AtomicAction A = new AtomicAction();

        A.begin();

        if (setlock(new Lock(LockMode.WRITE), retry) == LockResult.GRANTED)
        {
            state = value;

            if (A.commit() != ActionStatus.COMMITTED)
                throw new TestException("Action commit error.");
            else
                return;
        }

        A.abort();

        throw new TestException("Write lock error.");
    }

    public int get () throws TestException
    {
        AtomicAction A = new AtomicAction();
        int value = -1;

        A.begin();

        if (setlock(new Lock(LockMode.READ), retry) == LockResult.GRANTED)
        {
            value = state;

            if (A.commit() == ActionStatus.COMMITTED)
                return value;
            else
                throw new TestException("Action commit error.");
        }

        A.abort();

        throw new TestException("Read lock error.");
    }

    public boolean save_state (OutputObjectState os, int ot)
    {
        boolean result = super.save_state(os, ot);

        if (!result)
            return false;

        try
        {
            os.packInt(state);
        }
        catch (IOException e)
        {
            result = false;
        }

        return result;
    }

    public boolean restore_state (InputObjectState os, int ot)
    {
        boolean result = super.restore_state(os, ot);

        if (!result)
            return false;

        try
        {
            state = os.unpackInt();
        }
        catch (IOException e)
        {
            result = false;
        }

        return result;
    }

    public String type ()
    {
        return "/StateManager/LockManager/AtomicObject";
    }

    private int state;
}

What we have here is a durable and isolatable integer. It may look like quite a bit of code, but there's a lot of defensive programming here to illustrate explicitly some of the problems that could occur. It's left as an exercise for the reader to optimize the code.

However, what you can see here is the use of AtomicAction for creating transactions (which will be automatically nested if there's already a transaction associated with the thread of control) and then the acquiring of the right types of Lock within the application object code (READ or WRITE to signify whether or not the state of the object may be modified). If you consider what's needed to ensure that the object is transactional, including full recovery, there isn't a lot for the developer to do. OK, it's a bit more than you might expect if you're used to EJB3s, for example, but then there's a bit more going on here, and some of it could well be automated (future posting hint). But when you consider that an entire database system was built on this basis back in the early 1990's (OK, the C++ version of the transaction system and not Java, but very little difference), it helps to illustrate the power of the abstractions as well as the implementation.

OK, that's enough for now. In a later post we may go into more detail on some of the concepts mentioned here (or glossed over so far) before heading off into STM land. If there's anything specific you'd like concentrating on then just suggest via the comments.

Transactions: Over used or just misunderstood?

I gave a presentation at QCon London last week on the above subject. I was part of the Solutions Track and QCon have made the slides available. The room was packed out, which was good though always a little surprising: although transactions are dear to my heart it's not something that lights everyone's fire! It seemed to go down well and there were plenty of questions during and afterwards to convince me that I probably have enough material to do 4 or 5 deep dives into the issues raised by the current presentation. So maybe at the next JUDCon event you'll be able to hear me present on the duality between successful marriages and two-phase commit!

REST-Atomic Transaction

We've been working a lot on updating the REST transactions work for Bill's REST-* initiative. The latest drafts aren't up there at the moment but are accessible through the mail archives. The current draft 3 of the atomic model is still called REST-ACID due to some WS-ACID history, but the next draft will see a name change to REST-Atomic Transaction to more clearly call out what the model defines. Then of course there's the compensation based model that will see a new draft going up soon.

On a slight side note it's good to see others thinking about the same problem. Though I think this model has some issues it's good that others have recognized that the problem remains undefined within REST. Of course Eric and Phil wrote about some of this in their updated book (worth a read).

Someone at IBM needs to do their homework!

I have a lot of respect for IBM, with many good friends and colleagues who work there, or have worked there over the years. I've collaborated with them for a long time on various transaction standards, articles and books. They know a thing or two about transactions, being the home of CICS.

However, sometimes they slip up as do we all. Case in point is this quality article (yes, I'm being sarcastic) from some anonymous author(s) over there in IBM Land. It's a good example of FUD and a really bad example of scientific investigation. I'm not going to help them improve their act on getting those two things to work together, but I will point out a few areas that quickly made me consign the document to the trash.

For a start, making statements like "JBoss AS has a relatively new integration with a third-party transaction manager." and "Public forums seem to indicate that this transaction manager is not proven" could easily be fixed by just reading the community pages. But then I get the impression that the author(s) didn't go near the community pages much except to find "facts" to fit their aims, carefully ignoring anything that didn't fit (not good scientific method!)

Then they concentrate on a specific AS issue which is about configuring AS to automatically recover datasources. Manual editing of files is required at the moment. The author(s) didn't do their homework (or even bother posting on the free forums as far as I can tell) to locate the information they needed. Now maybe they're relatively new to open source and didn't realise you don't have to buy a support contract or buy the software in order to talk with the developers! Maybe even reading the documentation might have helped, although perhaps there was just far too much of it for the author(s).

But does this mean that there are fundamental problems with JBossAS or JBossTS around data integrity? Not that we're aware of and there are enough QA tests and samples to justify that position. The code for any tests that would suggest otherwise is conveniently absent from the author(s) document. If anyone has it or something similar then be a good community member and send it our way. Of course there may be things we've missed (even CICS has issues to this day). Until then I suggest doing your bit for the environment and not waste paper, ink or energy on this article. Your Carbon Footprint will thank you.