Monday, April 8, 2013

API Improvements for Compensation-based Transactions

In a recent post I talked about API improvements we are introducing for applications that use ACID transactions. In this post I’ll cover what are we doing for users of compensation based transactions.

Even though we only have WS-BA for compensation-based transactions at the moment, we are still following the principle that the same transaction API should be used to develop the application, regardless of what transport is actually used to distribute the transaction. This will become more important when we support compensations over other technologies, such as REST or JBoss remoting.

Unfortunately there is no standard API for compensation based transactions, so we needed to develop our own. This API is still in the early stages of development. However, we are keen to get community feedback, so we made the early version available as part of our recent Narayana 5.0.0.M2 release.

I’ll cover the basics of the API in this post; so as to give you a feel for what we are proposing. You can take a look at the Narayana quickstarts for more complete examples. We also intend to blog more on this subject in the coming months as we develop our ideas further.

I’ve omitted a recap on compensation-based transactions and when you need them. This will be the subject of a future blog posting.

Example

Hopefully this example will give you an idea of how the new API works

The following code comprises part of a "Warehouse Service", implementad as an EJB exposed as a JAX-WS Web Service:


@Compensatable
@Stateless
//JAX-WS annotations omitted
public class WarehouseServiceImpl implements WarehouseService {
{
    @DataManagement private Map txDataMap;
    @PersistenceContext protected EntityManager em;

    @WebMethod
    @ServiceRequest
    public void shipItem(String item, String address) throws {
        //Use em to add order to DB
        txDataMap.put("orderID", orderID);
    }

    @Compensate
    private void cancelOrder() {
        Integer orderID = txDataMap.get("order");
        //Use em to lookup order by ID and cancel it
    }
}

The @Compensatable annotation is used to state that methods on the class should be invoked in a compensation-based transaction. This is similar to the @TransactionAttribute provided by JTA.
The ‘shipItem’ method represents the business logic of the service. It has a corresponding compensation handler which can be used to undo this work.

When ‘shipItem’ is invoked, a new entry is added to the database. This update is done in a regular JTA transaction that commits when the method completes successfully. The ID of the order is saved in the ‘txDataMap’ to be used later by the compensation handler.

The @DataManagement injected Map stores state for the lifetime of the transaction. The state is isolated to the transaction and is garbage collected when the transaction ends. In future releases, this data will also be available at recovery time.

@Compensate annotates the compensation handler for undoing the work done in the business method. In this example, the compensation action is to look up the ID of the order and then make an update to the DB to mark it as cancelled. This DB update is done in a separate transaction that commits when ‘cancelOrder’ completes successfully.

The current release only supports a single @Compensation method for all @ServiceRequest methods in the class. Subsequent releases will remove this limitation.

Getting Started

Hopefully you are now eager to get started and want to know where to go next! Here are our suggestions:

* Download and try the Quickstarts from here.
* Provide feedback and get help through our forum.
* Track the progress of issues here.
* Subscribe to this blog.
* Fork the Narayana repo and contribute. Of course, we always welcome community contribution. We can advise on good issues for new contributors, or you can suggest a feature that interests you.



Acknowledgements

I'd like to say a bit thank you to Alessio Soldano and the JBossWS team. They provided a lot of advice and also added new features to the JBossWS SPI to support these features.

Sunday, March 31, 2013

STM, vert.x and the Pi Part 1

Given other priorities (aka 'my day job'), I'm slowly working my way through another pet project I've had for a while: adding transactions, and specifically STM, to vert.x. Of course at the moment I also try to add an influence from my other pet project, the Raspberry Pi, so I'm doing it on the Pi as much as possible. OK, it may not be the fastest machine on the block, but it's a lot of fun!

We've had STM in Narayana for a while, so I won't cover the basic again. I'm also not going to cover any Pi-related set up: by this point you should know enough to do this yourself. So let's start with cloning the most recent version of Narayana:


Make sure your path and JAVA_HOME are set so we're using JDK 7:

 
Next let's build the components we need for STM:


Don't forget that this is going to take a while, so maybe time to go grab a coffee, read a book, play with your kids, or watch an episode of Lost.


With that build complete, we've already run a lot of STM examples in the form of unit tests. However, let's take a simple example:

import java.io.IOException;

import org.jboss.stm.annotations.Transactional;
import org.jboss.stm.annotations.ReadLock;
import org.jboss.stm.annotations.WriteLock;
import org.jboss.stm.internal.RecoverableContainer;

import com.arjuna.ats.arjuna.AtomicAction;

/**
 * @author Mark Little
 */


public class Example
{  
    @Transactional
    public interface Atomic
     {
     public void change (int value) throws Exception;
       
     public void set (int value) throws Exception;
       
     public int get () throws Exception;
    }
   
    @Transactional
    public class ExampleSTM implements Atomic
     {  
        @ReadLock
     public int get () throws Exception
     {
         return state;
     }

        @WriteLock
     public void set (int value) throws Exception
     {
         state = value;
     }
       
        @WriteLock
     public void change (int value) throws Exception
     {
         state += value;
     }

     private int state;
    }
   
    public void testExample () throws Exception
    {
        RecoverableContainer theContainer = new RecoverableContainer();
        ExampleSTM basic = new ExampleSTM();
        Atomic obj = null;
       
        try
        {
        obj = theContainer.enlist(basic);
        }
        catch (final Throwable ex)
        {
        ex.printStackTrace();
           
        return;
        }
       
        AtomicAction a = new AtomicAction();
       
        a.begin();
       
        obj.set(1234);
       
        a.commit();

    System.out.println("Should get() 1234 after commit: "+obj.get());
       
        a = new AtomicAction();

        a.begin();

        obj.change(1);
       
        a.abort();

    System.out.println("Should get() 1234 after abort: "+obj.get());
    }

    public static void main (String[] args)
    {
    Example ex = new Example();

    try
    {
        ex.testExample();
    }
    catch (final Exception e)
    {
        e.printStackTrace();
    }
    }
}


Stick this at the same level of Narayana if you want to use the same CLASSPATH as here:

narayana/ext/jboss-logging.jar:narayana/ArjunaCore/arjuna/target/test-classes:narayana/common/target/common-5.0.0.M3-SNAPSHOT.jar:narayana/STM/target/core-5.0.0.M3-SNAPSHOT.jar:narayana/ArjunaCore/txoj/target/txoj-5.0.0.M3-SNAPSHOT.jar:narayana/ArjunaCore/arjuna/target/arjuna-5.0.0.M3-SNAPSHOT.jar:.

After compiling, run the example and you should see the following:


Despite what I said at the start of this entry, there's a distinct lack of vert.x involved here at the moment. That's because there are a few changes we need to make to the STM implementation to make it easier to share transactional objects between address spaces. This is all stuff that's available "natively" in TXOJ, so developers could use that API immediately. However, the STM approach simplifies a lot so once those additional features from TXOJ are integrated there, we'll have a follow up article to discuss what, how and why.

Monday, March 25, 2013

API Improvements for WS-AT and REST-AT

In this post I’ll introduce some API changes we are doing to improve the way developers use WS-AT and REST-AT.

I’m a strong believer that the internal architecture of your application, and the means by which client’s communicate with it, are two orthogonal issues. You wouldn’t want to have to change the internals of your application, just because you need to change the way clients invoke your service.

We think you should be able to develop your applications using a common Transactions API. You should then be able to invoke remote services over whatever transport is appropriate and automatically have the transaction distributed. Furthermore, you shouldn’t be restricted to having all participants use the same transport.

There really is only one option for a Java “common Transactions API” - it’s JTA. JTA is well supported in Java EE application servers and it is well understood by developers. Furthermore, there are many applications out there already using JTA.

This is what we are moving towards with each milestone release of Narayana 5.0.0. With milestone two we made it very simple for a pure JTA client to invoke a JTA application over Web Services in a distributed transaction. Take a look at the following example:

Client


@Stateless
public class OrderClient {

  @TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
  public void orderItem(String item, String address, double amount) {

    AccountService as = //Lookup AccountService WS client
    WarehouseService ws = //Lookup WarehouseService WS client

    ws.shipItem(item, address);
    as.invoiceCustomer(address, amount);
  }
}


Here the client is using a regular Stateless Session Bean. The method ‘orderItem’ is invoked in a JTA transaction that is committed if the method succeeds and rolled back if it fails.

The calls to the Warehouse and Account services automatically use WS-AT to distribute the JTA transaction. The middleware knows to do this because a JTA transaction was present when the Web Service call was invoked.

Transaction propagation can be configured, but that’s the subject of another blog post (stay tuned).

Service


@Transactional //Currently required, will be removed soon https://issues.jboss.org/browse/JBTM-1468
@Stateless
//JAX-WS annotations omitted for brevity
public class WarehouseServiceImpl implements WarehouseService {

    @PersistenceContext
    protected EntityManager em;

    @WebMethod
    public void shipItem(String item, String address) {
        // Use the Entity Manager to 
        // add an item and shipping address to DB
    }
}

Again the service is implemented as a simple Stateless Session bean that just happens to be offered as a Web Service. Here the application is using JTA to make a database update. The middleware automatically handles the mapping between the incoming WS-AT transaction and the JTA transaction.

What About Other Transports?

With milestone two we already support transparent propagation over Web Services (via WS-AT), Corba (via JTS) and JBoss Remoting. We hope to have REST-AT support in a subsequent milestone release (see JBTM-1468).

Getting Started

Hopefully you are now eager to try this out. The best place to look is at this quickstart, which demonstrates the new API in action.


Acknowledgements

I'd like to say a big thank you to Alessio Soldano and the JBossWS team. They provided a lot of advice and also added new features to the JBossWS SPI to support this feature.

Thursday, March 21, 2013

Narayana and BlackTie 5.0.0.M2 Released!

I am very proud to be able to announce the availability of version 5.0.0.M2 of the Narayana and BlackTie projects which has been a significant release for us on many levels.

One of the greatest achievements for this release is that it is the first version of Narayana (JBoss Transactions 5) that has been consumed by the AS8 project. This means you can start using it straight away by downloading the latest nightly build of AS8!


Narayana's integration with the app server contains the usual bits and pieces from JBoss Transactions (JTA, JTS, XTS) alongside our new APIs for using XTS. It also provides a suitable base line to deploy the BlackTie and REST-AT server-side components. With future milestone releases of Narayana we will add full out-of-the-box support for these extra components.

Some other features that have been added in this version (release notes here and here) which I would like to pick out are:
  • Added a significantly improved API for using XTS in the app server (see separate blog posts for more details)
  • Early release of a new API for compensation-based transactions
  • JDBC Object Store
  • Simplified BlackTie client library (without TAO dependency)
  • Windows 2008 Server support for BlackTie
  • The usual set of enhancements for our documentation and quickstarts

So, where can you download the software from? For Narayana standalone, you can download the software here:
https://www.jboss.org/jbosstm/
For BlackTie, it is available here (a Linux 64 bit distribution and a Windows 32 bit one are provided):
https://www.jboss.org/blacktie/
For a build of AS8 containing Narayana:
https://ci.jboss.org/hudson/job/JBoss-AS-latest-master/lastSuccessfulBuild/artifact/build/target/jboss-as-8.x.zip

As I mentioned on a previous blog, we are going to be merging the BlackTie and Narayana repos/Jira instances/blogs etc in the M3 timeframe, so in future everything will be available from the existing Narayana locations:

Source:
https://github.com/jbosstm/narayana (plus documentation and quickstart )

Issue management:


https://jira.jboss.org/browse/JBTM (use the blacktie component if raising a blacktie issue)

Please do let us know what you think of the release either in the comments on our blog (http://jbossts.blogspot.co.uk/) or forum (https://community.jboss.org/en/jbosstm/?view=discussions).

Happy evaluating!

Friday, March 1, 2013

Merging BlackTie and Narayana

Guys,

We would like to merge the BlackTie and Narayana projects together. As you will know, BlackTie is a C++ API onto the Narayana transaction manager and as such features are being developed which require changes in both components. Furthermore, Narayana already provides support for several other APIs within the main Narayana project so collecting them all together seems to make the most sense.

The changes would be two phase:

Phase 1:
1. Merge the blacktie github repo into the narayana github repo
2. Use a single blog: http://jbossts.blogspot.co.uk/
3. Use a single jira instance: https://jira.jboss.org/browse/JBTM
4. Use a single chatroom: jbossts@irc.freenode.net
5. Use the single community space on jboss.org: https://community.jboss.org/en/jbosstm/

Phase 2:
Merge our web site https://www.jboss.org/blacktie/ into the existing https://www.jboss.org/jbosstm pages

Please do let us know if you have any objections or can see a problem with this,

Tom

Wednesday, January 30, 2013

WS-BA Participant Completion Race Condition: Part Two


The Details

In a previous post, I described a benign race condition that in unusual circumstances can cause some Business Activities to be cancelled that would have otherwise been able to close. In this post I'll go into the details of how this happens.

First consider the following client code:

UserBusinessActivity uba = UserBusinessActivityFactory.userBusinessActivity();
uba.begin();
myWebServiceClient.invoke();
uba.close();


The client code is very simple, it just begins a business activity, invokes a Web service and then closes the business activity. The Web service uses the Participant-Completion protocol and so notifies the coordinator of completion just before returning control to the client.


Here's a diagram showing the pertinent message exchanges that occur under a normal situation.




The messages are numbered to indicate the order in which they are sent.
  • 1. request. This represents the application request made by the client.
  • 2. completed. After the participant has completed its work, it notifies the coordinator that it has completed.
  • 3. response. This represents the response to the client's application request.
  • 4. close. The client notifies the coordinator that it wishes to close the activity. It then waits for a 'closed' or failure response from the coordinator.
  • 5a. close/5b. closed. The coordinator has processed the '2.completed' message so can close the activity. It starts by sending the 'close' message to the participant and waits for the 'closed' response as confirmation. These two messages are asynchronous.
  • 6. closed. The coordinator now has all 'closed' acknowledgements so notifies the client that the activity successfully closed.

Messages '2.completed' and '4.close' are asynchronous (or 'one way' in Web services parlance) so effectively, we have a race condition with the following competing parties:
  • Party 1. The completed message '2.completed'.
  • Party 2. The response '3.response' followed by '4.close'.

When running in the same VM, or on a low latency network, '3.response' will be sent very quickly. This is because it is simply travelling on the HTTP response over an already open socket. This just leaves messages '2.completed' and '4.close' which will take much longer relative to '3.response'. To understand this, lets take a look at what happens when an asynchronous Web service call is made:

  1. The client sends the message to the Web service
  2. The server-side SOAP stack uses an existing thread from a pool dedicated to receiving SOAP messages.
  3. As the service is asynchronous, the message will be passed to another thread to be processed.
  4. The receiving thread will now return the HTTP response.
The race condition occurs because steps 1-3 can happen relatively quickly in a single VM, and thus it's likely that both messages 2 and 4, will be waiting to be processed at the same time. The order in which they are processed is dependent on the implementation of the thread pool and is also at the mercy of thread scheduling in the VM, so it's possible that either could be processed first.

This race condition is much less likely to happen in a distributed environment as the network costs will be significantly higher. As a result message '3.response' will take long enough to send, so as to give message '2.completed' enough of a head start. But it is still possible so the client application must be coded defensively to catch and handle a TransactionRollbackException. Your code ought to be doing this anyway to deal with server crashes.

Here's a diagram showing what messages are exchanged when the race condition occurs. You will see that the activity ends in a consistent state.



I've omitted messages 1-3 from the following explanation as they are the same as in the success case.
  • 4. close. This message is processed by the coordinator before message '2.completed'
  • 5a. cancel. The coordinator has not yet processed the '2.completed' message so cannot close the activity. The coordinator then sends a 'cancel' message to the participant as it thinks it has not yet completed. This message and subsequent retires, are dropped by the participant as they are not valid for a completed participant.
  • 5b. compensate/5c. compensated. After one or more unacknowledged 'cancel' messages, the coordinator switches to sending 'compensate' messages which will cause the participant to compensate the work. The participant acknowledges with a 'compensated' reply.
  • 6. Transaction rolledback exception. The coordinator notifies the client that the activity failed to close.

As you can see from the steps above, when this race condition arises, any work done by participants is compensated and the client is notified of the outcome. Thus a consistent outcome is achieved.

Thursday, January 24, 2013

WS-BA Participant Completion Race Condition: Part One

Overview

The WS-BA participant-completion protocol has a benign race condition that, in unusual circumstances, can cause some Business Activities to be cancelled that would have otherwise been able to close. This is safe as no inconsistency arises, but it can be annoying for users. This blog post explains why this can happen, under what conditions, and what you can do to tolerate it. This post gives you an overview of the issue and should provide enough details for most developers. A follow up post will get into the nitty-gritty details of what's really going on.

What's happening, in a nutshell

Imagine a scenario where the client begins a business activity and then invokes a Web service. If the Web service uses participant completion, it will notify the coordinator when it has completed its work and then return control to the client. This notification is asynchronous, so it's possible that the client will then ask the coordinator to close the activity before the coordinator processes (or even receives) the completed notification from the participant. In this situation the coordinator will cancel the activity as not all participants (from its perspective) have completed their work. As a result all completed participants are compensated (including, eventually, the participant with the late 'completed' notification) and the client receives a "TransactionRolledBackException".

When is it most likely to happen?

Typically this happens when the client, coordinator and participant are running inside the same VM. This scenario is unlikely to happen in production, but can happen regularly during development where a single VM is used to keep things simple.

How do I know if this is affecting my application?

If the client is occasionally receiving a TransactionRolledbackException when calling UserBusinessActivity#close(), but none of the machines involved in running the transaction have crashed, you could be affected by this. Especially if you are running the client, coordinator and participant(s) in the same server.

We've now added a log message to help you identify this. However, to see this, you will either need to be building transactions from the current source (4.17 or master branches in GitHub) or wait for the JBossTS 4.17.4 or Narayana 5.0.0.M2 release. This is the log message to look out for:

WARN  [com.arjuna.mw.wstx] (TaskWorker-2) ARJUNA045062: Coordinator cancelled the activity

This is only an indication that you are seeing this issue as the coordinator can elect to cancel the activity for other reasons. For example, network problems might mean the coordinator cannot tell the web service to close the activity.

Why can't this be avoided?

The short answer is that for the protocol to avoid this it would need to make the complete message synchronous, throttling throughput by slowing down both the participant and coordinator and holding sockets open for longer.

What can the application do to tolerate this?

A real, distributed deployment will rarely see this problem because communication latency between client, participant and coordinator will dominate the race condition. Even if it does happen your application should tolerate it. Transaction rollbacks and activity cancellations are inevitable in a distributed environment and can happen for many reasons. When handling TransactionRolledBack exceptions you can either retry the Transaction/Activity or notify the caller of the failure. What you choose to do will depend on the requirements of your application.

In part two, I'll get into the details of what's happening.