HPTS 2013

I was at HPTS 2013 a week or so ago. It's probably the one conference that I always try to get to no matter what. I've included my presentation here but you should check out the HPTS site to see all of the others. The site isn't complete yet, as some presenters haven't sent in their decks but we'll be updating it as they come in.

Narayana is now JTA 1.2 Compliant

I'm very proud to announce that with Narayana 5.0.0.M3 we are now JTA 1.2 compliant! What's more, with the release of JCA 1.7 in WildFly 8.0.0.Beta1 (WFLY-510), the new application server features are now available.

I'd like to say a big "thank you" to the community members who provided feedback during the design and implementation of this specification. Like most of our larger features, this was discussed over at the Narayana developer forums, giving the community the opportunity to shape the development of these features.

So what's new in JTA 1.2?

JTA 1.2 is considered a 'maintenance release', so the change list is quite small. However, that's not to say that the changes aren't exciting. Here's an overview of the three improvements:

@Transactional. This is the headline feature and brings the ability to place transaction annotations on any CDI managed bean. Prior to this feature, you had to make your class an EJB in order to use transactional annotations.
@TransactionScoped. This feature allows you to associate CDI beans with the scope of a transaction.
The third change clarifies when the container should call 'delistResource' on the transactional resource. This is a minor change and is of less interest to an application developer, so I'm not going to discuss it further here.

Tell me more about @Transactional and @TransactionScoped

The remainder of this post will tell you what you need to know about these new features and provide some examples, showing how they can be used.

@Transactional

This new annotation provides an alternative to javax.ejb.TransactionAttribute that can be placed on CDI managed bean classes and its methods. Prior to this feature, many developers were using EJBs just so that they could use declarative transactions. Now you can make an architectural decision of wether EJB or CDI is the right approach for your application, knowing that you will be able to use declarative transactions with either approach. The remainder of this section will show an example using @Transactional and highlight the differences between @Transactional and the EJB @TransactionAttribute.


@Transactional(Transactional.TxType.MANDATORY)
public class MyCDIBean {

    @Transactional(Transactional.TxType.NEVER)
    public void doSomethingWithoutATransaction() throws Exception
    {

        //Do something that must be done outside of a transaction
    }

    @Transactional(
         dontRollbackOn=MyNonCriticalRuntimeException.class,
         rollbackOn=TestException.class)
    public void doSomething() throws Exception {
//Do something that must be done inside a transaction
    }
}

The "@Transactional(Transactional.TxType.MANDATORY)" annotation on the class states that, by default, all methods must be invoked within the scope of a JTA transaction. This can be overridden on a per-method basis.

The 'doSomethingWithoutATransaction' method overrides the 'MANDATORY' type with "NEVER". Therefore, this method will fail if it is invoked in the scope of a JTA transaction.

Specifying what Exceptions should cause the transaction to rollback is specified differently with @Transactional. With EJB's @TransactionalAttribute, the developer had to declare on the actual Exception class whether an exception of that type, thrown from a method annotated with @TransactionalAttribute, should cause the active transaction to be marked for 'rollback only'. The problem with this approach was that it could not (easily) be applied to third-party exception implementations and it also applied for all usages of that exception. @Transactional fixes these issues by allowing the developer to specify on a per-method basis, which Exception types (and sub-classes of) should cause a rollback. This is done through the 'rollbackOn' and 'dontRollbackOn' attributes. By default, RuntimeException and its subclasses will cause a rollback.

Another difference to watch out for is the default behaviour when no transaction annotation is provided. With an EJB, the default value of TransactionalAttribute is TRANSACTION_REQUIRED. This means that a new JTA transaction is begun (when calling the method) if one doesn't already exist. This works for EJB as the developer has already opted-in through the use of an EJB annotation (@Stateless, @Statefull, etc). With CDI, a managed bean can have no annotations, thus it is difficult to differentiate between it and a regular POJO. Therefore, the default value for @Transactional is TxType.SUPPORTED. This means that the method will run within a transaction if one exists; otherwise it will run without one. This is essentially how transactions are handled with java POJOs.

@TransactionScoped

With @TransactionScoped brings an additional CDI context to accompany @SessionScoped, @RequestScoped and @ApplicationScoped. Annotating a bean with @SessionScoped ensures that the same instance of the bean is made available for all usages of it within the scope of a http session. This allows the developer to easily share state between multiple requests within a session, whilst also isolating it from other requests in a different session.

@TransactionScoped allows data to be shared between all usages of the bean within a particular transaction, whilst isolating the instance from other accesses of the bean within a different transaction. The TransactionScoped bean instance's lifecycle matches that of the transaction.

The following code shows an example of this in use:

@TransactionScoped
public class MyCDITransactionScopeBean implements Serializable {

    private int value = 0;
    public int getValue() ... 
    public void setValue(int value) ...
}

MyCDITransactionScopeBean represents the data to be associated with the active transaction. It is simple POJO annotated with @TransactionScoped and also marked as Serializable.

public class SomeClass {

    @Inject
    MyCDITransactionScopeBean myBean;
...
}

MyCDITransactionScopeBean can then be injected into other classes that need to use it.

Go and give it a try!

Just download the latest version of WildFly and start deploying code. We don't yet have any quickstarts for these features, but our Arquillian tests provide a complete example of how to use the functionality.

Get involved with Narayana!

If you are interested in getting involved in the open source community we would love to hear from you! We are always looking for new contributors to help out with the broad selection of enhancements and features we are looking to add to our project.
The best approach to take is to have a look at our Jira instance and see if there are any items on there that you would like to see added to the project. Our Jira instance is located over at: https://issues.jboss.org/browse/JBTM where we have all of the dev tasks waiting; from development to build management, and from documentation to JDF style quickstart writing.

Once you have picked out a task you would like to work on, feel free to have a chat with us over in our chatroom hosted on Freenode: http://webchat.freenode.net/?channels=#jbossts then (and if you haven't done so already) you will need to fork one of our projects over on github:

• For core related, it will be: https://github.com/jbosstm/narayana/
• If its docs it will be: https://github.com/jbosstm/documentation/
• For quickstarts, it will be: https://github.com/jbosstm/quickstart

Once you have developed your contribution, first make sure you have signed the JBoss Contributor License Agreement over here: https://cla.jboss.org/, then raise a Pull Request on our of our repos.
A good pull request has the following qualities:

• Ideally, each pull request is focused on a single functional area
• Each commit is clearly labeled with an issue number from Jira
• Where reasonable, each Jira issue is tackled in a single commit - time to squash!

After you raise a pull request on one of the Narayana repos, one of our team of highly skilled "buildbots" will take a look over it and run our comprehensive test suite against it. The buildbot will keep your pull request up to date with comments as it works its way through the various phases of test and hopefully at the end will be able to indicate a complete build success!

One of the devs will then be able to merge your update and your place in history as a contributor on the project is secured!

Narayana 5.0.0.M4 and JBoss Transactions 4.17.9.Final released!

Hi,

It is with great pleasure that I can announce Narayana 5.0.0.M4 has been released and integrated into the WildFly application server master branch! Maven central has been updated with the release, plus the binary zips and docs are now available here:
http://www.jboss.org/jbosstm

Source tags are located here:
https://github.com/jbosstm/narayana/tree/5.0.0.M4
https://github.com/jbosstm/quickstart/tree/5.0.0.M4
https://github.com/jbosstm/documentation/tree/5.0.0.M4

The release notes are available here:
https://issues.jboss.org/secure/ReleaseNote.jspa?version=12321873&styleName=&projectId=12310200&Create=Create&atl_token=AQZJ-FV3A-N91S-UDEU|38dea2e03ccddd2835723190cd876cad2317ad03|lin

Alongside the usual incremental fixes and enhancements, we are also happy to announce that the BlackTie project is now fully merged into the Narayana code base.

This is the penultimate milestone release for Narayana 5. We are intending to release Beta1 in October and the .Final shortly afterwards.

In other news, we recently released 4.17.9.Final of JBoss Transactions containing a few bug fixes, the release notes are over here:
https://issues.jboss.org/secure/ReleaseNote.jspa?version=12322397&styleName=&projectId=12310200&Create=Create&atl_token=AQZJ-FV3A-N91S-UDEU|394366ea879c98614444896710bd0831b550e086|lin

Enjoy,
Tom

Out of the Box Support for RESTful Transactions

The latest release of narayana includes a couple of great additions to the REST-AT module that represent a step change in the usability of restful transactions. The work has been contributed by transactions' team member Gytis Trikleris.

Mark has previously blogged about why RESTful transactions are useful and our implementation of the draft REST-AT specification is steadily finding real world uses (within BlackTie for example). However, although using REST-AT is easier than building upon a full web services stack containing support for WS-Atomic Transaction, there are still barriers to overcome such as making sure the REST-AT coordinator is managed and installed correctly, ensuring the coordinator endpoint is deployed and the responsibilities of the service writer to conform to the specification are certainly not trivial. Gytis' work addresses both deficiencies.

REST-AT Now Installs as a Wildfly Extension

Wildfly extensions provide a mechanism for extending the functionality provided by the Wildfly application server. Installing the REST-AT coordinator via an extension not only means that the coordinator will be available to all client applications but it will also benefit from the same management and configuration enjoyed by all subsystems. The application developer no longer needs to be concerned with deploying the coordinator, with managing versions of the coordinator, with managing conflicts with other applications that require REST-AT, nor does he need to ensure that any dependencies such as JAX-RS support or web services are available and compatible. The feature is available for use now in the latest version 8.0.0.Alpha3 of wildfly.

Simple Interface for Participating in REST-AT Transactions

Until now the service developer has needed to understand and comply with the requirements of the REST-AT specification including the responsibility of listening for transaction completion requests on HTTP endpoints. This forces the developer to design his service as a web service. The new mechanism means that any Java component can join a REST-AT transaction without this burden and consequently it becomes isolated from the details of using REST-AT. Writing an application that uses REST-AT means that services can take part in transactions that can potentially span multiple services and servers.
So how does it work in practice? The service writer simply registers his service to receive callback notifications when the global transaction is ready to complete. Specifically, you will need to depend on the RTS subsystem by including a dependency on it in your application manifest (add Dependencies: org.jboss.narayana.rts to MANIFEST.MF) and pull in the RTS library when building your services. For maven users you will need:

    <dependency>
        <groupId>org.jboss.narayana.rts</groupId>
        <artifactId>restat-integration</artifactId>
        <version>5.0.0.M3</version>
        <scope>provided</scope>
    </dependency>

The application developer interface to the RTS subsystem is

    String enlist(String applicationId,
                  String participantEnlistmentURL,
                  Participant participant);
    void registerDeserializer(String applicationId,
                  ParticipantDeserializer deserializer);
    void reportHeuristic(String participantId,
                  HeuristicType heuristicType);

The participantEnlistmentURL in the enlist method corresponds to a running REST transaction which the service acquires during normal interactions with service clients. To register for completion callbacks the service writer registers an interface using the enlist method and passes in an implementation of Participant:

    public interface Participant {
        Vote prepare();
        void commit() throws HeuristicException;
        void commitOnePhase();
        void rollback() throws HeuristicException;
    }

Now when a service client terminates a transaction the services' callback methods will be invoked (by a REST-AT coordinator which may or may not be running locally since these are distributed transactions). It is interesting to note that the wildfly application server is a modular container so subsystems and applications run in their own class loaders. In the event of failures a recovery system will need to recreate participant callback registrations in order to complete any pending transaction and therefore will no longer have access to the original class. The service writer must help the recovery system in this task via the registerDeserializer call to the RTS subsystem. The final method on the interface to RTS (reportHeuristic) is to allow services to independently abort or commit work before being asked to via the callback interface.

What's next

I think you will agree that this is a pretty simple, low overhead, interface that allows services to participate in global transactions and we encourage users to start experimenting with it. In forthcoming narayana snapshot releases we will be adding quickstarts and examples that showcase the functionality including how it is used in a J2SE environment. If you would like to try out the quickstarts now then go to our narayana quickstarts git repository and look for the service2 and recovery2 examples. We will also be providing more documentation of REST-AT features in the documentation repository.



In addition Gytis Trikleris will soon be integrating some more exciting work that he has been doing in this area: currently, if a service takes part in a REST-AT transaction which also starts JTA transactions then the two are not related, i.e. terminating the REST-AT transaction has no effect on the JTA transactions (and vice versa). Gytis has written a bridge that fixes this deficiency, "watch this space" as they say.

Compensating Transactions: When ACID is too much (Part 4: Long Lived Transactions)

In part one, I explained how ACID transactions can have a negative impact on applications whose transactions can take a relatively long time to run. In addition, another potential issue with ACID transactions is that the failure of one unit can cause the entire transaction to be rolled back. This is less of an issue for short running transactions, as the previously successful work can be retried quickly. However, for long running transactions, this previously completed work may be significant, leading to a lot of waste should it need to be rolled back. In this post, I'll show how a compensation-based transaction could be a better fit for long lived transactions.

A compensation-based transaction can be composed of multiple short-lived ACID transactions. When each transaction completes, it releases the locks on the resources it held, allowing other transactions, requiring those resources, to proceed. A compensation action is registered for each ACID transaction and can be used to undo any work completed, should the entire compensation-based transaction need to be aborted. Furthermore, should one of these short-lived ACID transactions fail, it could be possible to find an alternative, preventing the entire transaction from failing. This allows forward progress to be achieved. By composing the compensation-based transaction as several units of work, you also gain the opportunity to selectively abort (compensate) particular units as the compensation-based transaction progresses. A simple example should help to clarify these ideas...

For example, take a travel booking scenario. We begin by booking a flight. We then try to book a taxi, but that fails. At this point we don’t want to compensate the flight as it may be fully-booked next time we try. Therefore we try to find an alternative Taxi, which in this example succeeds. Later, in the compensation-based transaction, we may find a cheaper flight, in which case we want to cancel the original flight whilst keeping the taxi and the cheaper flight. In this case we notify our intentions to the transaction manager who ensures that the more expensive flight is compensated when the compensation-based transaction completes.

Code Example

In this example, I expand on the Travel Agent example from part 3 in this series. Here I will show how a failure to complete one unit of work does not have to result in the whole compensation-based transaction being aborted.

public class Agent {
 
 @Inject
 HotelService hotelService;
 
 @Inject
 Taxi1Service taxi1Service;
 
 @Inject
 Taxi2Service taxi2Service;
 
 @Compensatable
 public void makeBooking(String emailAddress, String roomType, String destination) throws BookingException {
 
  hotelService.makeBooking(roomType, emailAddress);
  
  try {
   taxi1Service.makeBooking(destination, emailAddress);
  } catch (BookingException e) {
    /**
     * Taxi1 rolled back, but we still have the hotel booked. We don't want to lose it, so we now try Taxi2
     */
     taxi2Service.makeBooking(destination, emailAddress);
  }
 }
}

For this example, you can imagine that the Hotel and Taxi services are implemented similarly to the HotelService in part 3.
The makeBooking method is annotated with @Compensatable, which ensures that the method is invoked within a compensation-based transaction. The method begins by making a Hotel reservation. If this fails, we don't handle the BookingException, which causes the compensation-based transaction to be canceled. We then move onto booking a taxi. If this particular booking fails, we catch the BookingException and try an alternative Taxi company. Because the Taxi service failed immediately, we know that it should (it's a requirement of using this API) have undone any of it's work. We can therefore chose to fail the compensation-based transaction or, in this case try an alternative Taxi company. The important thing to note here is that we still have the Hotel booked and we don't really want to lose this booking as the hotel may be fully booked next time we try. The code then goes on to attempt an alternative Taxi company. If this booking fails, we have no option but to cancel the whole transaction as we have no other alternatives.

Conclusion

In this blog post, I showed how a compensation-based transaction could be a good fit for long running transactions. In the next part I'll discuss the status of our API for compensation-based transactions.

Compensating Transactions: When ACID is too much (Part 3: Cross-Domain Distributed Transactions)

In part one in this series I explained why ACID transactions are not always appropriate. I also introduced compensation-based transactions as a possible alternative to ACID transactions. In this post I'll show how compensation-based transactions could be a better fit, than ACID transactions, for distributed applications that cross high latency networks or span multiple business domains.

When your application becomes distributed, and more systems become involved, you inevitably increase the chances of failure. Many of these failures can be tolerated using a distributed ACID transaction. However, an ACID transaction can be seen as impractical for certain distributed applications. The first reason for this, is that distributed transactions, that cross high latency networks (such as the Internet), can take a relatively long time to run. As I showed in part 1, increasing the time to run an ACID transaction can have negative impacts on your application. For example, the holding of database resources for prolonged periods can significantly reduce the throughput of your application. The second reason is due to the tight coupling between the participants of the transaction. This tight coupling occurs because the root coordinator of the transaction ultimately drives all the transactional resources through the 2PC protocol. Therefore, once prepared, a transactional resource has to either make a heuristic decision (bad) or wait for the root coordinator to inform it of the outcome. This tight-coupling may be acceptable if your distributed application resides in a single business domain where you have control of all the parties. However, it is less likely to be acceptable for distributed applications that span multiple business domains.

Compensation-based transactions could prove to be a better solution for these scenarios. As I showed in part 1, compensation-based transactions can be more suitable for longer lived transactions, as they don't need to hold onto database resources until the transaction completes. Compensation-based transactions can also be used to decouple the back-end resources from the transaction coordinator. This can be done by splitting the two phases of the protocol into abstract business operations, such as book/cancel. The 'book' operation makes an update to the database to create the booking. As this update is committed immediately, there is no tight-coupling between the database resources and the transaction coordinator. The cancel operation is invoked by the compensation handler, should the compensation-based transaction need to abort.

I'll also show, through this example, how isolation can be preserved, whilst moving the resource locking from the database-level up to the application-level where it is easier for the application to reason about.

Code Example

In this example we'll look at a simple travel booking example, in which a client makes a hotel and taxi booking with remote services, inside a compensation-based transaction. These remote services live in different business domains and are invoked over the Internet.

public class Client {
 
  @Compensatable
  public void makeBooking() throws BookingException {
 
    // Lookup Hotel and Taxi Web Service ports here...
 
    hotelService.makeBooking("dbl", "paul.robinson@redhat.com");
    taxiService.makeBooking("ncl", "paul.robinson@redhat.com");
  }
}

This code forms part of the client application. The 'makeBooking' method is annotated with '@Compensatable' which ensures that the method is invoked within a compensation-based transaction. The method invokes two Web services. These Web services support WS-BA, so the transaction is transparently distributed over these calls.

@WebService
public class HotelService {
 
 @Inject
 BookingData bookingData;
 
 @Compensatable(MANDATORY)
 @TxCompensate(CancelBooking.class)
 @TxConfirm(ConfirmBooking.class)
 @Transactional(value=REQUIRES_NEW, rollbackOn=BookingException.class)
 @WebMethod
 public void makeBooking(String item, String user) throws BookingException {
 
  //Update the database to mark the booking as pending...
 
  bookingData.setBookingID("the id of the booking goes here");
 }
}

Here's the code for the Hotel's Web Service. As well as the usual JAX-WS annotations that you would expect (some omitted for brevity), there are some extra annotations to manage the transactions. The first is @Compensatable(MANDATORY); this ensures that this method is invoked within the scope of a compensation-based transaction. The second annotation (@TxCompensate) provides the compensation handler, which you should be familiar with from Part 2 in this series. The third annotation (@TxConfirm), may be new to you. This annotation provides a handler that is invoked at the end of the transaction if it was successful. This allows the application to make final changes once it knows the transaction will not be compensated. Hopefully, the need for this feature will become more clear as we discuss this example further. Finally, this example uses the JTA 1.2 @Transactional annotation to begin a new JTA transaction. This transaction is used to make the update to the database and will commit if the 'makeBooking' method completes successfully. The JTA transaction will rollback if a BookingException (see the rollbackOn attribute) or a RuntimeException (or a subclass of) are thrown.

So, why does the JTA transaction commit at the end of this method call even though the compensation-based transaction is still running?

This is done to reduce the amount of time the service holds onto database resources. The application could simply add the booking to the database, but it's possible that at some time in the future it might need to be canceled. Therefore in this example, the application just marks the booking as pending. Therefore, any other transaction that reads the state of the bookings table will see that this particular booking is tentative. Remember, by using a compensation-based transaction, we have relaxed isolation, and this is one way in which the application can be modified to tolerate this.

public class ConfirmBooking implements ConfirmationHandler {
 
 @Inject
 BookingData bookingData;
  
 @Override
 @Transactional(REQUIRES_NEW)
 public void confirm() {
   //Confirm order for '" + bookingData.getBookingID() + "' in Database (in a JTA transaction)
 }
}

As mentioned above, the ConfirmationHandler provides a callback that occurs when the compensation-based transaction completes successfully. In this example, the confirmation handler begins a new JTA transaction and then updates the database to mark the booking as finalized.

public class CancelBooking implements CompensationHandler {
 
 @Inject
 BookingData bookingData;
 
 @Override
 @Transactional(REQUIRES_NEW)
 public void compensate() {
  //Cancel order for bookingData.getBookingID() in Database (in a new JTA transaction)
 }
}

Similarly, we also have a compensation handler that starts a new JTA transaction which cancels the booking.

In this example we are essentially making a trade-off between 2 shorter lived JTA transactions, with application-level locking (in this example) in place of 1 longer lived JTA transaction with database-level locking (had we used a distributed JTA transaction). The isolation level is roughly the same for both cases. We also needed to make a change to the application. Wether this is a sensible trade-off will depend largely on your application and throughput requirements.

It is also worth noting that the middleware can (we don't yet implement this, see here and here) reliably tie the compensation-based and the JTA transaction together. It does this by preparing the JTA transaction when the method completes, but holds off committing it until the compensation handler has been logged to the transaction log. This ensures that the work is only committed if it can later be compensated in the case of failure. The invocation of the CompensationHandler and the ConfirmationHandler is also reliable as they are invoked repeatedly until they complete successfully. Therefore, it is important for thier implementations to be idempotent.