Software Transactional Memory for the Cloud at µCon London 2017

I have just returned from µCon London 2017: The Microservices Conference. I was down there talking about our Software Transactional Memory implementation which our regular readers may recall was first introduced by Mark back in 2013. In particular I wanted to show how it can be used with the actor model features of Vert.x and the nice scaling features of OpenShift.

I have put the presentation slides in the narayana git repo so feel free to go and take a look to see what you missed out on. You can also access the source code I used for the practical demonstration of the technology from the same repository. The abstract for the talk gives a good overview of the topics discussed:

"Vert.x is the leading JVM-based stack for developing asynchronous, event-driven applications. Traditional ACID transactions, especially distributed transactions, are typically difficult to use in such an environment due to their blocking nature. However, the transactional actor model, which pre-dates Java, has been successful in a number of areas over the years. In this talk you will learn how they have been integrating this model using Narayana transactions, Software Transactional Memory and Vert.x. Michael will go through an example and show how Vert.x developers can now utilise volatile, persistent and nested transactions in their applications, as well as what this might mean for enterprise deployments."

And the demo that this abstract is referring to is pretty trivial but it does serve to draw out some of the impressive benefits of combining actors with STM. Here's how I introduced the technology:

1. Firstly I showed how to write a simple Vert.x flight booking application that maintains an integer count of the number of bookings.
2. Run the application using multiple Vert.x verticle instances.
3. Demonstrate concurrency issues using parallel workloads.
4. Fix the concurrency issue by showing how to add volatile STM support to the application.

I put the without STM and the with STM versions of the demo code in the same repository as the presentation slides.

I then showed how to deploy the application to the cloud using a single-node OpenShift cluster running on my laptop using Red Hat's minishift solution. For this I used "persistent" STM which allows state to be shared between JVMs and this gave me the opportunity to show some of the elastic scaling features of STM, Vert.x and OpenShift.

Software Transactional Memory with WildFly-Swarm

A long time ago (not in a Galaxy Far Far Away!) I wrote about the STM implementation we were adding to Narayana. Over the intervening years this implementation was added to Vert.x and even the RaspberryPi made an appearance! Now whilst the implementation has been available in WildFly for many years, we tend not to mention it because, well, it's not Java EE compliant. However, with the advent of WildFly-Swarm things may be about to change.

Sure, when you're looking at using Swarm it's likely that at least initially you'll be coming at a problem from the perspective of Java EE, but the more you look to decompose your application into constituent (micro) services the more chances there are that you'll also start to look at functionality and frameworks that aren't necessarily just about Java EE. As we've mentioned before, STM is compatible with JTA and JTS transactions as well, as long as you understand what it means to mix-and-match them. Therefore, we've added an example of STM usage within WildFly-Swarm, which hopefully will become part of the mainline Swarm examples eventually. Take a look and give us any feedback, either in the Swarm group/IRC/Twitter or the usual Narayana routes.

STM, vert.x and the Pi Part 1

Given other priorities (aka 'my day job'), I'm slowly working my way through another pet project I've had for a while: adding transactions, and specifically STM, to vert.x. Of course at the moment I also try to add an influence from my other pet project, the Raspberry Pi, so I'm doing it on the Pi as much as possible. OK, it may not be the fastest machine on the block, but it's a lot of fun!

We've had STM in Narayana for a while, so I won't cover the basic again. I'm also not going to cover any Pi-related set up: by this point you should know enough to do this yourself. So let's start with cloning the most recent version of Narayana:

 

Make sure your path and JAVA_HOME are set so we're using JDK 7:

 

Next let's build the components we need for STM:

Don't forget that this is going to take a while, so maybe time to go grab a coffee, read a book, play with your kids, or watch an episode of Lost.

With that build complete, we've already run a lot of STM examples in the form of unit tests. However, let's take a simple example:

import java.io.IOException;

import org.jboss.stm.annotations.Transactional;
import org.jboss.stm.annotations.ReadLock;
import org.jboss.stm.annotations.WriteLock;
import org.jboss.stm.internal.RecoverableContainer;

import com.arjuna.ats.arjuna.AtomicAction;

/**
 * @author Mark Little
 */


public class Example
{  
    @Transactional
    public interface Atomic
     {
     public void change (int value) throws Exception;
       
     public void set (int value) throws Exception;
       
     public int get () throws Exception;
    }
   
    @Transactional
    public class ExampleSTM implements Atomic
     {  
        @ReadLock
     public int get () throws Exception
     {
         return state;
     }

        @WriteLock
     public void set (int value) throws Exception
     {
         state = value;
     }
       
        @WriteLock
     public void change (int value) throws Exception
     {
         state += value;
     }

     private int state;
    }
   
    public void testExample () throws Exception
    {
        RecoverableContainer theContainer = new RecoverableContainer();
        ExampleSTM basic = new ExampleSTM();
        Atomic obj = null;
       
        try
        {
        obj = theContainer.enlist(basic);
        }
        catch (final Throwable ex)
        {
        ex.printStackTrace();
           
        return;
        }
       
        AtomicAction a = new AtomicAction();
       
        a.begin();
       
        obj.set(1234);
       
        a.commit();

    System.out.println("Should get() 1234 after commit: "+obj.get());
       
        a = new AtomicAction();

        a.begin();

        obj.change(1);
       
        a.abort();

    System.out.println("Should get() 1234 after abort: "+obj.get());
    }

    public static void main (String[] args)
    {
    Example ex = new Example();

    try
    {
        ex.testExample();
    }
    catch (final Exception e)
    {
        e.printStackTrace();
    }
    }
}

Stick this at the same level of Narayana if you want to use the same CLASSPATH as here:

narayana/ext/jboss-logging.jar:narayana/ArjunaCore/arjuna/target/test-classes:narayana/common/target/common-5.0.0.M3-SNAPSHOT.jar:narayana/STM/target/core-5.0.0.M3-SNAPSHOT.jar:narayana/ArjunaCore/txoj/target/txoj-5.0.0.M3-SNAPSHOT.jar:narayana/ArjunaCore/arjuna/target/arjuna-5.0.0.M3-SNAPSHOT.jar:.

After compiling, run the example and you should see the following:

Despite what I said at the start of this entry, there's a distinct lack of vert.x involved here at the moment. That's because there are a few changes we need to make to the STM implementation to make it easier to share transactional objects between address spaces. This is all stuff that's available "natively" in TXOJ, so developers could use that API immediately. However, the STM approach simplifies a lot so once those additional features from TXOJ are integrated there, we'll have a follow up article to discuss what, how and why.

Transactions and parallelism and actors, oh my!

I wanted to cross-post an article I just wrote on the benefits of transactions in a massively parallel environment (not just a distributed system), not just because some may find it interesting in itself, but also because it references some of the work we've been doing in the project over the past decades. I'm hoping that as we do more work on STM and mobile transactions, to name but two areas of work, that we'll bring some of the techniques that were written about 20+ years ago to a new audience and maybe out of the realm of research. We have got a great transaction system and team here that is second to none. I'm convinced that just as Arjuna pushed the boundaries of standards and adoption of transactions for the past decades, so too can JBossTS/Narayana into the future.

Workshop on the Theory of Transactional Memory

I'm just back from the WTTM in Rome. It was good to go to the workshop and listen to presentations from the likes of Maurice Herlihy on advances in both hardware and software transactional memory. I was there representing the Cloud-TM work that we're doing in collaboration with others and specifically around Infinispan and JBossTS.

One of my favourite presentations of the event was on the use of pessimistic concurrency control in STM. Most STM approaches use an optimistic approach, which obviously works well in cases where contention is limited. The problem with pessimistic is retaining locks for the duration even when updates are infrequent. However, as the presenter showed, with suitable a priori ordering, pessimistic can be more efficient even in low contention environments. This was good because we tend to use pessimistic concurrency control in JBossTS a lot, and as was shown in the presentation, it's also a lot easier for developers to understand and reason about.

Unfortunately I had to leave the workshop before the last session, so I didn't get to listen to Torvald who represents Red Hat on the transactional C++ working group. That's a shame, because I've been following this effort for several years and know that we may be seeing this in gcc sometime soon. And who knows, maybe EE8 will see some STM eventually.