Narayana and its relationship to Red Hat middleware strategy

Hi everyone,


You might have already seen that Red Hat announced significant changes to its middleware strategy last month (if not, please do check out the relevant “Red Hat Blog” article: Evolving our middleware strategy [1]) and so I want to speak a little to the change and its relevance to our Narayana project.


As you may know, Narayana is a part of a number of Red Hat products, in particular Red Hat’s JBoss Enterprise Application Platform product and so this makes the strategic decision relevant to the Narayana project. That said, a key point in that article from the “Red Hat Blog” with regards to our Narayana project is that all transitioning Red Hat technology will remain open source and continue to follow an upstream-first development model. So as well as the technology still relying on being able to upstream-first (in projects like Narayana), it’s also that this upstream should remain open source (you can find what open source means at Red Hat over here [2]). Not only is the Narayana source code open source, but moreover its project operates in an open source manner, exhibiting the principles of open source and gratefully benefits from a healthy community of users and contributors. This will help us to keep innovating in the area of transactions as we move forwards.


I will also take this opportunity to add a “Thank you” for being part of our Narayana community - I am excited to see the results of what we achieve together next!



Tom Jenkinson


[1] https://www.redhat.com/en/blog/evolving-our-middleware-strategy

[2] https://www.redhat.com/en/about/open-source

Managing the availability of LRA participants

This post is a continuation of a series of jbosts blogs that discuss the MicroProfile LRA specification.

Services manage their workloads by providing endpoints to an LRA coordinator which in turn uses those endpoints to drive the LRA protocol forward thereby enabling the construction of reliable services. These endpoints may need to be modified over the long run so it ought to be possible to replace them with different ones in response to changes to the environment in which the service executes. Although the specification does not discuss how the endpoints can be replaced, the Narayana LRA REST API for the coordinator includes Microprofile OpenAPI documentation for replacing endpoints.

There are various administrative and management reasons for why the capability can be useful, such as controlling where termination handling is to take place, or to facilitate service replacement, etc. It may also be desirable for work completion, compensation, status reporting and clean up activities to be handled on different endpoints and at different times and this goal is facilitated via annotations including @Compensate, @Complete, @Status, @Forget and @AfterLRA.

When a participant does work in the context of a long running action, a “recovery URL” is created which services may use to associate their work with various management actions such as changing the participant endpoints as the action proceeds, after all a long running action can be of arbitrary duration and the needs of a service may change as the action evolves. The example I created for this post halts the JVM during “complete”, asks the user to send a curl request to the LRA coordinator to provide it with a new participant completion endpoint, restarts the participant on the new endpoint and waits for recovery to resend the completion callback to the new endpoint.

By leveraging the feature admins may proactively react to changing conditions (connectivity, throughput, functionality updates, etc) and be able to tune and or reconfigure the environment accordingly, perhaps bringing up a more reliability aware service that more intelligently operates within the more limited environment.

Build and start a coordinator on port 8080

Use the quarkus-maven-plugin to create a project for the coordinator, adding a dependency on maven artifact org.jboss.narayana.lra:lra-coordinator-jar:0.0.10.Final to the resulting pom. Also specify that the build should produce an uber jar so that the coordinator can run standalone:

    mvn io.quarkus:quarkus-maven-plugin:3.3.1:create -DprojectGroupId=org.acme -DprojectArtifactId=narayana-lra-coordinator -Dextensions="rest-jackson,rest-client"
    cd narayana-lra-coordinator
    rm -rf src/test src/main/java # the sources created by the example aren't required
    echo "quarkus.package.jar.type=uber-jar" > src/main/resources/application.properties
    # don't forget to add a dependency on maven artifact: org.jboss.narayana.lra:lra-coordinator-jar:0.0.10.Final
    ./mvnw clean package

and then start it on port 8080 by running the resulting jar

   java -jar target/narayana-lra-coordinator-1.0.0-SNAPSHOT-runner.jar &

Build and start a participant on port 8081 and run an LRA but halt the JVM before closing it

The service will be quite basic:

@Path("/halt")
public class MigratableResource {
    private static final AtomicBoolean halt = new AtomicBoolean(false);

    @LRA(value = LRA.Type.REQUIRED)
    @PUT
    public void doInTransaction() {
        halt.set(true); // halt when compensate or complete are called
        // when the business method finishes the LRA is closed and the complete endpoint will be called
    }

    @PUT
    @Path("/compensate")
    @Compensate
    public Response compensate() {
        return Response.ok().build();
    }

    @PUT
    @Path("/complete")
    @Complete
    public Response complete(@HeaderParam(LRA_HTTP_RECOVERY_HEADER) String recoveryUrl) {
        if (halt.get()) {
            int port = 8082;
            String completionUrl = String.format("http://localhost:%d/halt/complete", port);

            System.out.printf("Ask the coordinator to send the completion notification on a new endpoint using:%n");
            System.out.printf("curl -X PUT %s -d '<%s>; rel=complete'%n", recoveryUrl, completionUrl);
            Runtime.getRuntime().halt(1);
        }
        System.out.printf("completed%n");
        return Response.ok().build();
    }
}

The interesting part happens during completion where the JVM is halted. Notice that the curl command for migrating the completion endpoint is printed prior to halting.

Now build and run the participant on port 8081 - the maven project is available from the narayana artifacts maven repository.

cd <participant directory>
mvn clean package
java -Dquarkus.http.port=8081 -jar target/quarkus-app/quarkus-run.jar &

and then call the service method using the curl utility, or otherwise:

curl -X PUT -I http://localhost:8081/halt

The service method is annotated with just @LRA(value = LRA.Type.REQUIRED) so when it finishes the completion callback will be invoked by the coordinator. Make a note of the curl request printed by the completion callback just before it halts the JVM. An example is (the Uids will change on each run):

curl -X PUT http://localhost:8080/lra-coordinator/recovery/0_ffffc0a801c7_9d57_677ad0a4_2/0_ffffc0a801c7_9d57_677ad0a4_5 \
  -d '<http://localhost:8082/halt/complete>; rel=complete'

Notice that the payload of the HTTP PUT request includes the specification of the new completion callback, namely <http://localhost:8082/halt/complete>; rel=complete.

The new endpoint will be used on the next recovery pass which is every two minutes by default.

Finally restart the service on the new endpoint (port 8082):

java -Dquarkus.http.port=8082 -jar target/quarkus-app/quarkus-run.jar &

When the coordinator next runs a recovery scan it should use the new endpoint and the service will report that it has completed its' service work by printing the text “completed” when the completion endpoint is by the coordinator.