How we evaluate performance improvements for Narayana

I would like to highlight a piece of work that our team (and special thanks to Mike) have been working on for a while now which is to provide a framework and methodology for us to use when developing performance improvements for Narayana.

The article itself is on our developers wiki over here:
https://developer.jboss.org/wiki/PerformanceGatesForAcceptingPerformanceFixesInNarayana

After you have read the article, he has also started a discussion on our forum where we can answer questions about the approach we have taken and whether our community have any comments/suggestions to what we can do to further refine this:
https://developer.jboss.org/message/922334?et=watches.email.thread#922334

It's worth pointing out that this is all fully integrated into our existing checks for pull requests on the Narayana repo. So not only will we execute about 12 hours of unit and integration tests on your modifications - we will now also run a set of tests to check the performance impact of functional changes or evaluate the effectiveness of improvements targeted at this area!

The article links to various tests that are executed so I won't expand here. What I can say is that although the suite is not exhaustive, if you propose a change that needs a specific style of test to verify its impact we will be really pleased to accept those changes too.

All of this performance testing is open source. You can see the tests we execute, the configurations we use and the results we have obtained on our hardware. If you want to run the tests yourselves it should be a case of "git clone" and a few simple steps as documented in our performance repo.

We have created this framework for anyone that wants to work on our project and we hope you find it useful! As a reminder - if you do wish to contribute to Narayana we recommend you take a look at this article to get started: https://developer.jboss.org/wiki/GetInvolvedWithNarayana

Thanks!