We have developed STMBench7: a benchmark for evaluating TM implementations. The benchmark aims at providing a workload that is both realistic and non-trivial to implement in a scalable way. The implementation (in Java and C++) contains a lock-based synchronization strategy that can serve as a baseline for comparison with various TMs. The following paper describes STMBench7 in more detail:

Guerraoui, R., Kapalka, M. and Vitek, J. (2007) STMBench7: A Benchmark for Software Transactional Memory. Proceedings of the Second European Systems Conference EuroSys 2007.

The following paper describes our recent experiments with the benchmark using various available STM implementations:

Dragojevic, A., Guerraoui, R. and Kapalka, M. (2008) Dividing Transactional Memories by Zero. 3rd ACM SIGPLAN Workshop on Transactional Computing (Transact 2008).

Important note: The benchmark is still a work in progress. The code and the specification may still undergo many, maybe significant, changes that should reflect the feedback we will get.

Any suggestions, comments or criticism are more than welcome and should be send to Michal Kapalka.

The new Java version (07.03.2008 Beta) of STMBench7 has been released. The new features and changes from the previous version (27.06.2007) are the following:

1. Added correctness tests (useful for validating whether a given synchronization technique gives correct results):
   • Invariant tests: the implementation checks whether all the invariants of the benchmark data structure are preserved.
   • Sequential replay of a concurrent execution: the implementation can replay a concurrent execution sequentially in order to check for violations of the opacity property.
2. The new version does no longer use AspectJ: the coarse-grained and medium-grained locking methods are now fully implemented in Java, without adding any additional overhead when they are not used.
3. The build procedure now uses Apache Ant instead of make.

Note that the new version of the benchmark has a slightly different API. The new API is cleaner and easier to use with an STM, but may introduce problems when used with implementations using the old version of STMBench7.

1. New version (07.03.2008 Beta) − source code and JAR binary file: stmbench7-07.03.2008-beta.tgz
2. Old version (21.02.2007):
   • Source code: stmbench7-21.02.2007.tgz
   • JAR binary file: stmbench7-21.02.2007.jar

The Java version of STMBench7 is created/maintained by Michal Kapalka

The following files are available for download:

• SwissTM, TinySTM and TinySTM STMBench7 source code: sb7_tt_20090910.tgz

Older versions:

• Source code (with locking and RSTM-based synchronization): stmbench7.cpp-27.06.2007.tgz
• A patch for RSTM that is needed to run the RSTM version of STMBench7: rstm_v3.epfl.2.tgz
• SwissTM-w, TinySTM and TinySTM STMBench7 source code: sb7_tt_20081117.tgz
• SwissTM-o STMBench7 source code: sb7_swisstm_20080825.tgz
• TL2/TinySTM source code: sb7_tt_20080519.tgz

The C++ version of STMBench7 is created/maintained by Aleksandar Dragojevic.