Last month, someone told me a new JPA implementation was trying to make noise. The author was making some buzz almost everywhere on social networks, commenting on popular articles and writing 2 articles claiming that this implementation was over 15 times faster than Hibernate. And here we are again. This sort of buzz happens every 2 years, it’s a cycle, there is nothing we can do about that.

Reading more about the implementation, the focus is made on fine grained tuning on every single line of code which is a really brave approach. However the way to announce « The New JPA Implementation That Runs Over 15 Times Faster… » is not a good way to promote the implementation. Anyway let’s have a closer look at the benchmark that allowed the author to conclude « 15 times faster ».

The benchmark is essentially composed of massive inserts, updates, deletes, queries by packet of 250 iterations. The domain model is composed of

• 4 entities, yes 4 entities,
• no inheritance, it sounds weird that a fabulous JPA implementation is not including more mapping configurations in its benchmark
• eager fetching enabled (no subselect fetching for example),
• cascading enabled,
• no versioning

Last but not least, the benchmark targets an in-memory database, yes this is where it becomes suspicious.

15 times faster… faster what?

The big problem behind the title is that faster is not at all related to response time. The author tries to explain it, it’s related to CPU load, fair enough but could he clearly explain the consequences on a full system?

The fact is that the results are really hard to read and there is no relative view of the benchmark. You are told that Batoo might take 15 times less CPU time than Hibernate and what? The danger here is to mix internal APIs CPU load with the full persistence cycle composed of JPA internals, JDBC driver, networking, database engine. We all know, at least I hope so, that most of the time is spent on network and database so is this really important to focus _that much_ on JPA internals? Is hibernate that slow? Will Batoo solve your performance issues?

Let’s be extremely clear here, most of the performance issues users are facing are NOT because of JPA internals CPU load, the problems are almost always related to a slow database, too fat entity managers, inefficient queries. There are 2 general ways to solve these issues:

• check your database !!! JPA is NOT going to fix a database internal problem
• make a better use of the database and JPA
• tune your own code, that requires various features and that is going to have a really big impact on the global performance numbers. In order to do so, the frameworks you are using must offer various features, this is exactly what Hibernate does: allows you to tune every single interaction your are triggering with the database.

I’m not going to rely on the timers provided by the benchmark. People are interested in checking how a system globally reacts when switching from one software option to another. I asked my friend and mate what tool a sysadmin would use to have a good view of the resources consumed by a process. « Easy » he told me, use the time command.

So I’m going to use time mvn test with the good format to grab the CPU numbers.

I remember 7 years ago, one guy at work was insulting Hibernate saying it was a piece a crap because he was observing memory issues and/or extremely long response times. One quick look at the hibernate documentation, 2 mapping parameter updates, more care of the entity manager (well Hibernate Session at that time), allowed to solve the memory issue and allowed a 10 000 faster execution. I promise it’s true, 10 000 times faster in response time.

<property name="hibernate.jdbc.batch_size" value="50"/>
<property name="hibernate.order_inserts" value="true"/>
<property name="hibernate.id.new_generator_mappings" value="true"/>

<property name="hibernate.connection.autocommit" value="false" />

• 3.625% of my global System CPU capacity with hibernate
• 2.75% of my global System CPU capacity with Batoo

A more realistic environment

I won’t stop here, the author says that his motivation is to reduce cluster size thanks to his optimized implementation. So let’s say I don’t have 8 core, only 2. Here I might be interested in reducing my CPU load.

• 1.5% of my global System CPU capacity with hibernate
• 1.25% of my global System CPU capacity with Batoo

• 0.75% of my global System CPU capacity with hibernate
• 0.62% of my global System CPU capacity with Batoo

Putting all of this together, Batoo MAY be slightly faster than Hibernate but the actual database load on these benchmarks far outweighs the load imposed by either Batoo or Hibernate.

And, since the internal ORM performance is a relatively minor issue, we should think about the products’ features.

• write stupid queries and you’ll get network, database and possibly memory issues (no matter if you use Hibernate or Batoo) -> query oriented advanced features are welcome
• be idiot using the entity manager and you’ll observe slowness and memory issues (no matter if you use Hibernate or Batoo) -> get some training and certification, persistence is not something easy, you need SKILLS 
• not talking about the DB slowness … if you do not understand RDBMS, you MUST work with a DBA, this is not an advice, this is a requirement.
• if you are an expert in JPA, and your application is slow because of the database, you may have hit the RDBMS limitations. Give a try to Hibernate OGM with a NoSql supported database.

Shoot 5: Remote MySql, BENCHMARK_LENGTH 100, 2 concurrent run

I’m not going to provide the results but response times are impacted by 50%. CPU load is close to 2%

Conclusion: I don’t care these CPU optimizations I prefer FEATURES!

I honestly cannot imagine JPA being a cause of CPU problem in a real life scenarios. I’m also not forgetting that Hibernate is a 12 years old project. Talented people are working full time on it, the major is 4, that means it has been heavily totally reworked 4 times. So that’s enough to understand that it is well written and that efforts are put in the most critical area: offering features.

I did online community support then official Red Hat support. I’ve seen query response times issues (1), memory issues (2) sometimes but I can’t remember about a CPU issue related to JPA.

To solve 1 and 2, you need skills but also features that help you tune your code. Like being able to use a specific database syntax (dialects), or  tune associations loading, I could continue on 300 pages here and I would recommend you simply consult the Hibernate documentation’s index.