quarkus: Real world JPA data model (300+ entities) for testing reload performance

Description

As discussed on twitter with @Sanne (Hi 👋 !) I have made available a quarkus application with a real-life JPA model consisting of 300+ entities. This is quite useful to test reload performance. On my machine reloading the application in development mode takes on average 4 seconds (a 3950x/64gb/fedora linux).

You can find the code here: https://github.com/topicusonderwijs/quarkus-tribe-krd

A Bit of History

This project is licensed under the AGPL license (because the forked project has that license). The original project was released as open source due to requirements of the public tender under which it was developed. The AGPL was picked at that time.

The model is complete for the purposes of the quarkus project, but not the complete model of the application. It was partially open sourced (only those parts that were required per the tender) in 2013-ish, so development has continued and the actual current model should be roughly 1200+ JPA classes big.

The original system is a Student Information System for vocational education (trade schools) in the Netherlands. It was expanded for college/university education as well. The JPA model in the quarkus project is a subset of all functionality, but most concepts still exist in the current application’s domain, albeit expanded.

How it works

It is just a quarkus project, so starting with mvn quarkus:dev should work ™️ . In the package org.acme is the standard quarkus hello resteasy resource. Whenever you modify this the application is reloaded by quarkus, causing Hibernate to reinitialise the JPA model.

The entities reside in the package nl.topicus.eduarte.entities and subpackages. Other packages exist to make the project compile and start up.

Queries?

I stripped the application of all functionality as the original code base is Hibernate 3.5.x based, and that doesn’t translate well to Hibernate 5.x easily. If you wish to see how queries are created you can look at the initial commit:

https://github.com/topicusonderwijs/quarkus-tribe-krd/tree/44dd1e302ce231ecca326605d51e361eb651d4db/eduarte/core/src/main/java/nl/topicus/eduarte/dao/hibernate

And if you want to see some horror methods take a look at:

You can imagine I did not port this functionality over.

Implementation ideas

As discussed with @Sanne on twitter, he told me that Hibernate developers lack real world domain models for testing Hibernate’s functionality against. I knew about the existence of this code base, so I stripped it to the bare minimum to make it work in modern quarkus (the original code base is Hibernate 3.5.x iirc). I did not convert any queries or other business functionality.

I have no further expectations but would like to know that you find this project useful and it helps Hibernate and Quarkus to improve startup times and other aspects, for example perhaps a Hibernate 6 API bridge between Hibernate criteria API and JPA criteria API (one can only wish)

Have fun with this project!

About this issue

Original URL
State: open
Created 3 years ago
Reactions: 2
Comments: 20 (13 by maintainers)

Most upvoted comments

🎄🎁 🎅

Wow, about a year ago the last comment in this ticket was made. A lot can happen in a year, and that is true for this ticket as well. I had a blast talking with you a couple of months ago, @sebersole @Sanne!

The code base has transferred with the purchase of Eduarte and all attached IP by our company, Topicus. This means we were able to re-license the IP ourselves. The project is now available at https://github.com/topicusonderwijs/tribe-krd-quarkus and licensed using Apache 2.0.

I hope you are able to work with the project and improve Hibernate and Quarkus with this real life object model.

https://github.com/topicusonderwijs/tribe-krd-quarkus

Happy holidays!

dashorst on Dec 24, 2022

Many thanks agasin @dashorst 😃

Admittedly I’m supposed to finish some “real” work first and I’ll have a better look later (holidays?), but I couldn’t resist to at least check what the primary waste of time is.

As expected, we’re spending a good amount of time in the actual DB to create the tables: since this is “dev mode” and it’s using an in-memory database it needs to regenerate the schema after booting.

I was able to reproduce similar numbers as yours; reconfiguring the app to use PostgreSQL and disabling schema generation (and verifications!) I was able to half the time to just over 2s:

quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=hibernate_orm_test
quarkus.datasource.password=hibernate_orm_test
quarkus.datasource.jdbc.url=jdbc:postgresql://localhost/hibernate_orm_test

quarkus.hibernate-orm.log.sql=false
quarkus.hibernate-orm.database.generation=none
quarkus.hibernate-orm.validate-in-dev-mode=false

Of course that’s not a viable setup as I expect people to want to have fun updating the model and getting immediate feedback in the form of schema changes (or at least validation) - but it serves me to get a feeling of were time is being spent.

It’s also nice to see that the Quarkus classloader and our full boot process is able to scale well with the number of entities 😃

When profiling this unreleastic configuration (disabling all schema validations) I can see a little bit of waste in the Alias generation in ORM; that’s a low hanging fruit we can improve, but clearly unless we significantly change the way schema validation/update is handled I have no hopes of getting this under 1s - I guess at best we can improve ~25% more.

For more impactful improvements we’ll need to get creative with the schema generation; I guess validation could be performed in parallel on each database element, but for “create” and “update” modes it will be a bit more complicated.

3950x/64gb/fedora linux

Funny, I’m working on a very similar setup: 3960x/64gb/fedora 😃

Sanne on Dec 14, 2021

I got you covered @sebersole

All you need to do is learn this:

550x690

dashorst on Mar 9, 2023

Hi @dashorst , sorry for the delay! I’ve been very focused on the Hibernate ORM 6 migration, catching up… Many thanks for this, we’ll make good use of it - probably not in Quarkus 3.0.0 but soon for sure.

Sanne on Mar 8, 2023

@sebersole right - with my comment I didn’t mean it needs to be included in 6.0 - I merely wanted to point out that I’d not want to do such an optimisation in 5.x - the version we have in Quarkus currently.

Sanne on Dec 15, 2021

very cool! Thanks. Always looking for something like this to test pgjdbc with.

davecramer on Dec 14, 2021

Thanks a lot for taking the time to add open source this work. This is very helpful!

geoand on Dec 14, 2021