graphhopper: Querying localhost is slower than using the Java API on certain hardware

Lately, we’ve been experiencing server response times that are slower than expected considering the query time we measure for the Java API. The problem only occurs on certain hardware and we haven’t really gotten to the root of the issue. To reproduce this we do the following:

# download GraphHopper source code and build the binaries using Maven
git clone https://github.com/graphhopper/graphhopper.git
cd graphhopper
git checkout 8112bd44a49e758170fed581dae3c861fc4480b9
mvn clean package -DskipTests
# download OSM data
wget https://download.geofabrik.de/europe/germany/bayern-220101.osm.pbf
# start a local GH server (it will run on port 8989)
java -Xmx10g -Xms10g -jar web/target/graphhopper-web-6.0-SNAPSHOT.jar server  jmh-test/config-jmh.yml
# in a separate terminal: run JMH benchmark. this will load the graph data and run queries against the Java API and the GH server we just started 
java -Xmx10g -Xms10g -jar jmh-test/target/benchmarks.jar

Example output 1 (normal/expected):

Benchmark                          (graph_folder)  (numPoints)                  (pbf)  Mode  Cnt   Score   Error  Units
RouteBenchmark.measureRoute      bayern-220101-gh            5  bayern-220101.osm.pbf  avgt   25  26.177 ± 0.301  ms/op
RouteBenchmark.measureRouteHttp  bayern-220101-gh            5  bayern-220101.osm.pbf  avgt   25  30.493 ± 0.229  ms/op

Example output 2 (shows the problem)

Benchmark                          (graph_folder)  (numPoints)                  (pbf)  Mode  Cnt   Score   Error  Units
RouteBenchmark.measureRoute      bayern-220101-gh            5  bayern-220101.osm.pbf  avgt   25  20.773 ? 0.203  ms/op
RouteBenchmark.measureRouteHttp  bayern-220101-gh            5  bayern-220101.osm.pbf  avgt   25  39.077 ? 0.233  ms/op

As we can see in the second case the http response is about x2 slower than the direct call of the Java API, while in the first case the difference is only around 4ms which seems ok considering there is some overhead when querying localhost, serializing JSON etc.

Like I said, the problem only occurs on certain hardware (and we aren’t really sure what hardware characteristic is important to reproduce it), but if it occurs it can be reproduced very consistently and for different JVM versions.

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 21 (21 by maintainers)

Most upvoted comments

Just for the record: We observed that the scaling governor does not rise the CPU frequency as it should which can result in a bigger slowdown. Both for AMD (with schedutil or ondemand governor) & Intel (with powersave governor) processors. The problem goes away by setting the scaling governor to performance (available for both drivers).

easbar on May 31, 2022

I created a separate repository for this benchmark: https://github.com/easbar/jetty_jmh_benchmark Possibly the problem is not really related to the hardware, frankly I do not know.

easbar on May 6, 2022