osrm-backend: osrm-extract for planet on EC2 hs1.8xlarge: "External memory block allocation error"

Still trying to see if I can get OSRM working for the planet… This time I decided to get serious and started up an hs1.8xlarge instance on AWS with ~117 GB RAM, running Ubuntu 12.04.

After a few minutes, osrm-extract fails:

$ ./run_osrm-extract.sh                                                                                         
Wed Nov 13 19:31:33 UTC 2013
[info] Input file: planet-131106.osm.pbf
[info] Profile: profile.lua
[info] Threads: 8
[info] Using script profile.lua
[STXXL-MSG] STXXL v1.3.1 (release)
[STXXL-MSG] 1 disks are allocated, total space: 25000 MiB
[info] Using turn restrictions
[info] Found 3 exceptions to turn restriction
[info]    motorcar
[info]    motor_vehicle
[info]    vehicle
[info] Parsing in progress..
[STXXL-ERRMSG] External memory block allocation error: 2097152 bytes requested, 0 bytes free. Trying to extend the external memory space...
[STXXL-ERRMSG] External memory block allocation error: 2097152 bytes requested, 0 bytes free. Trying to extend the external memory space...
...

Here are the contents of my .stxxl file (/home/ubuntu/osrm_ebs_volume is a 100 GB EBS volume that I attached to my EC2 instance):

$ cat .stxxl
disk=/home/ubuntu/osrm_ebs_volume/osrm/stxxl,25000,syscall

I also tried using a .stxxl capacity of 40000 which resulted in the same error.

You can see here that I have enough disk space for the stxxl file:

$ pwd
/home/ubuntu/osrm_ebs_volume/osrm
$ ls -lh stxxl 
-rw-rw---- 1 ubuntu ubuntu 25G Nov 13 20:25 stxxl
$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      7.9G  1.5G  6.1G  20% /
udev             58G   12K   58G   1% /dev
tmpfs            23G  256K   23G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none             58G     0   58G   0% /run/shm
/dev/xvdb2       99G   47G   47G  51% /home/ubuntu/osrm_ebs_volume

Why would osrm-extract fail like this considering that I have enough memory and disk space?

About this issue

  • Original URL
  • State: closed
  • Created 11 years ago
  • Comments: 16

Most upvoted comments

Success. Increasing the stxxl file capacity to 100 GB and moving it off the EBS volume to EC2 instance storage did the trick. For future reference, extract on hs1.8xlarge took 3 hr 15 min and used ~45 GB of memory.

Many thanks for the tips @JosefCS and @DennisOSRM !

Unfortunately prepare is failing; will open a separate issue for that problem since this one is focused on extract.