docker: Planet build import all fails

Hi,

We are trying to setup a full planet build. All steps succeed until we get to run “pelias import all” which fails with:

ERROR: Elasticsearch index pelias does not exist
You must use the pelias-schema tool (https://github.com/pelias/schema/) to create the index first
For full instructions on setting up Pelias, see http://pelias.io/install.html
/code/pelias/whosonfirst/node_modules/pelias-dbclient/src/configValidation.js:39
        throw new Error(`elasticsearch index ${config.schema.indexName} does not exist`);
        ^

Error: elasticsearch index pelias does not exist
    at existsCallback (/code/pelias/whosonfirst/node_modules/pelias-dbclient/src/configValidation.js:39:15)
    at respond (/code/pelias/whosonfirst/node_modules/elasticsearch/src/lib/transport.js:368:9)
    at /code/pelias/whosonfirst/node_modules/elasticsearch/src/lib/transport.js:396:7
    at Timeout.<anonymous> (/code/pelias/whosonfirst/node_modules/elasticsearch/src/lib/transport.js:429:7)
    at listOnTimeout (internal/timers.js:531:17)
    at processTimers (internal/timers.js:475:7)

The smaller portland-metro build works find.

About this issue

Original URL
State: open
Created 4 years ago
Reactions: 1
Comments: 16 (3 by maintainers)

Most upvoted comments

@xiaofengilove Placing an empty file in the blacklist folder named osm.txt resolved the parser errors for me.

darmentrout on Oct 20, 2020

Exactly! I’ve been struggling with this one as well! My experience:

Docker invocation for pelias import works for all and all other routines, EXCEPT wof.

I tried bash -x pelias import wof. This revealed the actual docker compose command. I run that right in the terminal, which works fine:

docker-compose run  whosonfirst ./bin/start

Thoughts: elasticsearch URL or network is off or pelias.json is not properly mapped into wof docker image. None of these theories make sense since all other importer routines work fine, and in these cases I’m using mostly defaults. Nothing to override really.

mubaldino on Sep 26, 2020

I’m hitting an identical looking error with a planet build, and similarly, everything seems to work fine with the smaller portland build.

Given the long time it takes before the command errors (+2minutes), maybe (just a guess) the root problem is hitting the 120s timeout and it’s being surfaced in a confusing way.

A corroborating data point: After having the planet sized import fail a couple dozen times with the default 2 minute timeout, I specified a timeout of 10 minutes and was able to complete the import on the first try.

pelias config:

{
  "esclient": {
    "requestTimeout": "600000",
    ...
  },
  ...
}

michaelkirk on Nov 10, 2022

Hi, the same error, only when importing whosonfirst data:

debug: [whosonfirst] Loading 'ocean' of whosonfirst-data-admin-latest.db database from /data/whosonfirst/sqlite
debug: [whosonfirst] Loading 'marinearea' of whosonfirst-data-admin-latest.db database from /data/whosonfirst/sqlite
debug: [whosonfirst] Loading 'continent' of whosonfirst-data-admin-latest.db database from /data/whosonfirst/sqlite
debug: [whosonfirst] Loading 'empire' of whosonfirst-data-admin-latest.db database from /data/whosonfirst/sqlite
debug: [whosonfirst] Loading 'country' of whosonfirst-data-admin-latest.db database from /data/whosonfirst/sqlite
debug: [whosonfirst] Loading 'dependency' of whosonfirst-data-admin-latest.db database from /data/whosonfirst/sqlite
debug: [whosonfirst] Loading 'disputed' of whosonfirst-data-admin-latest.db database from /data/whosonfirst/sqlite
debug: [whosonfirst] Loading 'macroregion' of whosonfirst-data-admin-latest.db database from /data/whosonfirst/sqlite
debug: [whosonfirst] Loading 'region' of whosonfirst-data-admin-latest.db database from /data/whosonfirst/sqlite
ERROR: Elasticsearch index pelias does not exist
You must use the pelias-schema tool (https://github.com/pelias/schema/) to create the index first
For full instructions on setting up Pelias, see http://pelias.io/install.html
/code/pelias/whosonfirst/node_modules/pelias-dbclient/src/configValidation.js:39
        throw new Error(`elasticsearch index ${config.schema.indexName} does not exist`);
        ^

Error: elasticsearch index pelias does not exist
    at existsCallback (/code/pelias/whosonfirst/node_modules/pelias-dbclient/src/configValidation.js:39:15)
    at respond (/code/pelias/whosonfirst/node_modules/elasticsearch/src/lib/transport.js:368:9)
    at /code/pelias/whosonfirst/node_modules/elasticsearch/src/lib/transport.js:396:7
    at Timeout.<anonymous> (/code/pelias/whosonfirst/node_modules/elasticsearch/src/lib/transport.js:429:7)
    at listOnTimeout (internal/timers.js:554:17)
    at processTimers (internal/timers.js:497:7)

dan83g on Sep 20, 2022