dfuse-eosio: mindreader merged blocks surpassing stop block in batch mode

Brief:

Goal is to create partial sync node for dappservices related actions. Started node at dappservices contract creation.

I am seeing merged block files written past the set stop block.

version

sha1sum /usr/local/bin/dfuseeos
508e7ea4e28f96ee7e0e2f7e648f79fb999778ec  /usr/local/bin/dfuseeos

yaml:

start:
  args:
  - mindreader
  flags:
    mindreader-log-to-zap: false
    mindreader-start-failure-handler: true
    mindreader-blocks-chan-capacity: 100000
    mindreader-number-of-snapshots-to-keep: 0
    mindreader-batch-mode: true
    mindreader-restore-snapshot-name: snapshot-2019-04-24-16-eos@0054672491.bin
    mindreader-start-block-num: 54672500
    mindreader-stop-block-num: 55972500

logs

2020-10-09T12:57:24.286Z (mindreader) temporary failure trying to upload mindreader merged block files, will retry (mindreader/merge_archiver.go:70){"error": "moving file \"/dfuse-data/mindreader/work/0061426500.merged\" to storage: writing \"/dfuse-data/mindreader/work/0061426500.merged\" to storage \"/dfuse-data/storage/merged-blocks/0061426500.dbin.zst\": rename: rename /dfuse-data/storage/merged-blocks/0061426500.dbin.zst.tmp /dfuse-data/storage/merged-blocks/0061426500.dbin.zst: no such file or directory"}
2020-10-09T12:57:24.787Z (mindreader) success uploading previously failed mindreader merged block files (mindreader/merge_archiver.go:74)
2020-10-09T12:57:26.451Z (mindreader) temporary failure trying to upload mindreader merged block files, will retry (mindreader/merge_archiver.go:70){"error": "moving file \"/dfuse-data/mindreader/work/0063835400.merged\" to storage: writing \"/dfuse-data/mindreader/work/0063835400.merged\" to storage \"/dfuse-data/storage/merged-blocks/0063835400.dbin.zst\": rename: rename /dfuse-data/storage/merged-blocks/0063835400.dbin.zst.tmp /dfuse-data/storage/merged-blocks/0063835400.dbin.zst: no such file or directory"}
2020-10-09T12:57:27.071Z (mindreader) temporary failure trying to upload mindreader merged block files, will retry (mindreader/merge_archiver.go:70){"error": "moving file \"/dfuse-data/mindreader/work/0063144100.merged\" to storage: writing \"/dfuse-data/mindreader/work/0063144100.merged\" to storage \"/dfuse-data/storage/merged-blocks/0063144100.dbin.zst\": rename: rename /dfuse-data/storage/merged-blocks/0063144100.dbin.zst.tmp /dfuse-data/storage/merged-blocks/0063144100.dbin.zst: no such file or directory"}
2020-10-09T12:57:27.571Z (mindreader) success uploading previously failed mindreader merged block files (mindreader/merge_archiver.go:74)

docker

var=1
for f in /root/work/yamls/mindreader/*.yaml
do
    if [ $START_BATCH -le $var ] && [ $END_BATCH -ge $var ]
    then
        name=dfuse-p1-b$var
        echo "Starting docker container $name | START_BATCH: $START_BATCH | END_BATCH: $END_BATCH | variable: $var"
        rm /root/work/dfuseeos/dfuse-data/mindreader/data/blocks/blocks.log
        docker run -d -v "/root/work/dfuseeos/dfuse-data:/dfuse-data" -v "/root/work/yamls/mindreader:/etc/dfuseeos-configs" -v "/root/work/dfuseeos/mindreader:/mindreader" --name $name dfuseeos_repo:latest /app/dfuseeos -c /etc/dfuseeos-configs/$name.yaml start -v
        sleep 150
    fi
    ((var++))
done

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 19 (12 by maintainers)

Most upvoted comments

Also, with this flag here: mindreader-log-to-zap: false you should see the nodeos logs passing through directly…

if it’s not in the logs, you need to add some logging level, ex: with -vv flag on command line or with environment variables like here: https://github.com/dfuse-io/dfuse-eosio/blob/develop/LOGGING.md

INFO=.* should do it. in worst case use DEBUG=.*

please try it with develop on hash 149d215 (current head)