ArchiveBox: Bugfix: docker-compose instructions create a sonic container that fails to start

Describe the bug

I followed the docker-compose instructions from the README. This is the result:

[root@Acheron archivebox]# docker-compose ps
         Name                        Command                State             Ports
--------------------------------------------------------------------------------------------
archivebox_archivebox_1   dumb-init -- /app/bin/dock ...   Up         0.0.0.0:8000->8000/tcp
archivebox_sonic_1        sonic -c /etc/sonic.cfg          Exit 101
[root@Acheron archivebox]# docker-compose logs sonic
Attaching to archivebox_sonic_1
sonic_1       | thread 'main' panicked at 'cannot read config file: Os { code: 21, kind: Other, message: "Is a directory" }', src/config/reader.rs:24:14
sonic_1       | note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
sonic_1       | thread 'main' panicked at 'cannot read config file: Os { code: 21, kind: Other, message: "Is a directory" }', src/config/reader.rs:24:14
sonic_1       | note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Search seems to work anyway.

I would expect one of:

a. sonic container is not created by default if it requires the user to manually create a config and is not necessary to run ArchiveBox b. config.cfg is created for me by the init script, using the environment variable I set in the docker-compose file c. config.cfg is not required by sonic (however, this is not the case: https://github.com/valeriansaliou/sonic/issues/197)

Steps to reproduce

From the README:

# create a new empty directory and initalize your collection (can be anywhere)
mkdir ~/archivebox && cd ~/archivebox
curl -O https://raw.githubusercontent.com/ArchiveBox/ArchiveBox/master/docker-compose.yml
docker-compose run archivebox init
docker-compose run archivebox --version

# start the webserver and open the UI (optional)
docker-compose run archivebox manage createsuperuser
docker-compose up -d
open http://127.0.0.1:8000

# you can also add links and manage your archive via the CLI:
docker-compose run archivebox add 'https://example.com'
docker-compose run archivebox status
docker-compose run archivebox help  # to see more options

ArchiveBox version

[root@Acheron archivebox]# docker-compose run archivebox --version
Starting archivebox_sonic_1 ... done
Creating archivebox_archivebox_run ... done
ArchiveBox v0.5.3
Cpython Linux Linux-5.9.1-arch1-1-x86_64-with-glibc2.28 x86_64 (in Docker)

[i] Dependency versions:
 √  ARCHIVEBOX_BINARY     v0.5.3          valid     /usr/local/bin/archivebox
 √  PYTHON_BINARY         v3.9.1          valid     /usr/local/bin/python3.9
 √  DJANGO_BINARY         v3.1.3          valid     /usr/local/lib/python3.9/site-packages/django/bin/django-admin.py
 √  CURL_BINARY           v7.64.0         valid     /usr/bin/curl
 √  WGET_BINARY           v1.20.1         valid     /usr/bin/wget
 √  NODE_BINARY           v15.5.1         valid     /usr/bin/node
 √  SINGLEFILE_BINARY     v0.1.14         valid     /node/node_modules/single-file/cli/single-file
 √  READABILITY_BINARY    v0.1.0          valid     /node/node_modules/readability-extractor/readability-extractor
 √  MERCURY_BINARY        v1.0.0          valid     /node/node_modules/@postlight/mercury-parser/cli.js
 √  GIT_BINARY            v2.20.1         valid     /usr/bin/git
 √  YOUTUBEDL_BINARY      v2021.01.03     valid     /usr/local/bin/youtube-dl
 √  CHROME_BINARY         v87.0.4280.88   valid     /usr/bin/chromium
 √  RIPGREP_BINARY        v0.10.0         valid     /usr/bin/rg

[i] Source-code locations:
 √  PACKAGE_DIR           22 files        valid     /app/archivebox
 √  TEMPLATES_DIR         3 files         valid     /app/archivebox/themes

[i] Secrets locations:
 -  CHROME_USER_DATA_DIR  -               disabled
 -  COOKIES_FILE          -               disabled

[i] Data locations:
 √  OUTPUT_DIR            6 files         valid     /data
 √  SOURCES_DIR           1 files         valid     ./sources
 √  LOGS_DIR              0 files         valid     ./logs
 √  ARCHIVE_DIR           1 files         valid     ./archive
 √  CONFIG_FILE           81.0 Bytes      valid     ./ArchiveBox.conf
 √  SQL_INDEX             204.0 KB        valid     ./index.sqlite3
[root@Acheron archivebox]# docker version
Client:
 Version:           20.10.2
 API version:       1.40
 Go version:        go1.15.6
 Git commit:        2291f610ae
 Built:             Tue Jan  5 19:56:21 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          19.03.13-ce
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.15.2
  Git commit:       4484c46d9d
  Built:            Sat Sep 26 12:03:35 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.4.1.m
  GitCommit:        c623d1b36f09f8ef6536a057bd658b3aa8632828.m
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
[root@Acheron archivebox]# docker-compose version
docker-compose version 1.27.4, build 40524192
docker-py version: 4.3.1
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 28 (14 by maintainers)

Most upvoted comments

Thanks @jdcaballerov, #625 fixes all of the false negatives I was seeing (and a few I didn’t catch before). I think all of the false positives can be chalked up to sonic’s fuzzy search which I wasn’t aware of at first.

A search for scar matches https://www.radiomods.co.nz/kenwood/kenwoodts440.html

jack fails to match that page but matches http://tarpn.net/t/faq/faq_networking_on_purpose.html and http://tarpn.net/t/faq/faq_packet_radio.html

brass, dirt, and mantra fail to match https://teddit.net/r/WritingPrompts/comments/5kxe94/wp_you_live_in_a_world_where_each_lie_creates_a/

dirt also matches https://nwavguy.blogspot.com/2011/07/o2-headphone-amp.html

I’m running the suggested docker-compose config with PDF, screenshot, DOM, readability, and archive.org saving turned off.

I added example.com as well, and am seeing the same behavior:

I know you mentioned “Sonic will only index the newly added links after it’s enabled,” so I think it should index this? And since the public search is returning, it seems unlikely that indexing is broken?

I apologize if I am missing something obvious, or keeping anyone up. This is certainly not urgent.

@JohnMaguire rebuilding the index is a task that is managed by sonic and doesn’t occur immediately after being instructed to do so. Allow some time without killing it and let us know.

After deleting the automatically generated directory and copying https://github.com/valeriansaliou/sonic/blob/master/config.cfg to ./etc/sonic/config.cfg, I get the following error when running docker-compose up -d… it seems this is because the original Sonic container is not recreated, and has the erroneous directory in it. Removing it manually by finding it in docker ps -a, then docker rm <id> and then running docker-compose up- d again fixes it:

[root@Acheron archivebox]# docker-compose up -d
Starting archivebox_sonic_1 ... error

ERROR: for archivebox_sonic_1  Cannot start service sonic: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: rootfs_linux.go:59: mounting "/opt/docker/projects/archivebox/etc/sonic/config.cfg" to rootfs at "/var/lib/docker/overlay2/13cc28834c925d7a3565efeb8f3b487349a039735d495dd48e1817cc67f39e57/merged/etc/sonic.cfg" caused: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

ERROR: for sonic  Cannot start service sonic: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: rootfs_linux.go:59: mounting "/opt/docker/projects/archivebox/etc/sonic/config.cfg" to rootfs at "/var/lib/docker/overlay2/13cc28834c925d7a3565efeb8f3b487349a039735d495dd48e1817cc67f39e57/merged/etc/sonic.cfg" caused: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
ERROR: Encountered errors while bringing up the project.