atuin: "failed to decrypt history! check your key: could not encrypt" when attempting to connection third+ machines to Atuin

New install on Mint workstation

$ atuin -V
atuin 17.2.1

$ cat /etc/lsb-release 
DISTRIB_ID=LinuxMint
DISTRIB_RELEASE=21.2
DISTRIB_CODENAME=victoria
DISTRIB_DESCRIPTION="Linux Mint 21.2 Victoria"

$ grep -v -E "^#" .config/atuin/config.toml |sed -e '/^$/d'
sync_address = "https://atuin.my.inernal.dns.domain.goes.here"
enter_accept = true

$ zsh --version
zsh 5.8.1 (x86_64-debian-linux-gnu)

Installed in a Docker container using the docker compose YAML here: https://docs.atuin.sh/self-hosting/docker/

$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"

$ docker -v
Docker version 25.0.1, build 29cf629

$ docker compose version
Docker Compose version v2.24.2

$ docker compose logs
postgresql-1  | The files belonging to this database system will be owned by user "postgres".
postgresql-1  | This user must also own the server process.
postgresql-1  | 
postgresql-1  | The database cluster will be initialized with locale "en_US.utf8".
postgresql-1  | The default database encoding has accordingly been set to "UTF8".
postgresql-1  | The default text search configuration will be set to "english".
postgresql-1  | 
postgresql-1  | Data page checksums are disabled.
postgresql-1  | 
postgresql-1  | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgresql-1  | creating subdirectories ... ok
postgresql-1  | selecting dynamic shared memory implementation ... posix
postgresql-1  | selecting default max_connections ... 100
postgresql-1  | selecting default shared_buffers ... 128MB
postgresql-1  | selecting default time zone ... Europe/London
postgresql-1  | creating configuration files ... ok
postgresql-1  | running bootstrap script ... ok
postgresql-1  | performing post-bootstrap initialization ... ok
postgresql-1  | syncing data to disk ... ok
postgresql-1  | 
postgresql-1  | 
postgresql-1  | Success. You can now start the database server using:
postgresql-1  | 
postgresql-1  |     pg_ctl -D /var/lib/postgresql/data -l logfile start
postgresql-1  | 
postgresql-1  | initdb: warning: enabling "trust" authentication for local connections
postgresql-1  | You can change this by editing pg_hba.conf or using the option -A, or
postgresql-1  | --auth-local and --auth-host, the next time you run initdb.
postgresql-1  | waiting for server to start....2024-02-01 17:36:31.310 GMT [48] LOG:  starting PostgreSQL 14.10 (Debian 14.10-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
postgresql-1  | 2024-02-01 17:36:31.325 GMT [48] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgresql-1  | 2024-02-01 17:36:31.336 GMT [49] LOG:  database system was shut down at 2024-02-01 17:36:28 GMT
postgresql-1  | 2024-02-01 17:36:31.342 GMT [48] LOG:  database system is ready to accept connections
postgresql-1  |  done
postgresql-1  | server started
postgresql-1  | CREATE DATABASE
postgresql-1  | 
postgresql-1  | 
postgresql-1  | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgresql-1  | 
postgresql-1  | waiting for server to shut down...2024-02-01 17:36:31.947 GMT [48] LOG:  received fast shutdown request
postgresql-1  | .2024-02-01 17:36:31.952 GMT [48] LOG:  aborting any active transactions
postgresql-1  | 2024-02-01 17:36:31.955 GMT [48] LOG:  background worker "logical replication launcher" (PID 55) exited with exit code 1
postgresql-1  | 2024-02-01 17:36:31.956 GMT [50] LOG:  shutting down
postgresql-1  | 2024-02-01 17:36:31.978 GMT [48] LOG:  database system is shut down
postgresql-1  |  done
postgresql-1  | server stopped
postgresql-1  | 
postgresql-1  | PostgreSQL init process complete; ready for start up.
postgresql-1  | 
postgresql-1  | 2024-02-01 17:36:32.077 GMT [1] LOG:  starting PostgreSQL 14.10 (Debian 14.10-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
postgresql-1  | 2024-02-01 17:36:32.077 GMT [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
postgresql-1  | 2024-02-01 17:36:32.077 GMT [1] LOG:  listening on IPv6 address "::", port 5432
postgresql-1  | 2024-02-01 17:36:32.081 GMT [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgresql-1  | 2024-02-01 17:36:32.099 GMT [63] LOG:  database system was shut down at 2024-02-01 17:36:31 GMT
postgresql-1  | 2024-02-01 17:36:32.105 GMT [1] LOG:  database system is ready to accept connections

Registering and syncing works no issues.

$ atuin status
Atuin v17.2.1 - Build rev 9f79a34a9dea207df7184972fa29f4fe5ec41b27

[Local]
Sync frequency: 10m
Last sync: 2024-02-01 18:41:25.718872828 +00:00:00
History count: 26682
Deleted history count: 0

[Remote]
Address: https://atuin.my.inernal.dns.domain.goes.here
Username: my.user.name
History count: 26668

I then setup another machine running Ubuntu 22.04 and had no issues.

I then tried to connect another 12 different machines and all received the following error:

$ atuin sync
0/0 up/down to record store
thread 'main' panicked at /home/runner/work/atuin/atuin/atuin-client/src/sync.rs:74:38:
failed to decrypt history! check your key: could not encrypt

Location:
    atuin-client/src/encryption.rs:132:22
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The 12 servers are all running Ubuntu 22.04 or Debian 11.8. All clients are the same version of atuin as above.

On a broken host

➜ atuin status
Atuin v17.2.1 - Build rev 9f79a34a9dea207df7184972fa29f4fe5ec41b27

[Local]
Sync frequency: 10m
Last sync: 1970-01-01 0:00:00.0 +00:00:00
History count: 26416
Deleted history count: 0

[Remote]
Address: https://atuin.my.inernal.dns.domain.goes.here
Username: my.user.name
History count: 26673

$ sudo ntpstat
synchronised to NTP server (185.248.188.98) at stratum 2 
   time correct to within 33 ms
   polling server every 1024 s

About this issue

Original URL
State: closed
Created 5 months ago
Comments: 21 (11 by maintainers)

Most upvoted comments

Sure, I’ll let you know if that crops up again. Perhaps you can answer a question for me. All of the clients show the following:

[Remote]
Address: https://atuin.my.domain
Username: $username
History count: 36040

The history count doesn’t increase. Is that normal behaviour? I would think the server side history should keep increasing?

sawasy on Feb 5, 2024

That’s correct! It’s still the version set in cargo.toml, until we release v18

ellie on Feb 5, 2024

No problem. Makes sense. This is a private setup, so I have no issues setting that param. Doing so solved the error in the logs, as such:

atuin-1       | 2024-02-01T20:51:58.188285Z DEBUG add{user.id=1}: atuin_server::handlers::history: request to add 1100 history items       
atuin-1       | 2024-02-01T20:51:58.455807Z DEBUG add{user.id=1}: atuin_server::handlers::history: request to add 1100 history items       
atuin-1       | 2024-02-01T20:51:58.725881Z DEBUG add{user.id=1}: atuin_server::handlers::history: request to add 1100 history items       
atuin-1       | 2024-02-01T20:51:58.893482Z DEBUG add{user.id=1}: atuin_server::handlers::history: request to add 1100 history items       
atuin-1       | 2024-02-01T20:51:59.113754Z DEBUG add{user.id=1}: atuin_server::handlers::history: request to add 1100 history items       
atuin-1       | 2024-02-01T20:51:59.282909Z DEBUG add{user.id=1}: atuin_server::handlers::history: request to add 1100 history items       
atuin-1       | 2024-02-01T20:51:59.456829Z DEBUG add{user.id=1}: atuin_server::handlers::history: request to add 1100 history items       
atuin-1       | 2024-02-01T20:51:59.616004Z DEBUG add{user.id=1}: atuin_server::handlers::history: request to add 1100 history items      
atuin-1       | 2024-02-01T20:51:59.804288Z DEBUG add{user.id=1}: atuin_server::handlers::history: request to add 1100 history items
atuin-1       | 2024-02-01T20:51:59.958571Z DEBUG add{user.id=1}: atuin_server::handlers::history: request to add 1100 history items

But then when I try it again from a “broken” host, it failed unfortunately. This was the log output for the sync:

atuin-1       | 2024-02-01T20:52:19.632775Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:19.796953Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:19.961030Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:20.128851Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:20.294825Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:20.724189Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:20.891532Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:21.058994Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:21.222739Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:21.406477Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:21.587644Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:21.764065Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:21.930648Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:22.085158Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:22.246173Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:22.430259Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:22.749437Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:22.911571Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:23.078196Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:23.239752Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:23.388583Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:23.554927Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:23.725798Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:23.888271Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:24.039908Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 315 items of history for user 1
atuin-1       | 2024-02-01T20:52:24.316450Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:24.976791Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.020031Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.065040Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.110737Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.150339Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.194356Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.234294Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.272727Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.309987Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.346572Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.382133Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.419627Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.456487Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.492209Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.526472Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.561097Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.595073Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.628316Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.661605Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.693228Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.729957Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.761413Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.792679Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 1100 items of history for user 1
atuin-1       | 2024-02-01T20:52:25.821078Z DEBUG list{user.id=1}: atuin_server::handlers::history: loaded 315 items of history for user 1

Edit: For reference this is the command I used - atuin logout; atuin login -u <USERID> -p <PASSWORD> -k "<KEY>"; atuin sync

sawasy on Feb 1, 2024