restic: Super slow backup to B2

Hey now !!! I just tested 0.8.3 and it’s finally working as expected !!! Great job to the developers. I’m still not sure why people didn’t have issues with 0.8.2 but that version was definitely slow for B2. But I am pleasantly pleased with 0.8.3. I just backed up 600MB consisting of 2800 files in only 3 minutes 20 seconds !!! Again kudos and GREAT JOB to all those responsible. You gotta winner !

B2 is known to have limited bandwidth per connection, I’ve seen this same issue with other backup software as well. The solution is always to use multiple connections to upload. I’m glad I found this thread because I didn’t know about the b2.connections option. Where are all the options documented?

I tried the latest dev version as suggested by @fd0. It was a bit faster (and I like the new output details) and the rest of the 300GB are done now after one day, so I can’t really reproduce it. But before (regular 0.8.3) lsof showed ten open files, using the dev version it opened only 2-3 files at the same time. I don’t know how much connections to B2 it opened. I did log iostat and vnstat every minute during the backup with the dev version. Gonna check the files and report in a new issue if I find something.

Just now I tried a restic check which failed because it said, that a lock is in place which was created five days ago. That was about the day and time when the first try of the 300GB backup was interrupted. Can’t imagine if that lock could slow down the backup process, just something I found. As it’s probably hard to reproduce now, digging in the dark isn’t worth the time I guess. Like @fd0 said, I’ll open another issue if it occours again.

I can confirm that 0.8.2 is quickest for me. The upload speed varies with B2, just the same now as it does with S3.

I’m currently uploading 0.01% every 18 seconds. I have 224GB I’m uploading.

Its taken 21hrs to upload 62% which is about 138GB.

I may have found a redundant call to one of the class B APIs. I’ll try to confirm and get a fix out tonight.

Hi, Adding my feedback here. My server is in Amsterdam, upload speed is about 100Mb/s. Restic version 0.7.3

I’m backing up my nextcloud data folder, about 65-70GB of various size unencrypted files I created to repositories, one local and one on B2. The local one seems to process the data at ~5MB/s

restic backup -r ./resticbackup /srv/nextcloud
scanned 6320 directories, 47379 files in 0:00
[2:53] 1.12%  4.326 MiB/s  748.373 MiB / 65.470 GiB  619 / 53699 items  0 errors  ETA 4:15:47

When I run the same initial backup to B2, after a short burst, the speed quickly goes down

I’ve waited a few minutes for the speed to go down after the burst and it sat around 1MB/s. It’s confirmed by using the tool “nethogs” to monitor bandwith by process, restic seems to oscillate between 500KB/s and 2000KB/s

I then used rclone with no special options to upload a similar folder structure as the backup repository and got around the same speeds

It oscillated mostly between 500 and 1500 KB/s during my tests

So imho the issue is not with restic itself but with B2.

Edit:

I’ve re-tried the backup with a huge B2 connections (-o b2.connections=50) and the speed drastically increased. I’m currently backup-ing at nearly 10MB/s (between 6.5MB and 10MB per second) so the limit seems to be per-connection to the API