butler: Fix wharf's resumable upload client behavior + collect error traces in case of service degradation
I’m getting the following error while trying to push new version of a file.
$ butler push linux quyse/flaw:model-editor-linux
> For channel `model-editor-linux`: last build is 3183, downloading its signature
> Pushing 8.0 MB (1 files, 0 dirs, 0 symlinks)
*errors.errorString itch.io API error: uploaded file does not exist in storage (build/4513/13440-signature-default: 404)
/usr/local/go/src/runtime/asm_amd64.s:1998 (0x490431)
What does it possibly mean? I’ve recently upgraded docker container which is pushing builds during CI, but I don’t think I’ve changed anything about butler.
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 91 (1 by maintainers)
Commits related to this issue
- Retry HTTP 408, 500, 502, 503 and 504. https://github.com/itchio/butler/issues/71 — committed to itchio/wharf by fasterthanlime 8 years ago
- Retry HTTP 408, 500, 502, 503 and 504. https://github.com/itchio/butler/issues/71 — committed to itchio/butler by fasterthanlime 8 years ago
- :arrow_up: wharf (retry on 308, cf. #71) — committed to itchio/butler by fasterthanlime 8 years ago
- Proper handling of 308 from GCS, query status logic on 5xx/408 errors, cf. https://github.com/itchio/butler/issues/71 — committed to itchio/wharf by fasterthanlime 8 years ago
- Deferred resumable code, see suggestion in #71 — committed to itchio/butler by fasterthanlime 8 years ago
I’m closing the issue since butler is now doing the right thing, we’re just waiting for the service to be restored fully.
butler 0.18.0 is out and uses @Capstan’s suggested approach.
Everyone is encouraged to upgrade as soon as possible. I’ve posted a full post-mortem if anyone’s curious!
@Capstan I just added a “deferred_resumable” upload type to our client, which is your “signed URL” suggestion, and the results are just beautiful:
We’ll be deploying that and urging our users to upgrade asap. (Figuring out the original issue would be nice too!)
@IBwWG unfortunately network errors seem to have come back full swing. on the plus side, this makes the client-side fixes that much easier to test ⛄
edit: just as I’m posting that, I’m not getting any errors on my test uploads. conditions seems to be changing every hour.
Confirmed, back to normal, no errors. 😃
Right, but I still think butler should only warn about not being able to chmod, not straight up fail 😃 (if it has read access, that is)