gcsfuse: interrupted system call error while cloning repository in gcsfuse mounted directory
Hi Team.
I have mounted a gcs bucket on a directory in my ubuntu gce using gcsfuse.
gcsfuse -o allow_other --gid=XXX --uid=XXX <gcs-name> <directory>
where gid and uid are values of a user, which will also have access to this shared directory. The clone command used to clone the repository is git clone --branch <branch-name> --depth=1 --single-branch <repo-path> <destination-path>, but it fails with error
fatal: cannot create directory at '<path>': Interrupted system call
though it did downloaded some of the content from repository but keeps on failing every-time at different-different paths.
While running gcsfuse in debugging mode using flags --debug_fuse --debug_gcs, the error captured is
gcs: Req 0x3c7: -> StatObject("<a-file-path-in-repository>") (10.194465ms): Error in fetching object attributes: context canceled
FlushFile: interrupted system call, FileInode.Sync: StatObject: Error in fetching object attributes: context canceled
fuse_debug: Op 0x000033a0 connection.go:500] -> Error: "interrupted system call"
fuse: *fuseops.FlushFileOp error: interrupted system call
I am currently running on Ubuntu 20:04 and gcsfuse version 0.42.2 (Go version go1.19.5). Kindly help.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 5
- Comments: 35 (2 by maintainers)
Hi @IchordeDionysos, we are still working on this and update here as soon as we have a fix available.
Hi @Chhinna,
Weāre rolling out the changes behind a flag initially. You should be able to access them around May 27th, 2024 when our next release (v2.1.0) goes live. Iāll share the specific flags/configurations needed to enable the feature soon.
Thanks, Ashmeen
@marcoa6 @ashmeenkaur We have had it running for almost a month now, and it works like a charm. We havenāt run into any issues with it so far.
Hey @ashmeenkaur, Iāve just tested it and it works amazingly š Thanks for the work you all put into this š
I will wait until the new version and flag are available natively in Cloud Run (as itās just easier to set up and configure).
Cloud Run will be updating to 2.1.0 once it is available. Please let us know the flag so we can prioritize having it allowlisted as well. Thanks.
With GCSfuse v2.3 this is now enabled by default. Thank you all for testing and providing feedback!
@jeremiele how can we stay up-to-date when this new version and the flag will be available in Cloud Run? š
Update: As of version 2.1 on 23-May-2024, we now offers enhanced control over how GCSfuse responds to interruptions during file system operations. This addresses running Git clone operations on a GCSFuse mounted directory.
You can configure GCSFuse to ignore interruptions during file system operations via CLI using āignore-interrupts flag (disabled by default) or via config-file using the following config:
Once we get enough feedback, we will enable this by default (targeting next month).
Iām a different person coming here for the same reason, namely, I want to support using git clone with a gcsfuse mount, and it doesnāt work by default. I just built the client from source, tried it with the ignore-interrupts option, and it works very well.
As an aside, Iāve been using gcsfuse for 5+ years, and hadnāt looked at it for a while, and Iām super impressed with the progress you have made recently, especially the improved support for caching to support ML workloads!
(Common update to issues #1016 #288, #539 and #562)
We have tracked the interrupt issue down to Golang sending SIGURG, which are then sent as interrupt signals to Fuse operations. https://github.com/golang/proposal/blob/master/design/24543-non-cooperative-preemption.md
I sent https://github.com/jacobsa/fuse/pull/153 to add an option to ignore interrupts. Once thatās approved, we can change gcsfuse to ignore interrupts by default.