gcsfuse: interrupted system call error while cloning repository in gcsfuse mounted directory

Hi Team.

I have mounted a gcs bucket on a directory in my ubuntu gce using gcsfuse. gcsfuse -o allow_other --gid=XXX --uid=XXX <gcs-name> <directory> where gid and uid are values of a user, which will also have access to this shared directory. The clone command used to clone the repository is git clone --branch <branch-name> --depth=1 --single-branch <repo-path> <destination-path>, but it fails with error

fatal: cannot create directory at '<path>': Interrupted system call

though it did downloaded some of the content from repository but keeps on failing every-time at different-different paths.

While running gcsfuse in debugging mode using flags --debug_fuse --debug_gcs, the error captured is

gcs: Req            0x3c7: -> StatObject("<a-file-path-in-repository>") (10.194465ms): Error in fetching object attributes: context canceled
FlushFile: interrupted system call, FileInode.Sync: StatObject: Error in fetching object attributes: context canceled
fuse_debug: Op 0x000033a0        connection.go:500] -> Error: "interrupted system call"
fuse: *fuseops.FlushFileOp error: interrupted system call

I am currently running on Ubuntu 20:04 and gcsfuse version 0.42.2 (Go version go1.19.5). Kindly help.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 5
  • Comments: 35 (2 by maintainers)

Most upvoted comments

Are there any updates on this? 😃

Hi @IchordeDionysos, we are still working on this and update here as soon as we have a fix available.

Hi @Chhinna,

We’re rolling out the changes behind a flag initially. You should be able to access them around May 27th, 2024 when our next release (v2.1.0) goes live. I’ll share the specific flags/configurations needed to enable the feature soon.

Thanks, Ashmeen

@marcoa6 @ashmeenkaur We have had it running for almost a month now, and it works like a charm. We haven’t run into any issues with it so far.

Hey @ashmeenkaur, I’ve just tested it and it works amazingly šŸ˜ Thanks for the work you all put into this šŸ‘Œ

I will wait until the new version and flag are available natively in Cloud Run (as it’s just easier to set up and configure).

Cloud Run will be updating to 2.1.0 once it is available. Please let us know the flag so we can prioritize having it allowlisted as well. Thanks.

With GCSfuse v2.3 this is now enabled by default. Thank you all for testing and providing feedback!

@jeremiele how can we stay up-to-date when this new version and the flag will be available in Cloud Run? 😃

Update: As of version 2.1 on 23-May-2024, we now offers enhanced control over how GCSfuse responds to interruptions during file system operations. This addresses running Git clone operations on a GCSFuse mounted directory.

You can configure GCSFuse to ignore interruptions during file system operations via CLI using –ignore-interrupts flag (disabled by default) or via config-file using the following config:

file-system:
    ignore-interrupts: true

Once we get enough feedback, we will enable this by default (targeting next month).

we’re hoping you can help us test the changes.

I’m a different person coming here for the same reason, namely, I want to support using git clone with a gcsfuse mount, and it doesn’t work by default. I just built the client from source, tried it with the ignore-interrupts option, and it works very well.

As an aside, I’ve been using gcsfuse for 5+ years, and hadn’t looked at it for a while, and I’m super impressed with the progress you have made recently, especially the improved support for caching to support ML workloads!

(Common update to issues #1016 #288, #539 and #562)

We have tracked the interrupt issue down to Golang sending SIGURG, which are then sent as interrupt signals to Fuse operations. https://github.com/golang/proposal/blob/master/design/24543-non-cooperative-preemption.md

I sent https://github.com/jacobsa/fuse/pull/153 to add an option to ignore interrupts. Once that’s approved, we can change gcsfuse to ignore interrupts by default.