spider: Operation not permitted from tokio-uring-0.4.0 in docker container
Hey there. I started to get following error while crawling;
thread '<unnamed>' panicked at /usr/local/cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-uring-0.4.0/src/lib.rs:230:46:
called `Result::unwrap()` on an `Err` value: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }
I started to get this error after recently updated spider package to latest(2.24.6). Running in docker container. If I set container to run in privileged mode, then it works fine. This is after v2.23.7 which introduces tokio-uring. spider = { version = “2.22.16”, features = [“reqwest_native_tls_vendored”] } works fine. spider = { version = “2.24.6”, features = [“reqwest_native_tls_vendored”] } gives the error.
I tried to disable io_uring flag in cargo.toml as mentioned in 2.23.7 release notes but I guess there is no easy way to only disable one feature flag keeping default ones(sorry, rust noob here).
Is there anything more to follow in a container environment to work with tokio-uring?
docker version
Client:
Version: 27.3.1
API version: 1.47
Go version: go1.23.1
Git commit: ce1223035a
Built: Wed Sep 25 14:43:43 2024
OS/Arch: linux/amd64
Context: default
Server:
Engine:
Version: 27.3.1
API version: 1.47 (minimum version 1.24)
Go version: go1.23.1
Git commit: 41ca978a0a
Built: Wed Sep 25 14:43:43 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v2.0.1
GitCommit: 88aa2f531d6c2922003cc7929e51daf1c14caa0a.m
runc:
Version: 1.2.3
GitCommit:
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker image: rustlang/rust:nightly
And installed these packages in container build;
libssl-dev openssl pkg-config ca-certificates
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 15 (7 by maintainers)
Yes, even though the docker container is using a linux image the machine compiling the image also needs to be linux with the correct kernel for it. No problem, ask any questions anytime!
Disabling seccomp
seccomp=unconfinedallows you to useio_uringin containers https://github.com/containers/podman/issues/16796Ok, I will try and let you know, thanks 🙏
On Wed, Jan 8, 2025, 16:14 Jeff Mendez @.***> wrote:
What do you mean by “compile the container on linux”? Compile the docker?
I can confirm that with
spider = { version = "2.24.8", features = ["basic"], default-features = false }it works without failing on io_uring/tokio_uring. Thanks for the fast communication/operation.For the your comment on:
I am on archlinux with 6.12 kernel. Compiling my app with spider in docker:rolling(vscode devcontainer). This is a RAG app that utilizes crawling capability of spider and does a lot of other stuff. And deploying to kubernetes. But currently io_uring is blocked by builtin seccomp profile in docker by default. I haven’t had a deep search on if I can enable aws eks to allow enabling io_uring, but I doubt it.
Again, thanks for fast communication and operation!