ripgrep: Cannot search on cygwin when explicit path is given
Hi,
when I try to search by giving ripgrep a specific folder, I get an error:
knezn % rg test ~/work/analytics-grid
/cygdrive/c/Users/knezn/work/analytics-grid: The system cannot find the path specified. (os error 3)
No files were searched, which means ripgrep probably applied a filter you didn't expect. Try running again with --debug.
knezn % rg test --debug ~/work/analytics-grid
DEBUG:grep::search: regex ast:
Literal {
chars: [
't',
'e',
's',
't'
],
casei: false
}
DEBUG:grep::literals: literal prefixes detected: Literals { lits: [Complete(test)], limit_size: 250, limit_class: 10 }
/cygdrive/c/Users/knezn/work/analytics-grid: The system cannot find the path specified. (os error 3)
No files were searched, which means ripgrep probably applied a filter you didn't expect. Try running again with --debug.
knezn % rg test --debug /cygdrive/c/Users/Knezn/work/analytics-grid
DEBUG:grep::search: regex ast:
Literal {
chars: [
't',
'e',
's',
't'
],
casei: false
}
DEBUG:grep::literals: literal prefixes detected: Literals { lits: [Complete(test)], limit_size: 250, limit_class: 10 }
/cygdrive/c/Users/Knezn/work/analytics-grid: The system cannot find the path specified. (os error 3)
No files were searched, which means ripgrep probably applied a filter you didn't expect. Try running again with --debug.
knezn % rg test /cygdrive/c/Users/knezn/work/analytics-grid
/cygdrive/c/Users/knezn/work/analytics-grid: The system cannot find the path specified. (os error 3)
No files were searched, which means ripgrep probably applied a filter you didn't expect. Try running again with --debug.
I’m on windows 7, running ripgrep 0.3.1 (expanded from here). My terminal is ConEmu, which runs zsh under cygwin 2.6.0-1.
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 30 (14 by maintainers)
we need rust cygwin target
@jibal I appreciate your apology, thanks. Let’s try to avoid statements about people, and also try to resist the temptation to wax poetic about Rust’s “maturity” in the systems programming space just because it doesn’t satisfy a particular user need. Telling us that we “don’t understand cygwin” isn’t helpful. Instead, let’s try to adopt a more supportive relationship and help each other fill the gaps. Certainly, I am not a Windows user. However, I do use cygwin whenever I do use Windows. That time is predominantly spent doing QA, but not just for ripgrep, but for anything that I intend to have run on Windows. Figuring out how path translation works and when it happens is certainly something that an end user comes to appreciate very quickly, but figuring out where exactly it happens in the stack of abstractions in the cygwin environment was not easy for me because I don’t have a strong familiarity with how cygwin is built. My understanding has improved over time though.
Essentially, what @retep998 says is accurate. The “correct” way for ripgrep to support cygwin is for the Rust distribution to provide a cygwin target. Such a thing does not exist, and as far as I know, nobody has ever championed such an idea. The Rust ecosystem does support GNU/MSYS2 builds, and to respond to your update at the top of your initial comment, ripgrep does actually distribute Windows binaries built with the GNU toolchain on Windows (in addition to binaries built with the MSVC toolchain). However, this does not resolve the path translation problems. As I understand it, path translation is fundamentally a cygwin thing, and the MSYS2 toolchain is not coupled to cygwin. But hey—I could be wrong here and maybe there is an easy answer lurking, but I doubt it.
In terms of championing a new cygwin target, and assuming one would be lobbying for the Rust project to take over maintenance of it, I suspect they would need some pretty compelling motivation for doing so. While I could be wrong, my perspective is that such a target would be for supporting cygwin environments as an end to itself. Popping up a level, my understanding of why cygwin even exists in the first place is to provide a compatibility layer for traditional Unix programs to run on Windows with minimal or zero modifications to the source code. From Wikipedia’s article on Cygwin:
At that level, cygwin isn’t a particularly compelling target to add to the Rust project because the Rust standard library was designed to be compatible with Windows from the very beginning. That is, from the perspective of Rust programmers, POSIX APIs tend to be implementation details of the standard library. That is, the standard library provides cross platform APIs that exist independently of POSIX. So the specific motivation that caused cygwin to exist in the first place doesn’t necessarily exist in the Rust ecosystem. Of course, this isn’t to detract from cygwin compatibility as an end unto itself; it would certainly be useful. However, the underlying motivation probably isn’t as compelling as it is for traditional Unix programs written in C. I suspect this is the key underlying reason why cygwin support has not materialized as a first class target in the Rust ecosystem. It is possible someone could build it, but I don’t know who would do it. As @retep998 mentions, your problems don’t end at the standard library. The entire library ecosystem effectively treats
#[cfg(windows)]as “this is where we use Windows API functions,” which would effectively bypass any POSIX compatibility layer built into the standard library. That is, the Rust standard library does not completely cover the entire Windows API. It’s up to the library ecosystem to provide less common facilities. So all of those library crates would need to be updated. Overall, it’s a tractable amount of work, but I’d still say it’s on the order of “many man months” in addition to lots of collaboration with folks and working toward consensus for how the cygwin target should be enabled. In other words, it’s a project that requires significant amounts of both technical and social capital.In light of the aforementioned status on a cygwin target, the other route we could explore would be specific mitigations in ripgrep for the most common cygwin problems, where path translation is undeniably at the top of that list. As I understand it—and this has been discussed in a few places on the issue tracker over the years—there are effectively three options for fixing this particular bug outside of adding a new cygwin target to Rust proper:
cygpathexecutable for path translation.A key problem/distinction between the above three approaches and how a POSIX compatibility layer would work is that the above is very explicit and the most logical place for it to happen is when the program starts. That is, ripgrep could do path translation on all the paths provided by the end user at the command line, and then use the result of translation throughout the rest of program execution while forgetting about the original path provided by the end user. The problem with this, as I understand it, is that it could cause problems with automatic filtering based on gitignore logic. In particular, last time I checked, I think the output of path translation is an absolute path, and it would be difficult to square this with gitignore filtering. The corner cases seem gnarly here. An alternative approach would be to perform path translation lazily and only immediately before calling into Windows APIs. This is, AIUI, effectively how the POSIX compatibility layer would work. This would probably be ideal to be honest, but it requires cooperation from everyone and all dependencies, which may be hard to achieve in practice (it’s a fairly similar problem to updating the ecosystem to support a hypothetical new cygwin target).
The bottom line here is that cygwin is a compatibility layer specifically designed for C programs written against the POSIX API. Reasonable assumptions were made, and getting all of the pieces to fit neatly in a completely different ecosystem that doesn’t treat POSIX as a first class API is tricky business. It has nothing to do with “maturity” or even “insularity.” It’s about different design decisions and priorities.
In principle, I am open to simple solutions to this problem that don’t dramatically increase maintenance burden or introduce unreliable behavior that results in new streams of bugs.
Hey @BurntSushi ,
Not really. The environment is Cygwin, which is a POSIX environment which happens to be running on Windows. It’s best treated as yet another Unix variant.
The vast majority treat it as yet another Unix variant. That’s really the whole reason Cygwin exists at all. So e.g., if you’re at a Cygwin bash prompt and you type:
… you don’t get a Windows path, you get
/home/Gary- which is actuallyC:\cygwin64\home\Garyvia the Cygwinmountmechanism. Any tools compiled against the Cygwin DLL are going to be getting and expecting POSIX-oid filesystem semantics, and for the most part don’t need to care that they’re running on Cygwin vs. say Linux or BSD or whatever. There of course are exceptions; e.g. Cygwin has gone to great pains to make some things such asls 'C:\'mostly do the right thing usually. But in general Cygwin apps don’t work well/at all with native Windows paths directly.You shouldn’t need to (but I don’t know how Rust works here, you may not have a good option). A “ripgrep for Windows” should be built such that it doesn’t know what Cygwin is. Likewise, a “ripgrep for Cygwin” would be built in the same way you’d build it for any other POSIX platform, and it wouldn’t know what Windows is (other than what leaks through the Cygwin DLL layer). The latter would require a “Rust for Cygwin” to exist, or at least a Rust which could cross-compile to Cygwin.