pyfilesystem2: fs.listdir and UnicodeError
On my system for whatever reason I have a file whith wrong encoding in the / dir.
Always if I want
for item in sorted(self.fs.listdir(_sel_dir)):
I have to encapsulate this by an exception for UnicodeDecodeError. I would prefer to not crash but just ignore this file.
(I am still looking on why that file anyway is there)
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 1
- Comments: 29 (22 by maintainers)
Commits related to this issue
- Use @pjdelport backports.os on Py2 #120 Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com> — committed to pombredanne/pyfilesystem2 by pombredanne 7 years ago
- Add new tests for non-unicode bytes paths #120 Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com> — committed to pombredanne/pyfilesystem2 by pombredanne 7 years ago
- Add optional "as_bytes" arg #120 Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com> — committed to pombredanne/pyfilesystem2 by pombredanne 7 years ago
- Use fsencoded paths on *nix #120 The approach is that unicode is used everywhere unless when on *nix and that real access to files is needed. In this case the patch is encoded to bytes using the file... — committed to pombredanne/pyfilesystem2 by pombredanne 7 years ago
- Ensure tests pass on Python3 #120 Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com> — committed to pombredanne/pyfilesystem2 by pombredanne 7 years ago
- fsencode only on *nix and Python2 #120 Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com> — committed to pombredanne/pyfilesystem2 by pombredanne 7 years ago
- Remove as_bytes arg from getsyspath #120 Instead I added doc to explain that fsencode how can be used if needed. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com> — committed to pombredanne/pyfilesystem2 by pombredanne 7 years ago
- Refactor common checks in _get_validated_syspath() #120 * Avoid code duplication with a new _get_validated_syspath() method * Remove as_bytes arg from getsyspath #120 Signed-off-by: Philippe Ombre... — committed to pombredanne/pyfilesystem2 by pombredanne 7 years ago
- Remove as_bytes arg #120 * This was mistakenly left over * Remove as_bytes arg from getsyspath #120 Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com> — committed to pombredanne/pyfilesystem2 by pombredanne 7 years ago
- Pin backports.os with correct Python version #120 Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com> — committed to pombredanne/pyfilesystem2 by pombredanne 7 years ago
- Improve Python idioms and variable overriding #120 Following @willmcgugan in #121 this is: - removing and/or shortcuts and - does not override path arg variables Signed-off-by: Philippe Ombredanne ... — committed to pombredanne/pyfilesystem2 by pombredanne 7 years ago
- Cleanup scandir bytes vs unicode handling #120 * I had somehow introduced a regression with the previous commit Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com> — committed to pombredanne/pyfilesystem2 by pombredanne 7 years ago
listdirin the FS interface should definitely return unicode. It’s a guarantee made by the api which tries to isolate the developer from precisely this kind of real world nastiness. The boundary points would have to be internal where OSFS callslistdirandscandirgetsyspathcould be an exception as it only needs to be a ‘path understood by the OS’. So I’d consider allowing that to return bytes, but I’m not sure what that would break off hand.Maybe a
getnativepathwould be warranted? Would much rather add to the api than risk break anything.