youtube-dl: «File name too long» should not error
Checklist
- I’m reporting a feature request
- I’ve verified that I’m running youtube-dl version 2021.06.06
- I’ve searched the bugtracker for similar feature requests including closed ones
Description
To reproduce:
- try to download any video with a really long title
- you get
ERROR: unable to open for writing: [Errno 36] File name too long: - the video is not downloaded
My suggestion:
- instead of an
ERRORput aWARNING - download the video regardless, just truncate the title to an appropriate size, similar to what suggested in #29627
This will improve UX, check bug tracker to see how common this issue is.
#7765 talks about this too (with a linked pull request) but:
- does not elaborate on whether it is actually a good thing / possible drawbacks
- is limited to collaborators
About this issue
- Original URL
- State: open
- Created 5 years ago
- Reactions: 7
- Comments: 33 (17 by maintainers)
Commits related to this issue
- Shorten title attribute for #29912 — committed to palewire/youtube-dl by palewire 4 years ago
- Add "Name too long" error handler, fix bugs If the file name is too long, its word-by-word name is cuting until it is acceptable for filesystems (255 bytes) (https://github.com/ytdl-org/youtube-dl/i... — committed to NikitaBeloglazov/ytcon by NikitaBeloglazov 3 years ago
But my ticket is addressing a different point. Shrinking/truncating long filenames should be default. It would be an immense step forward in terms of UX to go to «it just works, emits a warning» from «it does not work, errors out; search the error / skim the FAQ».
Of course, if the user has specific needs, they should be able to turn this off and get an error. (maybe via
--fixupoption).Il 06 settembre 2021 alle 19:33 dirkf ha scritto:
This is particularly annoying when downloading videos from Twitter as it takes the whole post as file name. I usually use --id and rename the file manually.
Yeah, having the tool just fail on long file names without the user applying different parameters to mitigate the issue is very rational and an awesome user experience. I mean just defaulting to a template that causes video downloads not to fail with reasonable truncation would be ridiculous. The legions of people coming to the Issues and hopefully reading the FAQ before creating this issue repeatedly shows the wisdom of this decision. 👍
Plainly it’s not a showstopping issue but it is one that generates repeated complaints. It’s still open in case someone is eventually intrigued enough to work out a solution that meets an adequate subset of the problem constraints, after spending however long it takes locked in the basement with the necessary cold towels wrapped round the head. But we should fix Twitter/X if the extractor is ever viable and updated post-Musk.
--trim-filenamesis a poorly implemented solution from youtube-dlc and I would not recommend porting it as-is. Even in yt-dlp, I recommend users to use the % formatting to control filename length instead. E.g. https://github.com/yt-dlp/yt-dlp/issues/3494This is actually a bug and shows how poorly implemented the option is! See https://github.com/yt-dlp/yt-dlp/issues/2314. Do not rely on this behavior…
The general solution to this is a (new) FAQ entry
I don’t see that
--trim-filenames ...is much of an improvement on setting an explicit length for a problem field (title, I’m looking at you) in the output template.A good solution would:
Clearly these are not completely consistent requirements, as an output template might be, say,
%(id)16777216s.%(ext)s. Some prioritisation is necessary.I still hold that the excessive filename length provoked by Twitter is an extractor bug.
It works the other way. Just set maximum lengths for the fields in the output template that will meet the limits of your OS/filesystem, as well as any others to which you might copy the downloaded file. You should do this for the documented default (set in a config file), and for any specific output template that you pass in
--output/-o .... Then any mutilation of template fields is entirely under your control.The combined abilities of the many contributors to this project and yt-dlp have not so far found an acceptable solution to the general problem posed by this issue. I don’t expect the issue to be closed soon or ever, but who knows?
Some fields are more important than others, like the unique ID’s, hashes, and file extensions can never be allowed to be truncated. As few as possible known fields could have an importance priority floating point number internally assigned in the extractor.
^ Then, generally, you can truncate all other fields equally to fit, as it’ll be more important to have some knowledge of the contents in the filename than have none at all, and 125, not to mention 255, chars per segment (or even a full path) is a lotta text. If the User is hellbent on changing a sensible default into something very much non-sensible like
ID + hash + keywords/tags + collection name + description, that’s their fault.Something like that.
@bryanpaget, you might like to reveal more details of how you “[ran] into this issue”, or is it just Twitter again?
Also, feel free to review the discussion in PR #29989 to understand why this is a difficult issue to fix retrospectively (other than by following the approach in https://github.com/ytdl-org/youtube-dl/issues/29912#issuecomment-913945852), and possibly contribute to the design of a general solution.
The defaults really do work in the vast majority of cases.
You can write a config file with your preferred options and select it with
--config-locationin your script if some default setting should not be satisfactory (or just make it your default config).For instance, you could put
-o %(title).50s-%(id)s.%(ext)sto avoid downloads being broken by unreasonably long titles.If anyone has a suggestion for what (reliably concise) metadata item could be used as the title for media extracted from Twitter, we could make a PR for Twitter specifically.
I don’t see any other reasonable solution for the Twitter extractor. Maybe it could pre-truncate the post? That’s not any better than just doing it later when constructing the output file name though.
I’ve run into this issue many times, as I usually want to use
youtube-dlto quickly download a video to my downloads folder and I don’t really care about the name but would prefer it give as much information as possible (so I don’t want to just default to using the id).I would really, really love it if somebody made a
youtube-dlflag--just-download-it-seriously-i-dont-care-about-the-details-just-do-your-best-i-believe-in-you, bonus points if that’s the default, but even just having it available so I can write a wrapper script or bash alias that uses it would be amazing.#30839: “if you were actually running yt-dlp …”
Also, 90 isn’t a “constant”: pick a number that works.
Depends on who/what you blame.
I’ve just tonight run into this bug with the Twitter extractor. You can reproduce it with the following:
Would you be open to a Twitter specific workaround patch? Perhaps the title could be changed to a combination of the uploader name and date, with the tweet text moved to the description field?
The previous discussions on this point show that the maintainers (Sergey, at least) won’t accept any solution that they see as ad hoc. The project sees the
idfield in particular as inviolable, and won’t accept a solution that might accidentally truncate anidembedded in a filename. However, if the project’s maintainers should be permanently absent, the project might be adopted by maintainers with different views.It could have been reasonable to add some sort of filename truncation functionality to the
--restrict-filenamesoption, where it has already been accepted to modify embedded fields, but the existing definition of that option doesn’t allow it.The cross-platform Python APIs don’t offer a way of telling if the proposed file name is too long. In general this is a tricky issue because the error is only found once the OS kernel has called into the filesystem driver for the device that is to be written to.
We could try modifying
sanitize_open()inutils.py, s/t like below:Concerns here:
There might also be additional guidance for developers to avoid extracting
titleandidfields that might exceed typical filename lengths.