term-image: Scanning directories and loading animated images too slow
Description
First of all, great work. I’m really interested on it. Now, everything goes fine when you open a single file or several files using a glob expression (e.g. ~/Downloads/*.png
). However, whenever you want to browse a directory, it takes too long (like 20 secs or more) to load the TUI viewer (while recursively scanning image files in the given directory). Once the TUI finally loads, it is also a bit slow whenever you hover a directory (a few seconds depending on the directory).
To Reproduce
term-img DIR
or term-img DIR/*
Expected behavior
term-img
should be quicker and smoother when recursively browsing directories
Desktop:
- OS: Arch
- Kernel version: Linux 5.15.12 (libre-hardened)
Package info:
- Python version: 3.10.2
- Package version: 0.1.1
- Installation method: clone repo and then
pip install
Terminal Emulator:
- Name: Tested on Xterm (370) and Alacritty (0.10.0)
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 23 (14 by maintainers)
Commits related to this issue
- Significantly upgraded directory grid display - Fix: Eliminated delay when switching to directory entries with much image contents. - Addresses #10. - Switching between directory entries is now j... — committed to AnonymouX47/term-image by AnonymouX47 2 years ago
- Improved menu scanning - Change: Increased fuildity and responsiveness when switching menus. - Futher adresses #10. - Change: Eliminated unnecessary waits/blocks. - Removed while loop blocks and ... — committed to AnonymouX47/term-image by AnonymouX47 2 years ago
- [tui] Changed TUI animation frame caching criteria - Change: Caching of animated image frames in the TUI is now based on image file size instead of frame count. - Adresses #10. NOTE: This change is ... — committed to AnonymouX47/term-image by AnonymouX47 2 years ago
Thanks! That would be great.
Hey @AnonymouX47! Sorry for the delay.
It’s much better now! Congrats and thanks! I really like this. I guess PIL internally uses something along the lines of cacalib or chafa (i.e., ASCII/ANSI rendering). Do you plan to add support for ueberzug, w3img, sixel or kitty? These protocols are really good at displaying images on the terminal, and besides they are more and more widely adopted.
I myself implemented a files previewer (not only images, but also PDF, document, postscript files, and even sound files) using fzf and ueberzug for my CliFM, but only as a plugin (a shell script indeed): it’s nice and all, but far from ideal. A third party utility able to do this smoothly, and able to be integrated into CliFM, would be really nice. And
term-img
is on the right track for sure.Keep up the good work!
In addition to these… I’ll also be implementing the following before releasing 0.2.0:
Hello @leo-arch !
It’s definitely been a while 😃 but the good news is… time hasn’t passed in vain.
Here is an highlight of the performance-related fixes and improvements I’ve implemented since, as regards this issue:
For the library:
TermImage.n_frames
[1].ImageIterator
class for efficient iteration through frames of animated images, optionally with frame caching.ImageIterator
class.For the CLI:
For the TUI:
ImageIterator
class.Footnotes
I reported the issue to the Pillow developers and opened a PR to fix the issues, though my changes would’ve broken some key things. 😞 So, one of the core developers implemented the improvements in a better way that wouldn’t break things. See https://github.com/python-pillow/Pillow/pull/6077 The changes have been merged and will be released with the next feature release 9.1.0 by April 1. When this version is out, I plan to revert the changes to
TermImage.n_frames
because the performance will be smooth enough.When Pillow 9.1.0 is out, I’ll be extending the caching criteria of
ImageIterator
to cache frames based on a given maximum number of frames, which is a better metric than file size.If you don’t mind, please update your installation from the main branch to test the changes.
I’ll gladly appreciate your feedback. Thanks 😃
Good. I’ll approach it the problem this way then.
Thanks so much.
Absolutely. Simple image files like PNG and JPG do not seem to cause any issue (as of now). For the time being, both GIF and postscript should be taken into account regarding max size.
Exactly the same thoughts here.
As for the “forced load”, I already implemented something similar for images above the “max pixels” so I’ll simply extend that.
As to the idea of gradually loading/displaying images, I guess that’s a nice approach, provided the loading process runs on a separate thread and the TUI stuff on another one. The process should be as smooth as possible, even if this implies not previewing some files at all. Maybe a message in the previewing panel warning the user about the file not being displayed and allowing him/her to forcefully load the image by means of some keyboard shortcut. Just thinking aloud.
Thanks
This is the big GIF file I was talking about:
https://mega.nz/file/x4p1ma4B#PpYiUtwWFl5kUKOkyGavV9lXLuD4qsHB9uLRs1rfZvM
I really appreciate your quick reply… that’s one golden principle.
I’m currently looking into it. 😃 Thank you very much.
Whichever way you find to tackle this issue will be fine.
This is a principle I follow whenever I write some new feature for my programs: no matter how cool I think it is, if it might noticeable reduce the program’s performance (or somehow produce a negative impact on the user), just disable it by default and allow the user to enable it via some command line switch or option in the config file. Users get quickly disappointed and give up as soon as they experience some issue with your program, and that’s something that we, as developers, need to prevent at all costs.
Please let me know if you find some way to improve this loading issue.
As for the code responsible…
term_img.tui.main.scan_dir()
scans a directory and createsImage
widgets for every image in it and a generator for subdirectories.term_img.cli.check_dir()
handle the scanning of directories to determine if they contain images or subdirectories that to.Wow! That’s a lot of time… It took way less than that for the entire DCIM directory (with thousands of images) on my (not recent) android device.
It seems to be an edge case like you’ve described. I’ll try getting a sample image of that kind to test.
EDIT: I suppose the file will also be really large… among the planned features is an option to specify a maximum file size which I plan to set to a reasonable default.
Well, after some tests I conclude that the source of the issue is actually related to some file types. Previewing postscripts files could be a bit slow. In my case, the main obstacle was a really big GIF file (2187 frames!). I guess you could provide a command line switch to disable automatic previews for GIF files, or, even better, find a programmatic way to not automatically reproduce GIF files with more than, say, 100 frames.
Thanks for your quick answer! Here’s a more precise description of the issue:
I run:
./term-img ~/Downloads
, and this is the output:It gets stuck here for exactly 46 secs, taking 50% of my CPU.
Once it finally loads the TUI, it gets stuck there (hovering on
Downloads/
) for another 40 secs or so (taking 50% of the CPU again) before I can actually do something.As to recommendations to make the loading thing faster, I didn’t actually look at the code, so I cannot recommend anything useful. Maybe I could take a look at it to see what could be causing this issue.
EDIT: It seems to happen mostly with my
Downloads
folder. With other directories it works much faster (almost as expected, I’d say). My bet is that whatever image previewer you’re using is having trouble dealing with some specific file type.