OF-Scraper: Scraper gets stuck, causing BSOD
This is technically a follow up to a different thread, but that thread had completely derailed into a new issue That thread (has a little more info)
For some reason, around 1 month ago the scraper started getting stuck on the message “Setting date for modified directories” after completely scraping a models page, upon getting stuck the RAM usage of the program would slowly climb until my computer would bluescreen with an unknown error. It’s mildly inconsistent and seems to only be happening to me? (Judging by the lack of other issue threads mentioning this)
No matter who I scrape, there’s around a 1/4 chance of it getting stuck, which would probably be fine if I weren’t subbed to 22 accounts (only 2 I actually paid for), but I am.
The full message it gets stuck on:
[download.setDirectoriesDate:872] PID: 13240 Setting Date for modified directories download.py:872
Things I’ve tried:
- Restarting my PC
- Running as admin
- Running the .zip version
- Running the .zip version as admin
- Updating twice
- Downgrading to 3.3.3
- Scraping individual models (this is kind of an alternative fix. The issue occurs after scraping is complete so as long as I close the program before it kills my computer I’m good)
- Changing damn near every config option
- “save_location”
- “file_size_limit”
- “file_size_min”
- “dir_format”
- removed audios from the filter
- “threads”
- “partfileclean”
- “download-sems”
- “maxfile-sem”
- “downloadbars”
The only potential solution I can come up with now is reinstalling Windows, as I currently use a custom debloated installation of Windows 10 Pro, I plan to try this within the next month, and when I do I will update this thread. Otherwise I will be waiting for a response or an update. I am open to any questions or suggestions.
Video Config (Very slightly modified from last I shared it) Log (Fresh as of making this post) Batch file I use
Edit: Forgot to include the vid with --output debug
About this issue
- Original URL
- State: closed
- Created 9 months ago
- Comments: 16
I already did in the initial post
BUT Massive update, I got it working.
This was my downfall:
Simply put: it is not the same.
More complicatedly put (and possibly inaccurate, I’m guessing): The “save_location” is the path you want the directory to be in, the “dir_format” is the format of that directory. As for why the scraper had an issue with what I was trying, I’m not really sure, but if I had to guess, I had the save location as " " which made the scraper default to saving to C:/, but then in the dir format I was telling it to output to Z:/ etc, so it was looking for a separate mounted drive within my c drive…?
Anyway. It works now, everything I tried for the last like 2 months didn’t work bc of my own stupidity, and I swear at some point I moved my config and it still had issues but I guess it didn’t and I’m dumb, or I somehow failed to move a fucking file or smth, idk.