lgogdownloader: Occurrences of invalid downloads when using more than default amount of threads
When using --galaxy-install with 10 threads for downloading (--threads 10), sometimes it ends up having invalid / corrupted files while it reports as everything succeeding downloading. Re-running that update usually catches it.
Not sure if the root cause is using many threads (I have to do it since using default 4 can’t saturate my gigabit connection, I guess there is some inefficiency in how single connection is used by lgogdownloader), but I haven’t noticed this before I started using 10 threads. Could it be some race condition somewhere?
That especially happens with Stalker 2 which has really huge individual files (some are like > 30 GB). Corrupted files are random, not necessarily those huge ones.
About this issue
- Original URL
- State: open
- Created a year ago
- Comments: 22 (8 by maintainers)
A big chunk of GOG staff is currently on vacation, so the fixes on their side will come eventually.
Looking at the code I see that after reaching the max retry count it will actually write incorrect chunk data. 0ef5be3 should fix that issue Can’t really test things properly at the moment because of GOG server issues
They do answer questions or at least they did in the past. I haven’t asked any questions lately and I don’t have a direct contact with anyone. The last couple times I contacted them via GOG support and as you probably know the support can be a bit slow to respond to anything.
711120f Galaxy: Check chunk hash after downloading Retry chunk if hash check fails
21e9950 Galaxy: Handle duplicate file entries when installing Duplicate files from DLCs overwrite base game versions If there are multiple DLCs that have duplicate files then the last entry parsed wins
I was testing a few different solutions earlier. I still need to do some thinking how to tackle this. Currently I’m thinking that dlc should always overwrite the duplicate file but I don’t think there’s a good way to figure out which dlc should win. The easiest solution would be to let the last one parsed from api responses win.