PodFetch: Very large podcasts has duplicate downlods

Describe the bug

I have a very large podcast I am trying to archive, with thousands of episodes. Unfortunately, it is occasionally duplicating some of the episodes and I am not sure why, or if there is a way to clean it up. If it will auto clean up after it’s done downloading the thousands of episodes, let me know and this can be closed. Screenshot_2024-11-19_15-58-08

Reproduction

added podcast that is very large and set to download 999999 episodes

System Info

Using docker on unraid. I tried to run the command listed, but it errors out because apparently the command is not in /bin nor does this container apparently have a bash shell.

Used Package Manager

docker

Validations

  • Follow our Code of Conduct
  • Read the Contributing Guide.
  • Check that there isn’t already an issue that reports the same bug to avoid creating a duplicate.
  • Check that this is a concrete bug. For Q&A, please open a GitHub Discussion instead.
  • The provided reproduction is a minimal reproducible of the bug.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 22 (3 by maintainers)

Most upvoted comments

I pushed a new version. It contains a bunch of improvements and it doesn’t bubble the errors if the tagging goes wrong. I have no idea. It definetely worked flawlessly before and it was tested back then. I also went back a few versions with that particular library and it still errors. I’ll need to investigate but you can again download things without an issue

same here, even with newest version, e.g. with the podcast https://podcast.darknetdiaries.com Waitin a few hours or refreshing creates a new directory (e.g. 2024-12-03_152-_ Stacc Attack-1) with the same podcast again, filling up the disk space over a few days

2024-12-04T21:28:01 ℹ️ - Downloading podcast episode: 152: Stacc Attack 2024-12-04T21:28:10 ❌ - File format not supported: Id3v2 thread ‘<unnamed>’ panicked at src/controllers/podcast_episode_controller.rs:188:14: called Result::unwrap() on an Err value: Conflict(“File format not supported”) 2024-12-04T21:28:26 ℹ️ - Refreshing podcast: Darknet Diaries 2024-12-04T21:28:26 ℹ️ - Downloading podcast episode: 152: Stacc Attack 2024-12-04T21:28:35 ❌ - File format not supported: Id3v2 2024-12-04T21:28:35 ❌ - Error downloading podcast episode: The following error occurred: File format not supported 2024-12-04T21:28:35 ℹ️ - Succesfully refreshed podcast. 2024-12-04T21:28:35 ℹ️ - Downloading podcast episode: 152: Stacc Attack 2024-12-04T21:28:43 ❌ - File format not supported: Id3v2 2024-12-04T21:28:43 ❌ - Error downloading podcast episode: The following error occurred: File format not supported

I just made a new podfetch instance and added only one podcastfeed. After adding the feed it starts downloading and every file downloaded throws an id3v2 error. The .mp3 and .jpg file is downloaded. After one hour it starts again downloading. The container runs for about three hours and there are already treetimes downloaded podcast.

podfetch  | 2024-12-04T10:16:06 ℹ️ - Downloading podcast episode: Deus Ex! (SF 2)
podfetch  | 2024-12-04T10:16:10 ❌ - File format not supported: Id3v2
podfetch  | 2024-12-04T10:16:10 ❌ - Error downloading podcast episode: The following error occurred: File format not supported
podfetch  | 2024-12-04T10:16:10 ℹ️ - Downloading podcast episode: Revolution! (SF 1 Classic)
podfetch  | 2024-12-04T10:16:14 ❌ - File format not supported: Id3v2
podfetch  | 2024-12-04T10:16:14 ❌ - Error downloading podcast episode: The following error occurred: File format not supported
podfetch  | 2024-12-04T10:16:14 ℹ️ - Downloading podcast episode: Baphomets Fluch (SF 1 REMAKE)
podfetch  | 2024-12-04T10:16:23 ❌ - File format not supported: Id3v2
podfetch  | 2024-12-04T10:16:23 ❌ - Error downloading podcast episode: The following error occurred: File format not supported
podfetch  | 2024-12-04T10:16:23 ℹ️ - Polling for new episodes successful
podfetch  | 2024-12-04T10:16:24 ℹ️ - Polling for new episodes
podfetch  | 2024-12-04T10:16:25 ℹ️ - Downloading podcast episode: Fable (SSF 79)
podfetch  | 2024-12-04T10:16:39 ❌ - File format not supported: Id3v2
podfetch  | 2024-12-04T10:16:40 ❌ - Error downloading podcast episode: The following error occurred: File format not supported
podfetch  | 2024-12-04T10:16:40 ℹ️ - Downloading podcast episode: Ascendancy (SF 149)
podfetch  | 2024-12-04T10:16:53 ❌ - File format not supported: Id3v2
podfetch  | 2024-12-04T10:16:53 ❌ - Error downloading podcast episode: The following error occurred: File format not supported
podfetch  | 2024-12-04T10:16:53 ℹ️ - Downloading podcast episode: Deus Ex (SF 148)
podfetch  | 2024-12-04T10:17:07 ❌ - File format not supported: Id3v2
podfetch  | 2024-12-04T10:17:08 ❌ - Error downloading podcast episode: The following error occurred: File format not supported

Downloades on 2024-12-04T10:16:14 was the first episode, and again on 2024-12-04T10:16:25 it starts getting the latest

drwxrwxrwx 2 10000 lxc_shares     4 Dec  4 08:54 'Zelda-_ A Link to the Past (SSF 32)'
drwxrwxrwx 2 10000 lxc_shares     4 Dec  4 09:49 'Zelda-_ A Link to the Past (SSF 32)-0'
drwxrwxrwx 2 10000 lxc_shares     4 Dec  4 10:51 'Zelda-_ A Link to the Past (SSF 32)-1'

Edit: I just realized, this feed never had any issues in earlier podfetch versions.