recyclarr: HTTP error while communicating with Radarr: Flurl.Http.FlurlHttpException

Issue

When running the program in docker, I get HTTP errors differing based on endpoints. both 400 and 500.

It seems to me that the 400 happens when trying to put scores into the quality profile, from scratch or trying to update.
My hunch for the 500 error seems to be when creating/adding multiple CFs from scratch (ie. adding 10+ new CFs) the server can’t handle all the api calls and times out

Logs

HTTP error 500

[INF] Processing server http://hostname:7878
[INF] Processing Quality Definition: Movie
[INF] Number of updated qualities: 0

HTTP error while communicating with Radarr: Flurl.Http.FlurlHttpException: Call failed with status code 500 (Internal Server Error): POST http://hostname:7878/api/v3/customformat?apikey=SNIP
   at Flurl.Http.FlurlRequest.HandleExceptionAsync(FlurlCall call, Exception ex, CancellationToken token)
   at Flurl.Http.FlurlRequest.SendAsync(HttpMethod verb, HttpContent content, CancellationToken cancellationToken, HttpCompletionOption completionOption)
   at Flurl.Http.FlurlRequest.SendAsync(HttpMethod verb, HttpContent content, CancellationToken cancellationToken, HttpCompletionOption completionOption)
   at Flurl.Http.ResponseExtensions.ReceiveJson[T](Task`1 response)
   at TrashLib.Radarr.CustomFormat.Api.CustomFormatService.CreateCustomFormat(ProcessedCustomFormatData cf) in D:\a\recyclarr\recyclarr\src\TrashLib\Radarr\CustomFormat\Api\CustomFormatService.cs:line 27
   at TrashLib.Radarr.CustomFormat.Processors.PersistenceSteps.CustomFormatApiPersistenceStep.Process(ICustomFormatService api, CustomFormatTransactionData transactions) in D:\a\recyclarr\recyclarr\src\TrashLib\Radarr\CustomFormat\Processors\PersistenceSteps\CustomFormatApiPersistenceStep.cs:line 11
   at TrashLib.Radarr.CustomFormat.Processors.PersistenceProcessor.PersistCustomFormats(IReadOnlyCollection`1 guideCfs, IEnumerable`1 deletedCfsInCache, IDictionary`2 profileScores) in D:\a\recyclarr\recyclarr\src\TrashLib\Radarr\CustomFormat\Processors\PersistenceProcessor.cs:line 72
   at TrashLib.Radarr.CustomFormat.CustomFormatUpdater.Process(Boolean isPreview, RadarrConfiguration config) in D:\a\recyclarr\recyclarr\src\TrashLib\Radarr\CustomFormat\CustomFormatUpdater.cs:line 47
   at Recyclarr.Command.Services.RadarrService.Process(IRadarrCommand cmd) in D:\a\recyclarr\recyclarr\src\Recyclarr\Command\Services\RadarrService.cs:line 52
   at Recyclarr.Command.Services.ServiceBase`1.Execute(T cmd) in D:\a\recyclarr\recyclarr\src\Recyclarr\Command\Services\ServiceBase.cs:line 18

HTTP error 400

[INF] Processing server http://hostname:7878
[INF] Processing Quality Definition: Movie
[INF] Number of updated qualities: 0

HTTP error while communicating with Radarr: Flurl.Http.FlurlHttpException: Call failed with status code 400 (Bad Request): PUT http://hostname:7878/api/v3/qualityprofile/1?apikey=SNIP
at Flurl.Http.FlurlRequest.HandleExceptionAsync(FlurlCall call, Exception ex, CancellationToken token)
at Flurl.Http.FlurlRequest.SendAsync(HttpMethod verb, HttpContent content, CancellationToken cancellationToken, HttpCompletionOption completionOption)
at Flurl.Http.FlurlRequest.SendAsync(HttpMethod verb, HttpContent content, CancellationToken cancellationToken, HttpCompletionOption completionOption)
at Flurl.Http.ResponseExtensions.ReceiveJson[T](Task`1 response)
at TrashLib.Radarr.CustomFormat.Api.QualityProfileService.UpdateQualityProfile(JObject profileJson, Int32 id) in D:\a\recyclarr\recyclarr\src\TrashLib\Radarr\CustomFormat\Api\QualityProfileService.cs:line 26
at TrashLib.Radarr.CustomFormat.Processors.PersistenceSteps.QualityProfileApiPersistenceStep.Process(IQualityProfileService api, IDictionary`2 cfScores) in D:\a\recyclarr\recyclarr\src\TrashLib\Radarr\CustomFormat\Processors\PersistenceSteps\QualityProfileApiPersistenceStep.cs:line 74
at TrashLib.Radarr.CustomFormat.Processors.PersistenceProcessor.PersistCustomFormats(IReadOnlyCollection`1 guideCfs, IEnumerable`1 deletedCfsInCache, IDictionary`2 profileScores) in D:\a\recyclarr\recyclarr\src\TrashLib\Radarr\CustomFormat\Processors\PersistenceProcessor.cs:line 76
at TrashLib.Radarr.CustomFormat.CustomFormatUpdater.Process(Boolean isPreview, RadarrConfiguration config) in D:\a\recyclarr\recyclarr\src\TrashLib\Radarr\CustomFormat\CustomFormatUpdater.cs:line 47
at Recyclarr.Command.Services.RadarrService.Process(IRadarrCommand cmd) in D:\a\recyclarr\recyclarr\src\Recyclarr\Command\Services\RadarrService.cs:line 52
at Recyclarr.Command.Services.ServiceBase`1.Execute(T cmd) in D:\a\recyclarr\recyclarr\src\Recyclarr\Command\Services\ServiceBase.cs:line 18

Docker Compose

version: '3'

services:
  recyclarr:
    image: ghcr.io/recyclarr/recyclarr:latest
    container_name: recyclarr
    hostname: recyclarr
    restart: unless-stopped
    init: true
    environment:
      - PUID=1000
      - PGID=1000
    volumes:
      - ./config:/config
      - /etc/localtime:/etc/localtime:ro

networks:
  default:
    name: docker
    external: true

Recyclarr.yml

radarr:
   # Set the URL/API Key to your actual instance
  - base_url: http://hostname:7878
    api_key: API_KEY
    quality_definition:
      type: movie
    delete_old_custom_formats: true
    custom_formats:
      - trash_ids:
        # Unwanted
        - ed38b889b31be83fda192888e2286d83 # BR-DISK
        - 90cedc1fea7ea5d11298bebd3d1d3223 # EVO (no WEBDL)
        - 90a6f9a284dff5103f6346090e6280c8 # LQ
        - dc98083864ea246d05a42df0d05f81cc # x265 (720/1080p)
        - b8cd450cbfa689c0259a01d9e29ba3d6 # 3D
        - ae9b7c9ebde1f3bd336a8cbd1ec4c5e5 # No-RlsGroup
        - 7357cf5161efbf8c4d5d0c30b4815ee2 # Obfuscated
        - 923b6abef9b17f937fab56cfcf89e1f1 # DV (WEBDL)
        # Misc
        - e7718d7a3ce595f289bfee26adc178f5 # Repack/Proper
        # HQ Source Groups
        - 26fa26253af4001701fedb56cec376dc # HQ-WEBDL
        quality_profiles:
          - name: 1080p
      - trash_ids: 
        - 1c7d7b04b15cc53ea61204bebbcc1ee2 # HQ
        quality_profiles:
          - name: 1080p
            score: 1000
      # Audio
      - trash_ids:
        - 77ff61788dfe1097194fd8743d7b4524 # 5.1 Surround
        quality_profiles:
          - name: 1080p
            score: 15

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 21 (10 by maintainers)

Most upvoted comments

Thanks to @anthr76’s help, I was able to figure out his issue. Sonarr v4, which is not publicly available, breaks backward compatibility with Sonarr v3. As such, Recyclarr will not work against that early release. To make things a little more fool-proof, I will make a change that prints an error message if v4 or greater is detected.

rcdailey on Jul 14, 2022

@anthr76 Can you DM me on Discord? I’d like to ask you about your specific use case in more detail (those questions won’t be appropriate here). My handle in discord is voidpointer#3380.

EDIT: Also please share your redacted YAML. I will try to reproduce your scenario.

rcdailey on Jul 9, 2022

I think at this point there’s nothing to be done on my side. As far as I’m able to tell, Recyclarr is respecting the API contract with Radarr & Sonarr. It looks like, for whatever reason, when an update/creation happens for a single CF, that operation doesn’t fully complete before the next one. It might be due to the delay between Radarr and its database. I do not see these race conditions when I run Recyclarr in a docker container running directly on a machine with SSD for storage (which will likely have relatively much better timings than your setup, even though I wouldn’t call .12ms ping bad by any stretch).

If I were you, I would approach the Radarr/Sonarr developers about this. There are a few options that can be taken on their side to resolve this issue, assuming I’m not wrong on the root cause:

Investigate why the database is locked between CF updates/creations and make the web api wait to return back to the client between each call.
Implement a new version of the /api/v3/customformat endpoint (for POST and PUT) that takes an array of custom format objects. This would allow clients to send all updates in 1 call, allowing Radarr more control over how the database is utilized and will likely be more performant too.

The second option requires a lot of work and I doubt they will do it. But the first option might be reasonable to ask for.

The real issue here is that Radarr is sending an HTTP response back to clients before the database is ready/done, best I can tell. That’s not something I can fix, and any workarounds I could implement (e.g. artificial delays between API calls) is not something I really want to do.

rcdailey on Jul 8, 2022

This is a Kubernetes cluster. All in a 10G network.

Sonarrs are using Rook-Ceph for block storage backed by nvme’s with sqlite Radarrs are using an external postgres database that’s quite unbusy with nvme

Im seeing a .12 ms ping between the nodes.

anthr76 on Jul 8, 2022

Do you actually have base_url: http://hostname:7878 written in your recyclarr.yml file or did you just replace/redact it for the issue

For future reference, the logging system in Recyclarr automatically obfuscates sensitive information, namely the API key and hostname. When folks post logs on Github or elsewhere I don’t want to have to worry about them exposing that information by accident.

rcdailey on Jun 16, 2022

Sounds like this is a problem with your Radarr setup and not with Recyclarr…

Just to be sure: Do you actually have base_url: http://hostname:7878 written in your recyclarr.yml file or did you just replace/redact it for the issue? Because this base URL probably doesn’t exist.

It was redacted for the issue. As far as I can tell, you are technically correct that it was an issue with the radar set up, but only because recyclarr had that initial 500 error. Once I got rid of the problem CFs, it synced properly. However, those CFs were only added by recyclarr making that the issue. I saw in the discord someone else had the 500 error with sonarr, so it has happened to more people than just me as well.

tiemonl on Jun 8, 2022