gotenberg: Images Fail to Load on /convert/url Endpoint Due to Authentication Headers Rejected by CDNs

I noticed that when I send a request to the /convert/url endpoint with the following data:

{
  { “extraHeaders”: { “Authentication”: “Basic base64data” }
}

Some images, selected randomly each time, do not load. I believe this is because Gotenberg sends authentication headers for each frontend request, and some CDN servers reject these headers, especially the Authentication header. This seems to be the reason why some images are not loading. I ran into a similar problem using the Puppeteer npm package, but Puppeteer offers an async method page.authenticate that handles authentication by acting like a native browser passing basic authentication. The authentication header is not sent along with the FE request to the third-party APIs (loading images for display). Do you have any suggestions or solutions to help with this issue?

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 19

Most upvoted comments

@gulien,

I tested with the URL https://authenticationtest.com/HTTPAuth, and it works as expected. I also re-tested on https://headers.4tools.net, and the Authorization headers appeared.

We also applied the snapshot image to our staging environment, and it looks like the issue with images has been resolved.

We’ll continue testing for another hour, and if everything looks good, we will let you know. If you’ll be able to create a new release, that would be great.

Thank you.

I just want to clarify why I initially said that it worked before: This was because my first request used extraHeaders, which caused the assets to be cached. So, when I tested again using basicAuthenticationUsername and basicAuthenticationPassword, it seemed to work because all the assets were already cached.

@gulien,

I tested with the URL https://authenticationtest.com/HTTPAuth, and it works as expected. I also re-tested on https://headers.4tools.net, and the Authorization headers appeared.

We also applied the snapshot image to our staging environment, and it looks like the issue with images has been resolved.

We’ll continue testing for another hour, and if everything looks good, we will let you know. If you’ll be able to create a new release, that would be great.

Thank you.

@gulien Thank you very much for your work, we appreciate it. It would be great if you would release this version.

If its works for you, I can release a version today 😉

@forg002-ctrl Ok so I’ve found a working solution lol:

curl --request POST \
--url http://localhost:3000/forms/chromium/convert/url \
--header 'content-type: multipart/form-data' \
--form url=https://authenticationtest.com/HTTPAuth/ \
--form-string 'extraHttpHeaders={"Authorization":"Basic dXNlcjpwYXNz;scope=https?:\\/\\/([a-zA-Z0-9-]+\\.)*authenticationtest\\.com\\/.*"}'

Notice --form vs --form-string 😄

An alternative solution could be to scope the extra HTTP headers to specific domains.

Yep, it only works for the main page for now 🤔

I can create a new release once your testing is done!