storage: Storage v3 - Resumable uploads error
Bug report
- [ X] I confirm this is a bug with Supabase, not with my own application.
- [ X] I confirm I have searched the Docs, GitHub Discussions, and Discord.
Describe the bug
I was reading the blog about the new storage features. I found interesting that you implemented the Tus protocol. I was trying to build a simple demo with Uppy as you show in the blog but I get a 404 error.
https://supabase.com/blog/storage-v3-resumable-uploads#deep-dive-into-resumable-uploads
To Reproduce
I created a simple Codesandbox with Svelte and I added the code inside the layout file:
https://codesandbox.io/p/sandbox/muddy-cache-xr8ct2?welcome=true
I didn’t add the anon key and my project ref.
I get this error:
{"message":"Route POST:/upload/resumable not found","error":"Not Found","statusCode":404}
I guess the URL is not correct? But I could’t figure out how to make this work.
Expected behavior
I expect to be able to upload a JPG inside my bucket folder.
Additional context
I also tried to create an upload URL with the Js Client using store.createSignedUploadUrl. It doesn’t give the HTTP error anymore but I get an error message inside the Uppy component saying the Location header is missing.
The bucket is public and I created this 2 policies:
create policy "Public Access Bucket Images Get"
on storage.objects for select
using ( bucket_id = 'users' );
create policy "Public Access Bucket Images Post"
on storage.objects for insert
with check ( bucket_id = 'users' );
I don’t know if they are correct. I just want to allow to upload files for the authenticated users. I tried to make the buket open to test if everything works well.
Thank you.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 17 (3 by maintainers)
Didn’t work for me even with the modifications in kong.yml,
TUS_URL_PATHin .env, and the downgrade. The environment variable is to be included in the .env file on the docker folder, right? My browser keeps hitting localhost:8000/storage/v1/upload/resumable.@fenos Really appreciate the response and help! Got it working. 🙌
I added the
TUS_URL_PATHand didn’t have to include the/storage/v1in the kong config. So I ended up with the following.When running locall, I added this to the
kong.ymlunder the storage service:On a server with nginx as reverse proxy, I tried this, and it worked. (I couldn’t figure out how to make nginx / kong forward the protocol correctly without this).
I presume that if I’m running the servers behind a load balancer, then I would add the “host” part and set it to the domain pointing to the load balancer.
If you know any resources on what the best configuration with nginx is, then please do share. I’m using Docker Swarm to orchestrate the services.
Do you know why there’s a hard limit of 6mb for the chunk size? The docs just say this is currently a limitation with the storage-api, but adds no further explanation. Would be cool to understand more. 😄 Thanks again!
This is more than a little infuriating…
So it was actually working (a file is definitely uploaded and visible in the studio!) without the
Forwarded: ...and then once it stops working, namely it goes from correctly usinghttpsto suddenly usinghttpfor thesupabase/konggenerated links. It doesn’t seem to matter what I put in thekongconfig, it won’t come back. SettingNODE_ENV === 'production'doesn’t seem to help either.I have tried adding this to
kongin my kubernetes helm chart (running behind aingress-nginxcontroller):But this just has no effect. I checked and the
kongconfig file is definitely updated in the container… I am still getting"x_forwarded_proto":"http"in the logs of thestoragecontainer.I also note that the
kongdocs mention this:For the docs of the
request-transformerplugin… So I’m completely lost.EDIT: Unbelievable! Somehow I missed that there was a cached value in
localStoragewith the wrong URL, and it wasn’t getting updated… So it appears that it does work… Sigh.I’m having the same issue with uploads to storage not returning the correct proxy url:
I tried the fix @Obeyed proposed here but it still doesn’t work at all. Why would we even have to add the “Forwarded” header when there are already proper “X-Forwarded-*” headers sent to storage api?
I also tried fully inserting the Forwarded header with a dynamic value for host, but for some reason kong does not seem to insert any value here:
EDIT
I checked the used @tus/server library and noticed that is seems they themselves are not properly handling all the X-Forwarded headers properly yet, but I think it should be possible to fix that on the storage implementation side here: https://github.com/supabase/storage/blob/830ea71d90c079f487f41d878aafed935f87d25e/src/http/routes/tus/lifecycle.ts#L54-L68
That would also make this here unnecessary: https://github.com/supabase/cli/blob/b380dfbc53d494a7b2e0c0f55b9503c6fd7d9796/internal/start/templates/kong.yml#L98-L102
EDIT 2
I at least found out what the issue was with Forwarded header: Because there is an env substitution happening to kong.yaml, the
$(headers.host)was being replaced at that time instead of inside kong. I escaped it like this and now it’s working:I still think it would be better to change the code to properly handle the proxy headers though.
This. i looked everywhere, tinkered with traefik for six hours, then i remembered reading this. it works ❤️
@Obeyed Glad you got it working!
In terms of configurations, as far as the x-forwarded-proto and x-forwarded-host are sent over to the storage-api the corresponding host and protocol will be used.
Sure, the reason behind the 6MB limit is that S3 imposes the part size upload to be a minimum of 5MB and a maximum of 5GB we chose to go with 6MB since for large files seems the right middle ground
https://docs.aws.amazon.com/AmazonS3/latest/userguide/qfacts.html
with a 6MB chunk and 1000 parts limits we can handle up to 60GB file uploads
Yes, that’s correct! If you reach out to us we will enable it for you
We are going to release it globally very soon