rclone: WebDAV: Redirects for metadata queries are interpreted as "file not found"

To change this we’d have to look at why it is like that.

Can you look back through the history to find the relevant issues?

I remember making a go bug report about this too so it may not be easy to fix.

https://github.com/golang/go/issues/41377

So the go stdlib changes the PROPGET into a GET on 302 redirects which will break stuff.

If we can fix it and not break other WebDAV backend then I’m all for that.

Looks like I’m running into the same issue with PhotoPrism WebDAV server running behind a Traefik reverse proxy. The other WebDAV clients I’ve tried work well with this setup, but rclone goes belly-up with

read metadata failed: 307 Temporary Redirect

To simplify the example, I tried rclone copy next — and this fails during PROPFIND, since it does not follow the redirect, i.e. in short:

> PROPFIND
< 307 with Location header
Attempt 1/3 failed with 1 errors and: read metadata failed: 307 Temporary Redirect

I’m confused — it seems rclone mount PROPFIND calls follow 307 redirects, but rclone copy PROPFIND calls error out when getting 307. If I read the code correctly, any redirects are blocked by NoRedirect independent of status code for PROPFIND as-is. Is this on purpose?

Going the ugly route (due to my limited go knowledge to do this right…) and replacing:

        clientCopy.CheckRedirect = func(req *http.Request, via []*http.Request) error {
                if req.Response.StatusCode != 307 {
                        return http.ErrUseLastResponse
                } else {
                        return nil
                }
        }

fixes the PROPFIND calls when using rclone copy, so these now follow 307 redirects as expected. But PUT still fails to follow redirects, and I do not understand why.

So I’m unsure how a server which has to redirect (since it holds neither data nor metadata) should respond for rclone to follow along the redirect.

Can you look back through the history to find the relevant issues?

I will, thanks for pointing to the Go bugreport! This was an interesting read indeed. Similar to your conclusion, I am not sure I can derive an argument which is compelling enough to change the server-side implementation, especially since it is used for many petabytes of data transfers for decades (in some cases, with multi-tiered redirections), and none of the other existing clients (davix, gfal, curl etc.) have similar behaviour, so they may be afraid to break other clients by changing the server-side behaviour.

I think I will proceed in two stages:

• Ask XRootD upstream about their opinion. I think the quote from RFC2616 (even though not carried over to newer RFCs):

The status codes 303 and 307 have been added for servers that wish to make unambiguously clear which kind of reaction is expected of the client.

may be compelling to at least consider it.

• If that is denied, I’ll skim through the history.

The latter may also allow me to find out why your proposal to:

I think you can do that with CheckRedirect in http.Client and that would be my workaround.

was not implemented, maybe there was a reason not go this way.