seaweedfs: Webdav server stops responding after saving some files

WebDav server stop responding after file with some extension was uploaded.

My configuration: I have 3 nodes in cluster. Every node runs 2 docker-container:

  • official cockroachdb container for filer metadata (v.20.1.0)
  • my custom seaweedfs container with:
  • weed master -ip=b-img02 -mdir=/opt/seaweedfs/data/master -peers=b-img00:9333,b-img01:9333,b-img02:9333 -defaultReplication=200
  • weed volume -ip=b-img02 -mserver=b-img00:9333,b-img01:9333,b-img02:9333 -dataCenter=b-img02 -rack=rack -dir=/opt/seaweedfs/data/
  • weed filer -collection=filer -defaultReplicaPlacement=200
  • weed webdav Weed version: 30GB 1.79

Screenshot_20200603_140300

All works fine until I trying to upload file in filer.

In examples I create test file, upload it into filer and download from webdav.

Example#1 - all works fine

echo ok > data.html

curl -F file=@data.html http://b-img00:8888/
>> {"name":"data.html","size":3,"fid":"7,08ba03a642ff84","url":"http://b-img02:8080/7,08ba03a642ff84"}

curl -vvv http://b-img00:7333/data.html
*   Trying x.x.x.x...
* TCP_NODELAY set
* Connected to b-img00 (x.x.x.x) port 7333 (#0)
> GET /data.html HTTP/1.1
> Host: b-img00:7333
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Content-Length: 3
< Content-Type: text/html; charset=utf-8
< Etag: "161503d94245de003"
< Last-Modified: Wed, 03 Jun 2020 11:04:35 GMT
< Date: Wed, 03 Jun 2020 11:04:57 GMT
<
ok
* Connection #0 to host b-img00 left intact

curl -X DELETE -vvv http://b-img00:7333/data.html
*   Trying x.x.x.x...
* TCP_NODELAY set
* Connected to b-img00 (x.x.x.x) port 7333 (#0)
> DELETE /data.html HTTP/1.1
> Host: b-img00:7333
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 204 No Content
< Date: Wed, 03 Jun 2020 11:08:15 GMT
<
* Connection #0 to host b-img00 left intact

Ok. Example#2 - I change file extention from .html to .txt After file uploaded webdav stops to serv folder with this file (it can be downloaded via filer successfully). Another folders in webdav works fine.

echo ok > data.txt

curl -F file=@data.txt http://b-img00:8888/

curl -vvv http://b-img00:7333/data.txt
*   Trying xx.xx.xx.xx...
* TCP_NODELAY set
* Connected to b-img00 (xx.xx.xx.xx) port 7333 (#0)
> GET /data.txt HTTP/1.1
> Host: b-img00:7333
> User-Agent: curl/7.58.0
> Accept: */*
>
(( I await some time - but no response received ))
^C

curl -vvv http://b-img00:8888/data.txt
*   Trying xx.xx.xx.xx...
* TCP_NODELAY set
* Connected to b-img00 (xx.xx.xx.xx) port 8888 (#0)
> GET /data.txt HTTP/1.1
> Host: b-img00:8888
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Content-Disposition: inline; filename="data.txt"
< Content-Length: 3
< Content-Type: text/plain
< Etag: "908e596f"
< Last-Modified: Wed, 03 Jun 2020 11:09:35 GMT
< Date: Wed, 03 Jun 2020 11:10:06 GMT
<
ok
* Connection #0 to host b-img00 left intact

curl -X DELETE http://b-img00:8888/data.txt
(ok)

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 27

Commits related to this issue

Most upvoted comments

@chrislusf

My log format same as your - look at log json field.

For data.html I see logs as you:

I0605 14:44:19    15 webdav_server.go:185] WebDavFileSystem.OpenFile /data.html 0,
I0605 14:44:19    15 webdav_server.go:564] WebDavFile.Stat /data.html,
I0605 14:44:19    15 webdav_server.go:543] WebDavFile.Seek /data.html 0 2,
I0605 14:44:19    15 webdav_server.go:543] WebDavFile.Seek /data.html 0 0,
I0605 14:44:19    15 webdav_server.go:462] WebDavFileSystem.Read /data.html,
I0605 14:44:19    15 webdav_server.go:487] WebDavFileSystem.Read /data.html: [0,3),
I0605 14:44:19    15 webdav_server.go:450] WebDavFileSystem.Close /data.html

I watched a source code (I dont know GO language so sorry for my mistakes).

Webdav server used http.serveContent method. This method execute auto-detection content-type header bases on:

  1. file extention.
  2. parsed content

After this serveContent calls Seek function double times to determine size-of-content. I think this calls we see in logs.

In successfull case I see this calls, In fail case - I dont see it. So, I think problem in step#2 of content-type detection - serveContent tried to read stream, but… it is too small?

I looked for a step#1 of content-type detection. On unix host system it used some predefined extensions, and then - used 3 mime-type files. I looked this files in my docker-container, and not found any.

I mount /etc/mime.types from host system, and error is dissapier.

So I think problem in Read function - maybe it have to raise some end-of-stream error? I dont know how it must work on GO.

Can you try to delete /etc/mime.types file and try to repeat samples?