wekan: Non-ASCII attachment filename will crash when downloading

On a source build on Ubuntu, when I upload a file with Chinese characters, it will be uploaded successfully but the server will crash when someone tries to download it. The error log is:

W20191018-07:03:45.395(0)? (STDERR) /home/romulus/.meteor/packages/promise/.0.11.2.1rxqxdq.d4m4g++os+web.browser+web.browser.legacy+web.cordova/npm/node_modules/meteor-promise/promise_server.js:218
W20191018-07:03:45.396(0)? (STDERR)       throw error;
W20191018-07:03:45.396(0)? (STDERR)       ^
W20191018-07:03:45.396(0)? (STDERR)
W20191018-07:03:45.397(0)? (STDERR) TypeError: The header content contains invalid characters
W20191018-07:03:45.397(0)? (STDERR)     at validateHeader (_http_outgoing.js:494:11)
W20191018-07:03:45.397(0)? (STDERR)     at ServerResponse.setHeader (_http_outgoing.js:498:3)
W20191018-07:03:45.398(0)? (STDERR)     at ServerResponse.res.setHeader (/home/romulus/.meteor/packages/simple_json-routes/.2.1.0.3oj2dz.6hqih++os+web.browser+web.cordova/npm/node_modules/connect/lib/patch.js:134:22)
W20191018-07:03:45.398(0)? (STDERR)     at packages/cfs_http-methods.js:622:17
W20191018-07:03:45.399(0)? (STDERR)     at Function._.each._.forEach (packages/underscore.js:147:22)
W20191018-07:03:45.399(0)? (STDERR)     at packages/cfs_http-methods.js:618:11
=> Exited with code: 1

When I check the uploaded file on the server, it’s filename is original Chinese, not encoded: attachments-gAjLYeSrtAneFBdzt-提議者電子郵件(第一波+第二波).xlsx

I guess the problem is the server does not encode the filename when saving it. It is always a bad idea. I want to make the server encode filename when saving it, and of course, decode it when trying to download. For the old files, I may create a process to try both decoded and non-decoded filename. Will this make any problem, or anyone has a better idea? Thank you.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 28 (21 by maintainers)

Most upvoted comments

@urakagi @neutrolino @exejay @xavier-GitHub76 @gpalyu @tovkotov @devilmengcry @dvdent

Please upgrade to Wekan v5.27 where I fixed uploading and downloading Non-ASCII attachment filename. Please test does it work now. Thanks!

@exejay

Attachment filenames not URL-decoded moved to https://github.com/wekan/wekan/issues/3774

About files 0 bytes long, of course not existing 0 byte file can not be displayed. See https://github.com/wekan/wekan/issues/3583#issuecomment-829357679

@xet7 @urakagi I just finished upgrading and testing, and it seems to work, with a caveat: The filename encoding seems to be working. The filenames display correctly and the URLs are also working. I have tried old files and newly uploaded files.

But issue is that the filenames don’t get decoded on download, so I get an encoded filename string for the downloaded file. This may be partially a browser issue too, as the filename changes depending on which I use. Same filename:

  1. orig: test测试
  2. Firefox: testæµ è¯
  3. Edge: test%E6%B5%8B%E8%AF%95

So it is per se usable, but not ideal and not the behaviour from the older versions. Another thing I noted is that if the uploaded file is 0 bytes long, the “file type preview” seems to break and you can’t download the file.

@urakagi

The benefit of you having now added your In Progress pull request is that I was able to test your In Progress code and add some comments to it. Thanks!

@urakagi

  1. Build a migrating tool in wekan, check and migrate old attachments on each startup.

For example at #2635, old attachments disappear, that should be fixed so that all attachments are kept.

With each file, it should be checked, something like this:

  1. Does destination exist.
  2. Check is there enough space for migrating all the files.
  3. Migrate one file.
  4. Check did file migration work correctly.
  5. Update path of migrated file in database.
  6. Remove original file.
  7. Go back to step 1 for next file.

@urakagi

When I upgraded Wekan from Meteor 1.6.x to 1.8.1, this fix does not work anymore https://github.com/wekan/wekan/issues/784

It’s here at build script: https://github.com/wekan/wekan/blob/master/rebuild-wekan.sh#L150 https://github.com/wekan/wekan/blob/master/Dockerfile#L253 https://github.com/wekan/wekan/blob/master/snapcraft.yaml#L198

I think it does not make changes to correct directories and files. You could try to find where to make the changes.