sprockets: FileStore cache patch in v3.6 causes directory name collisions in Windows
The changes introduced in bc40f15dd10701034524e2cb53d373190fe0014c add a namespace directory to the digest key.
Unfortunately, on Windows the namespace folder causes conflicts due to case sensitivity. For example: lc
, Lc
, lC
all resolve to the same folder when creating the directory. When it comes time to resolve the asset in the folder the correct file cannot be found.
Steps to reproduce:
- Use sprockets v3.6 on a case-insensitive file system
- Clear the
tmp
folder for the application (a rails application in my case) - Start the application (
thin start
in my case)- sprockets starts to generate a bunch of cache files (for example
/home/vagrant/baw-server/tmp/cache/assets/sprockets/v3.0/k5/k5Tf66eynpbdWEXajNU1EtcNgCTk8C201A0Vy-aBzgg.cache
- sprockets starts to generate a bunch of cache files (for example
- Try and load a page that depends on a sprockets resource
This exception occurs:
ActionView::Template::Error (Not a directory @ rb_sysopen - /home/vagrant/baw-server/tmp/cache/assets/sprockets/v3.0/K5/K5HCuh0G2rUW6bW13cTaz7GiHa1K1gCReNVyrOUHixw.cache.5855600.7132.945301):
14: / Le HTML5 shim, for IE6-8 support of HTML elements
15: /[if lt IE 9]
16: = javascript_include_tag '//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.2/html5shiv.min.js'
17: = stylesheet_link_tag (content_for?(:stylesheet_link_tag_name) ? content_for(:stylesheet_link_tag_name) : 'application'), media: 'all'
18: %link(href="images/apple-touch-icon-144x144.png" rel="apple-touch-icon-precomposed" sizes="144x144")
19: %link(href="images/apple-touch-icon-114x114.png" rel="apple-touch-icon-precomposed" sizes="114x114")
20: %link(href="images/apple-touch-icon-72x72.png" rel="apple-touch-icon-precomposed" sizes="72x72")
app/views/layouts/application.html.haml:17:in `_app_views_layouts_application_html_haml__1709115770063515875_49642660'
app/controllers/public_controller.rb:39:in `index'
It appears that when sprockets asked for the K5
directory to be created, it returned with no side effect because the directory already existed (as k5
). Then when actually creating the file, the ENOTDIR
error was raised because the directory didn’t actually exist.
Rolling back to version 3.5.2 fixes the issue for me.
I suspect similar problems may have been possible before the bc40f15dd10701034524e2cb53d373190fe0014c patch, however, such a large collision space (10^105) means the problem didn’t occur. Once you account for case-insensitive operation, there’s only 1,444 combinations available in a 2 character sequence.
Other details:
-
I’m using Vagrant
- host: Windows 10, NTFS file system
- guest: Ubuntu 14.04.4 LTS
- synced folder using nfs
-
I’ve tried to isolate the problem by testing the following:
-
mkdir K5
also fails (does not create theK5
directory, does not error) in Ubuntu -
mkdir.exe
fails in Windows withmkdir.exe: cannot create directory
K5’: File exists` -
FileUtils.mkdir_p
on Ubuntu, in nfs mounted directory, gives (only one folder created and has casing from first command)2.2.4 :002 > require 'fileutils' => true 2.2.4 :003 > FileUtils.mkdir_p 'K5' => ["K5"] 2.2.4 :004 > FileUtils.mkdir_p 'k5' => ["k5"]
-
FileUtils.mkdir_p
on Windows gives (only one folder created and has casing from first command)irb(main):001:0> require 'fileutils' => true irb(main):010:0> FileUtils.mkdir_p 'K5' => ["K5"] irb(main):011:0> FileUtils.mkdir_p 'k5' => ["k5"]
I’m not sure what the best solution is, but the only thing that comes to mind is to
downcase
the namespace before using it. -
About this issue
- Original URL
- State: open
- Created 8 years ago
- Reactions: 7
- Comments: 27 (13 by maintainers)
Commits related to this issue
- More work on analysis job items Started adding routes and route specs. Rolled back sprockets dependency due to https://github.com/rails/sprockets/issues/283 — committed to QutEcoacoustics/baw-server by atruskie 8 years ago
- Feature: AnalsisJobsItem - table, model, controller, and tests Note: this work does not include functionality for generating AnalysisJosItems. Note: Rolled back sprockets dependency due to https://g... — committed to QutEcoacoustics/baw-server by atruskie 8 years ago
- https://github.com/rails/sprockets/issues/283 — committed to discourse-pro/sprockets by dmitrii-fediuk 8 years ago
- Bundle update - needed to do windows fix for sprockets bug at https://github.com/rails/sprockets/issues/283 - fsutil.exe file SetCaseSensitiveInfo C:\folder\path enable — committed to NEU-Libraries/charon by dgcliff 4 years ago
- Fix sprockets bug on Windows Sprockets weirds out on Windows, see https://github.com/rails/sprockets/issues/283#issuecomment-578728257 — committed to rafaelnogalha/Reserva-de-Salas by LeoRiether 3 years ago
@nicwillemse literally also just stumbled onto this due to trying something out in Docker Desktop for Windows and the issue is that a base64 digest is case sensitive so all versions since 372301c could cause a conflict. However the namespace change in 3.6 greatly increases the chances of that occurring since it’s just two characters.
I’m working around the problem for now by mounting a docker volume as a custom cache path for Sprockets, e.g:
Full commit here
Like @Smolations commented, looks like this is related with how Windows manages filenames.
Enabling case sensitivity on /tmp folder fixed the issue for me:
fsutil.exe file SetCaseSensitiveInfo C:\folder\path enable
I’m seeing this issue as well when running inside Docker for MacOSX using Rails 5.2.0 and Sprockets 3.7.2. I cannot load any pages in development - I get the following error: File exists @ dir_s_mkdir - /home/app/webapp/tmp/cache/assets/sprockets/v3.0/[directory name]. My Docker image is based off of the phusion passenger image: https://github.com/phusion/passenger-docker
Precompiling assets will temporarily resolve the issue, but this isn’t a tractable solution for development work. If I backdate to an image that was based off of Rails 4.2, I don’t have this problem. So there’s something now with the combination of Rails 5 & sprockets 3.7 that is at fault. I updated to 3.7.2 because of CVE-2018-3760, so I can’t backdate my version.
My Gemfile.lock: https://github.com/broadinstitute/single_cell_portal_core/blob/development/Gemfile.lock
Any suggestions?
Thanks @pheuko - but this unfortunately will not work as the attributes are not inherited in the folder structure, see https://devblogs.microsoft.com/commandline/per-directory-case-sensitivity-and-wsl/ for more detail.
Just adding to the error report + possible temporary solution.
My dev setup is a Debian box in Vagrant on Windows 7 host with winnfsd plugin for vagrant to simulate NFS. I’m getting the same error. Temporary solution for me is to explicitly set sprockets to version 3.5.2 in Gemfile.