moby: Cannot specify --storage-opt to "docker build"
Description
We are working on containerizing a very large application where the default volume size may be insufficient but cannot pass --storage-opt to docker build. Changing the daemon configuration may be a blocker for customers wishing to build upon our base image we’re trying to build. We can change the amount of memory provided to the container(s) which is necessary during build, so it would be nice to be able to specify the volume size as well.
But rather than merely support storage-opt for the build command, would it make more sense to be allowed to pass through arguments from build to use when running containers for each RUN statement?
Output of docker version:
Client:
Version: 17.06.2-ce
API version: 1.30
Go version: go1.8.3
Git commit: cec0b72
Built: Tue Sep 5 19:57:19 2017
OS/Arch: windows/amd64
Server:
Version: 17.06.2-ce
API version: 1.30 (minimum version 1.24)
Go version: go1.8.3
Git commit: cec0b72
Built: Tue Sep 5 19:59:47 2017
OS/Arch: windows/amd64
Experimental: true
Output of docker info:
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 28
Server Version: 17.06.2-ce
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: l2bridge l2tunnel nat null overlay transparent
Log: awslogs etwlogs fluentd json-file logentries splunk syslog
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 15063 (15063.0.amd64fre.rs2_release.170317-1834)
Operating System: Windows 10 Enterprise
OSType: windows
Architecture: x86_64
CPUs: 8
Total Memory: 31.92GiB
Name: heathswrk
ID: PPYQ:6372:4QVR:ERCP:U676:ZHX5:T6FG:RKYE:O4XN:FDP4:7DMT:CQ7P
Docker Root Dir: E:\Docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: -1
Goroutines: 32
System Time: 2017-09-22T13:28:45.4667296-07:00
EventsListeners: 0
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional environment details (AWS, VirtualBox, physical, etc.):
Using Hyper-V isolation on Windows 10 (currently all that’s supported).
About this issue
- Original URL
- State: open
- Created 7 years ago
- Reactions: 4
- Comments: 19 (6 by maintainers)
After discussing different options for implementation with @jhowardmsft and @johnstep, I’m working on a change to add
--storage-optto thebuildcommand akin to theruncommand.I agree that this is something we want. We’ve had multiple asks for this (ex. https://github.com/moby/moby/issues/33469), and it’s much more reasonable to have a base image size specified at build time than being a global daemon option imo.
cc @jhowardmsft @johnstep
@jhowardmsft, certain installs I manage are also starting to grow beyond 127GB for a full install of all first- and third-party software. We will need larger sizes - something like my original change would have partly provided. Seems only fitting that if
docker runcan specify a larger size,docker buildshould be able to as well.@TBBle it was done but rejected. It’s been long enough I imagine it would have to be done again, but I’d be happy to contribute. The problem with the config file is that users have to change it every time they need to increase the build size. The convenience of plumbing through the command line option is being able to change it as needed only - same as with passing storage opts to
docker runor even memory opts todocker build.20GB limit for COPY was never merged to 20.10.x branch.
@crossan007 How large? The default scratch is now 127GB. Bigger than that?
@johnstep created PR #35780 but it was rejected after the default size was raised to 127GB (same as the default for Hyper-V). Apart from being rather short-sighted, IMO (using build tools containers for large solutions might require even more space, which means it should be configurable), I don’t know if this made it into Docker 4 Windows in either the edge or stable channels. It’s been nearly a year so I’d hope so.
Possibly. There’s basically two goals here. We’re containerizing Build Tools for Visual Studio 2017 but its not yet as feature-rich (i.e. doesn’t support building all common project types) as Visual Studio 2017 itself, so even if we made a base image of Build Tools available on which customers may build, customers may also need to install Visual Studio 2017 into an image for now. So besides chasing down any blocking bugs to just building a complete image ourselves, I also need to provide guidance for customers building their own. So being able to pass more options (or just specific ones, like
--storage-opt) to control intermediate containers would be very helpful.