clipper: How to debug cause of "ImportError when running Container"
I am a bit stuck: I’m trying to deploy a pytorch model which runs fine on my local computer but hangs when trying to deploy it. The model’s docker container gives me this error log:
$ docker logs 4437076d479c
Starting PyTorchContainer container
Connecting to Clipper with default port: 7000
Encountered an ImportError when running container. You can use the pkgs_to_install argument when calling clipper_admin.build_model() to supply any needed Python packages.
It seems like it is not able to import some dependency. I installed the same model in a clean anaconda environment on my computer and checked exactly which dependencies I needed to install. I listed these in pkgs_to_install. I have inspected all the log files of the docker containers and I can’t find the actual error message which could give me a hint as to which dependency is missing. May some error log be getting swallowed somewhere?
The model I’m trying to run is the xinntao/ESRGAN for image super-resolution.
Dependencies: Python 3 PyTorch >= 0.4.0 Python packages: pip install numpy opencv-python
The command to deploy the model:
deploy_pytorch_model(
clipper_conn,
name="superresolution-model",
version=1,
input_type="bytes",
func=image_enhance,
pytorch_model=model,
pkgs_to_install=['opencv-python','numpy','six', 'Pillow','wheel',]
)
the clipper_service.py is where I define the model and try to deploy.
About this issue
- Original URL
- State: open
- Created 5 years ago
- Comments: 15 (6 by maintainers)
Thank you guys for your fast response… @simon-mo I had actually tried consolidating those 3 files into one and had not gotten any further. But it’s good to know I need to do that. I still haven’t completely wrapped my head around what gets run where.
I will try building a custom model container with the latest pytorch and consolidate the files into one and let you know if it works. Thanks!