cog: Prediction failed for an unknown reason. It might have run out of memory? (exitcode 2)
When running locally the cog predict -i it working fine but when pushed on replicate it giving this error
Traceback (most recent call last): File "/root/.pyenv/versions/3.8.16/lib/python3.8/site-packages/cog/server/runner.py", line 334, in predict return _predict( File "/root/.pyenv/versions/3.8.16/lib/python3.8/site-packages/cog/server/runner.py", line 370, in _predict for event in worker.predict(input_dict, poll=0.1): File "/root/.pyenv/versions/3.8.16/lib/python3.8/site-packages/cog/server/worker.py", line 135, in _wait raise FatalWorkerException( cog.server.exceptions.FatalWorkerException: Prediction failed for an unknown reason. It might have run out of memory? (exitcode 2) any idea @anotherjesse @bfirsh
About this issue
- Original URL
- State: open
- Created a year ago
- Comments: 17 (2 by maintainers)
I’ve not worked on cog from a lot of time, but this might help someone:
I was able to run it locally as I had better configurations locally than the VM on which I was trying to deploy. I will suggest reducing the model size or increasing the memory of the machine on which you are trying to deploy.
Additionally try to add a try-except block and do following everywhere in the exception block:
Same issue 🤔