clipper: Website tutorial keeps timing out on trying to add model image (all i changed with the prometheus port)
The Getting started tutorial for clipper just keeps making a readtimeout error. Here’s the python.
from clipper_admin import ClipperConnection, DockerContainerManager
from clipper_admin.deployers import python as python_deployer
# set prometheus port to 9290 because the default 9090 is used
print("Setting prometheus port")
clipper_conn = ClipperConnection(DockerContainerManager(prometheus_port=9290))
#stop any existing containers
print("Stoping all containers")
clipper_conn.stop_all()
#start clipper
print("Starting clipper")
clipper_conn.start_clipper()
#add hello-world app
print("Adding hello world app")
clipper_conn.register_application(name="hello-world", input_type="doubles", default_output="-1.0", slo_micros=100000)
#define feature_sum function
def feature_sum(xs):
return [str(sum(x)) for x in xs]
#deploy feature_sum function to sum-model model
print("Deploying feature_sum model")
python_deployer.deploy_python_closure(clipper_conn, name="sum-model", version=1, input_type="doubles", func=feature_sum)
#route hello world app to feature_sum model
print("Routing hello world app to feature_sum model")
clipper_conn.link_model_to_app(app_name="hello-world", model_name="sum-model")
Here’s the bash
clipper@ip-172-31-26-36:~/repo$ python test.py
Setting prometheus port
Stoping all containers
18-08-28:14:08:41 INFO [clipper_admin.py:1258] Stopped all Clipper cluster and all model containers
Starting clipper
18-08-28:14:08:41 INFO [docker_container_manager.py:119] Starting managed Redis instance in Docker
18-08-28:14:08:57 INFO [clipper_admin.py:126] Clipper is running
Adding hello world app
18-08-28:14:08:57 INFO [clipper_admin.py:201] Application hello-world was successfully registered
Deploying feature_sum model
18-08-28:14:08:57 INFO [deployer_utils.py:44] Saving function to /tmp/clipper/tmpf6HPvf
18-08-28:14:08:57 INFO [deployer_utils.py:54] Serialized and supplied predict function
18-08-28:14:08:57 INFO [python.py:192] Python closure saved
18-08-28:14:08:57 INFO [python.py:198] Using Python 2 base image
18-08-28:14:08:57 INFO [clipper_admin.py:452] Building model Docker image with model data from /tmp/clipper/tmpf6HPvf
18-08-28:14:09:04 INFO [clipper_admin.py:456] {u'stream': u'Step 1/2 : FROM clipper/python-closure-container:0.3.0'}
18-08-28:14:09:04 INFO [clipper_admin.py:456] {u'stream': u'\n'}
18-08-28:14:09:04 INFO [clipper_admin.py:456] {u'stream': u' ---> bfcdf8ffce0c\n'}
18-08-28:14:09:04 INFO [clipper_admin.py:456] {u'stream': u'Step 2/2 : COPY /tmp/clipper/tmpf6HPvf /model/'}
18-08-28:14:09:04 INFO [clipper_admin.py:456] {u'stream': u'\n'}
18-08-28:14:09:04 INFO [clipper_admin.py:456] {u'stream': u' ---> 793a132eea38\n'}
18-08-28:14:09:04 INFO [clipper_admin.py:456] {u'aux': {u'ID': u'sha256:793a132eea3837a49b311e28325d50c3082601b3145d756283a9f3942bc23d95'}}
18-08-28:14:09:04 INFO [clipper_admin.py:456] {u'stream': u'Successfully built 793a132eea38\n'}
18-08-28:14:09:04 INFO [clipper_admin.py:456] {u'stream': u'Successfully tagged sum-model:1\n'}
18-08-28:14:09:04 INFO [clipper_admin.py:458] Pushing model Docker image to sum-model:1
18-08-28:14:09:04 INFO [docker_container_manager.py:257] Found 0 replicas for sum-model:1. Adding 1
Traceback (most recent call last):
File "test.py", line 28, in <module>
python_deployer.deploy_python_closure(clipper_conn, name="sum-model", version=1, input_type="doubles", func=feature_sum)
File "/home/clipper/.local/lib/python2.7/site-packages/clipper_admin/deployers/python.py", line 222, in deploy_python_closure
registry, num_replicas, batch_size, pkgs_to_install)
File "/home/clipper/.local/lib/python2.7/site-packages/clipper_admin/clipper_admin.py", line 338, in build_and_deploy_model
num_replicas, batch_size)
File "/home/clipper/.local/lib/python2.7/site-packages/clipper_admin/clipper_admin.py", line 544, in deploy_model
num_replicas=num_replicas)
File "/home/clipper/.local/lib/python2.7/site-packages/clipper_admin/docker/docker_container_manager.py", line 192, in deploy_model
self.set_num_replicas(name, version, input_type, image, num_replicas)
File "/home/clipper/.local/lib/python2.7/site-packages/clipper_admin/docker/docker_container_manager.py", line 262, in set_num_replicas
image)
File "/home/clipper/.local/lib/python2.7/site-packages/clipper_admin/docker/docker_container_manager.py", line 238, in _add_replica
**self.extra_container_kwargs)
File "/home/clipper/.local/lib/python2.7/site-packages/docker/models/containers.py", line 772, in run
container.start()
File "/home/clipper/.local/lib/python2.7/site-packages/docker/models/containers.py", line 374, in start
return self.client.api.start(self.id, **kwargs)
File "/home/clipper/.local/lib/python2.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "/home/clipper/.local/lib/python2.7/site-packages/docker/api/container.py", line 1065, in start
res = self._post(url)
File "/home/clipper/.local/lib/python2.7/site-packages/docker/utils/decorators.py", line 46, in inner
return f(self, *args, **kwargs)
File "/home/clipper/.local/lib/python2.7/site-packages/docker/api/client.py", line 194, in _post
return self.post(url, **self._set_request_timeout(kwargs))
File "/home/clipper/.local/lib/python2.7/site-packages/requests/sessions.py", line 559, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/home/clipper/.local/lib/python2.7/site-packages/requests/sessions.py", line 512, in request
resp = self.send(prep, **send_kwargs)
File "/home/clipper/.local/lib/python2.7/site-packages/requests/sessions.py", line 622, in send
r = adapter.send(request, **kwargs)
File "/home/clipper/.local/lib/python2.7/site-packages/requests/adapters.py", line 526, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
clipper@ip-172-31-26-36:~/repo$
How do i increase that timeout?.. or better yet, what or where exactly is the problem?
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 16
@finsqm @ebanner : Please refer to this issue, https://github.com/ucbrise/clipper/issues/568