keylime: invoke_get_quote fails on tornado 6.0

The 'callback` function has been deprecated in 6.0 release:

https://www.tornadoweb.org/en/stable/releases/v6.0.0.html

This results in the following code breaking:

https://github.com/keylime/keylime/blob/7c6e0ea539d3952991e77eaefb57fbf14b83ea4e/keylime/cloud_verifier_tornado.py#L295-L307

and possibly…

https://github.com/keylime/keylime/blob/7c6e0ea539d3952991e77eaefb57fbf14b83ea4e/keylime/revocation_notifier.py#L71-L90

Using config file /etc/keylime.conf
2019-10-17 12:09:44.918 - keylime.cloudverifier - INFO - Starting Cloud Verifier (tornado) on port 8881, use <Ctrl-C> to stop
2019-10-17 12:09:44.919 - keylime.cloudverifier_common - INFO - Setting up TLS...
2019-10-17 12:09:44.920 - keylime.cloudverifier_common - INFO - Existing CA certificate found in /var/lib/keylime/cv_ca, not generating a new one
2019-10-17 12:09:44.922 - keylime.cloudverifier - INFO - Starting service for revocation notifications on port 8992
2019-10-17 12:10:33.188 - keylime.cloudverifier - ERROR - Polling thread error: __init__() got an unexpected keyword argument 'callback'
2019-10-17 12:10:33.189 - keylime.cloudverifier - ERROR - __init__() got an unexpected keyword argument 'callback'
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/keylime-1.2-py3.7.egg/keylime/cloud_verifier_tornado.py", line 406, in process_agent
    self.invoke_get_quote(agent, True)
  File "/usr/local/lib/python3.7/site-packages/keylime-1.2-py3.7.egg/keylime/cloud_verifier_tornado.py", line 307, in invoke_get_quote
    client.fetch(url, callback=cb)
  File "/usr/local/lib64/python3.7/site-packages/tornado/httpclient.py", line 284, in fetch
    request = HTTPRequest(url=request, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'callback'
2019-10-17 12:10:33.192 - keylime.cloudverifier - INFO - POST returning 200 response for adding agent id: D432FBB3-D2F1-4A97-9EF7-75BD81C00000
2019-10-17 12:10:33.486 - keylime.cloudverifier - INFO - PUT returning 404 response. agent id: D432FBB3-D2F1-4A97-9EF7-75BD81C00000 not found.

I guess we will need to replace this with async code.

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 16 (14 by maintainers)

Commits related to this issue

Port verifer to tornado release 6.0.x `callback` has been depreciated from tornado 6.0 This code changes the `invoke_get_quote` from a callback to a while loop Resolves: #196 — committed to keylime/keylime by deleted user 5 years ago
Port verifer to tornado release 6.0.x `callback` has been depreciated from tornado 6.0 This code changes the `invoke_get_quote` from a callback to a while loop Resolves: #196 — committed to keylime/keylime by deleted user 5 years ago
Async implementation to support Tornado version 6 callbacks have now been depreciated so we need to move to using asyncio. Resolves: #196 — committed to keylime/keylime by deleted user 5 years ago
Async implementation to support Tornado version 6 callbacks have now been depreciated so we need to move to using asyncio. Resolves: #196 — committed to keylime/keylime by deleted user 5 years ago
Async implementation to support Tornado version 6 callbacks have now been depreciated so we need to move to using asyncio. Resolves: #196 — committed to keylime/keylime by deleted user 5 years ago
Async implementation to support Tornado version 6 callbacks have now been depreciated so we need to move to using asyncio. Resolves: #196 — committed to keylime/keylime by deleted user 5 years ago
Async implementation to support Tornado version 6 Callbacks have now been depreciated so we need to move to using asyncio. Also includes better detection of CA directory Resolves: #196 — committed to keylime/keylime by deleted user 5 years ago

Most upvoted comments

Hi @lukehinds

I haven’t found a way to do this without using the nest-asyncio PIP package. It sounds like python and tornado developers are trying to prevent developers from doing recursive event loops, which we need here.

If you’re willing to use (and install) the nest-asyncio pip package, then here is a patch to cloud_verifier_tornado.py that works on my end.

diff --git a/keylime/cloud_verifier_tornado.py b/keylime/cloud_verifier_tornado.py
index f884952..31b9ca6 100644
--- a/keylime/cloud_verifier_tornado.py
+++ b/keylime/cloud_verifier_tornado.py
@@ -24,6 +24,8 @@ import traceback
 import sys
 import functools
 import asyncio
+import nest_asyncio
+nest_asyncio.apply()
 
 import tornado.ioloop
 import tornado.web
@@ -293,6 +295,9 @@ class AgentsHandler(BaseHandler):
 
 
     def invoke_get_quote(self, agent, need_pubkey):
+        asyncio.get_event_loop().run_until_complete(self._invoke_get_quote(agent, need_pubkey))
+
+    async def _invoke_get_quote(self, agent, need_pubkey):
         params = cloud_verifier_common.prepare_get_quote(agent)
         agent['operational_state'] = cloud_verifier_common.CloudAgent_Operational_State.GET_QUOTE
         client = tornado.httpclient.AsyncHTTPClient()
@@ -302,11 +307,8 @@ class AgentsHandler(BaseHandler):
             partial_req = "0"
 
         url = "http://%s:%d/quotes/integrity?nonce=%s&mask=%s&vmask=%s&partial=%s"%(agent['ip'],agent['port'],params["nonce"],params["mask"],params['vmask'],partial_req)
-        # the following line adds the agent and params arguments to the callback as a convenience
-        cb = functools.partial(self.on_get_quote_response, agent, url)
-        client.fetch(url, callback=cb)
-
-    def on_get_quote_response(self, agent, url, response):
+        response = await client.fetch(url)
+        
         if agent is None:
             raise Exception("agent deleted while being processed")
         if response.error:
@@ -341,17 +343,19 @@ class AgentsHandler(BaseHandler):
 
 
 
+
     def invoke_provide_v(self, agent):
+        asyncio.get_event_loop().run_until_complete(self._invoke_provide_v(agent))
+
+    async def _invoke_provide_v(self, agent):
         if agent['pending_event'] is not None:
             agent['pending_event'] = None
         v_json_message = cloud_verifier_common.prepare_v(agent)
         agent['operational_state'] = cloud_verifier_common.CloudAgent_Operational_State.PROVIDE_V
         client = tornado.httpclient.AsyncHTTPClient()
         url = "http://%s:%d/keys/vkey"%(agent['ip'],agent['port'])
-        cb = functools.partial(self.on_provide_v_response, agent, url)
-        client.fetch(url, method="POST", callback=cb, headers=None, body=v_json_message)
+        response = await client.fetch(url, method="POST", headers=None, body=v_json_message)
 
-    def on_provide_v_response(self, agent, url_with_params, response):
         if agent is None:
             raise Exception("Agent deleted while being processed")
         if response.error:

What do you think?

jetwhiz on Nov 1, 2019