st2: st2 fails to store a Key-Value in the datastore if there is a "/" in the key name.

SUMMARY

st2 fails to store a Key-Value in the datastore if there is a “/” in the key name. I am sorry if I missed anything in the docs, which says that you cannot store a key with “/” in it and please feel free to close this issue if that is the case. Thank you.

STACKSTORM VERSION

st2 3.1.0, on Python 2.7.5

OS, environment, install method:

CentOS/Docker, custom install.

Steps to reproduce the problem

bash-4.2$ st2 key set "foo / bar" "foo_bar"
ERROR: 404 Client Error: Not Found
MESSAGE: The resource could not be found. for url: http://127.0.0.1:9101/v1/keys/foo%20/%20bar

Expected Results

st2 should have accepted the key-value pair and stored it.

Actual Results

st2 has failed to store the key-value pair.

About this issue

Original URL
State: open
Created 5 years ago
Comments: 18 (8 by maintainers)

Most upvoted comments

Thank you for the input @amanda11. I just tried with double encoding/quoting the key name if it contains a / while I set or get the KV pair and it is working as expected. If this is OK, I will go ahead and submit a pull request. @Kami @armab

git diff:

diff --git a/st2client/st2client/commands/keyvalue.py b/st2client/st2client/commands/keyvalue.py
index 8eed47364..d1a2305eb 100644
--- a/st2client/st2client/commands/keyvalue.py
+++ b/st2client/st2client/commands/keyvalue.py
@@ -21,6 +21,7 @@ import logging
 from os.path import join as pjoin

 import six
+from six.moves.urllib_parse import quote

 from st2client.commands import resource
 from st2client.commands.noop import NoopCommand
@@ -141,6 +142,8 @@ class KeyValuePairGetCommand(resource.ResourceGetCommand):
     @resource.add_auth_token_to_kwargs_from_cli
     def run(self, args, **kwargs):
         resource_name = getattr(args, self.pk_argument_name, None)
+        if '/' in resource_name:
+            resource_name = quote(quote(resource_name, safe=''))
         decrypt = getattr(args, 'decrypt', False)
         scope = getattr(args, 'scope', DEFAULT_GET_SCOPE)
         kwargs['params'] = {'decrypt': str(decrypt).lower()}
@@ -185,8 +188,14 @@ class KeyValuePairSetCommand(resource.ResourceCommand):
     @resource.add_auth_token_to_kwargs_from_cli
     def run(self, args, **kwargs):
         instance = KeyValuePair()
-        instance.id = args.name  # TODO: refactor and get rid of id
-        instance.name = args.name
+        key_name = args.name
+        # urllib_parse.quote the key name to support keys with '/' in them.
+        # We double quote it here, as it will unquoted once on the API side.
+        if '/' in key_name:
+            key_name = quote(quote(args.name, safe=''))
+
+        instance.id = key_name  # TODO: refactor and get rid of id
+        instance.name = key_name
         instance.value = args.value
         instance.scope = args.scope
         instance.user = args.user

RaviTezu on Jun 7, 2020

I did this little test based on the code in router.py:

import webob.compat path=“http://127.0.0.1:9101/v1/keys/foo%2Fbar” webob.compat.url_unquote(path) ‘http://127.0.0.1:9101/v1/keys/foo/bar’

So it looks like it gets altered to foo/bar by the url_unquote call.

I believe the webob.compat inturn calls the urllib method, which is defined to be: urllib.parse.unquote(string, encoding=‘utf-8’, errors=‘replace’)¶ Replace %xx escapes by their single-character equivalent. The optional encoding and errors parameters specify how to decode percent-encoded sequences into Unicode characters, as accepted by the bytes.decode() method.

So that seems to be the behaviour of the unquote.

amanda11 on Jun 5, 2020