temporal: Error: Add search attribute failed
Expected Behavior
I should be able to add a new search attribute “UserID” of type Keyword to Temporal for filtering.
Actual Behavior
I initially receive an error when trying to add attribute. Upon listing added keys, I see the newly added key but it is of the wrong type (String).
Steps to Reproduce the Problem
- Deploy the latest Temporal 0.26.0 with Elasticsearch enabled.
- Add a new search attribute by running the following command:
tctl --namespace default adm cl asa --search_attr_key UserID --search_attr_type 1 - Observe the following error and stack trace:
bash-5.0# tctl --namespace default adm cl asa --search_attr_key UserID --search_attr_type 1
Are you trying to add key [UserID] with Type [Keyword]? Y/N y
Error: Add search attribute failed.
Error Details: Failed to update dynamic config, err: unable to update key.
Stack trace:
goroutine 1 [running]:
runtime/debug.Stack(0xd, 0x0, 0x0)
/usr/local/go/src/runtime/debug/stack.go:24 +0x9d
runtime/debug.PrintStack()
/usr/local/go/src/runtime/debug/stack.go:16 +0x22
github.com/temporalio/temporal/tools/cli.printError(0x1c5008f, 0x1c, 0x2003cc0, 0xc00000c300)
/temporal/tools/cli/util.go:526 +0x2ad
github.com/temporalio/temporal/tools/cli.ErrorAndExit(0x1c5008f, 0x1c, 0x2003cc0, 0xc00000c300)
/temporal/tools/cli/util.go:537 +0x49
github.com/temporalio/temporal/tools/cli.AdminAddSearchAttribute(0xc000127b80)
/temporal/tools/cli/adminClusterCommands.go:61 +0x50f
github.com/temporalio/temporal/tools/cli.newAdminClusterCommands.func1(0xc000127b80)
/temporal/tools/cli/admin.go:812 +0x2b
github.com/urfave/cli.HandleAction(0x18cf500, 0x1cc9b88, 0xc000127b80, 0xc000127b80, 0x0)
/go/pkg/mod/github.com/urfave/cli@v1.22.4/app.go:528 +0x7c
github.com/urfave/cli.Command.Run(0x1c3692d, 0xf, 0x0, 0x0, 0xc0006998c0, 0x1, 0x1, 0x1c4d6ff, 0x1a, 0x0, ...)
/go/pkg/mod/github.com/urfave/cli@v1.22.4/command.go:174 +0x57a
github.com/urfave/cli.(*App).RunAsSubcommand(0xc0004d9c00, 0xc000127760, 0x0, 0x0)
/go/pkg/mod/github.com/urfave/cli@v1.22.4/app.go:407 +0x915
github.com/urfave/cli.Command.startApp(0x1c2a872, 0x7, 0x0, 0x0, 0xc0006999f0, 0x1, 0x1, 0x1c551de, 0x1e, 0x0, ...)
/go/pkg/mod/github.com/urfave/cli@v1.22.4/command.go:373 +0x845
github.com/urfave/cli.Command.Run(0x1c2a872, 0x7, 0x0, 0x0, 0xc0006999f0, 0x1, 0x1, 0x1c551de, 0x1e, 0x0, ...)
/go/pkg/mod/github.com/urfave/cli@v1.22.4/command.go:102 +0xa2b
github.com/urfave/cli.(*App).RunAsSubcommand(0xc0004d9a40, 0xc000127600, 0x0, 0x0)
/go/pkg/mod/github.com/urfave/cli@v1.22.4/app.go:407 +0x915
github.com/urfave/cli.Command.startApp(0x1c27ae6, 0x5, 0x0, 0x0, 0xc000699970, 0x1, 0x1, 0x1c3df67, 0x13, 0x0, ...)
/go/pkg/mod/github.com/urfave/cli@v1.22.4/command.go:373 +0x845
github.com/urfave/cli.Command.Run(0x1c27ae6, 0x5, 0x0, 0x0, 0xc000699970, 0x1, 0x1, 0x1c3df67, 0x13, 0x0, ...)
/go/pkg/mod/github.com/urfave/cli@v1.22.4/command.go:102 +0xa2b
github.com/urfave/cli.(*App).Run(0xc0004d96c0, 0xc00010c000, 0xa, 0xa, 0x0, 0x0)
/go/pkg/mod/github.com/urfave/cli@v1.22.4/app.go:279 +0x7c7
main.main()
/temporal/cmd/tools/cli/main.go:37 +0x4b
- List the search attributes and notice the wrong type:
bash-5.0# tctl --namespace default cl get-search-attr
+-----------------------+------------+
| KEY | VALUE TYPE |
+-----------------------+------------+
| BinaryChecksums | Keyword |
| CloseTime | Int |
| CustomBoolField | Bool |
| CustomDatetimeField | Datetime |
| CustomDoubleField | Double |
| CustomIntField | Int |
| CustomKeywordField | Keyword |
| CustomStringField | String |
| ExecutionStatus | Int |
| ExecutionTime | Int |
| HistoryLength | Int |
| NamespaceId | Keyword |
| RunId | Keyword |
| StartTime | Int |
| TaskQueue | Keyword |
| TemporalChangeVersion | Keyword |
| UserID | String |
| WorkflowId | Keyword |
| WorkflowType | Keyword |
+-----------------------+------------+
Additional Questions
Now that I have an incorrect attribute added, is there any way to edit or remove it? Does the search attribute need to be registered explicitly by the CLI before it can be used elsewhere (go-sdk)? The docs mention something here but it’s still a bit unclear: https://docs.temporal.io/docs/learn-workflow-filtering#search-attributes-go-client-usage
Specifications
- Version: 0.26.0
- Platform: Ubuntu 18.04
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 19 (10 by maintainers)
If you happen to find this issue today, this is no longer relevant (might be obvious, I don’t know). As per https://github.com/temporalio/temporal/issues/1442#issuecomment-878705053 and https://docs.temporal.io/docs/server/workflow-search#local-testing, you have to use tctl now.
Hey Paul,
First of all, the way of building the docker image is much preferred over that short-cut I mentioned earlier. Thanks for pointing this out…I will update my answer to incorporate your suggestion to just clone the repo and build directly.
Secondly, I forgot one more vital step: In addition to updating the development_es.yaml file, you also have to update the schema/elasticsearch/visibility/index_template.json file so that it actually adds the index to the Elastic Search cluster when the Temporal server boots up. I went ahead and added a key/value pair “UserID” : { “type”: {“keyword”} to the properties dictionary under “Attr” and tested locally, and after that, I was able to query workflows using the property.
So TL;DR, it looks like there are TWO files that have to be customized on the server image:
development_es.yaml schema/elasticsearch/visibility/index_template.json
Really sorry for the inconvenience here. That should fix it, and we’ll look to make this clearer via documentation and have a more seamless way to configure these types of things in the future.
FYI, we will try and make the injection of dynamic configuration more seamless as well so that you don’t have to create a custom image as has been outlined above. For now, the above steps should unblock you for experimentation purposes though.
Ok, let me dig into this and get back to you regarding the error you are getting.
On a separate FYI, we’re landing a change so that the --search_attr_type parameter takes in the actual case-insensitive type-name (e.g. string, keyword, int, etc) so that you don’t have to look at a reverse mapping to an integer. We were able to confirm that adding attributes / searching workflows by those attributes worked properly with that change.
@cullywakelin can you look into updating the docs to reflect this?