ml-commons: [INFO] custom pre-processing function in ML connectors is returning `Invalid JSON in payload`
What is the bug? I am trying to deploy an ML model that connects to an external resource
I am trying to write a pre-process function that will pass the following request body to my Sagemaker endpoint
{
"inputs": [
{
"name": "query",
"shape": [${parameters.input.length}, 1],
"datatype": "BYTES",
"data": ${parameters.input}
}
]
}
I have created the ML connector and deployed the model following the steps below. When I do this I get the following Invalid JSON in payload error. Could someone assist me in understanding why the pre-processing function is not working as expected?
How can one reproduce the bug? Steps to reproduce the behavior:
- create connector
POST /_plugins/_ml/connectors/_create
{
"name": "my_connector",
"description": "My Connector",
"version": 1,
"protocol": "aws_sigv4",
"credential": {
"roleArn": "my_role_arn"
},
"parameters": {
"region": "us-west-2",
"service_name": "sagemaker"
},
"actions": [
{
"action_type": "predict",
"method": "POST",
"headers": {
"content-type": "application/json"
},
"url": "my_sagemaker_endpoint",
"request_body": "{ \"inputs\": [{ \"name\": \"query\", \"shape\": [${parameters.length}, 1], \"datatype\": \"BYTES\", \"data\": ${parameters.input} }] }",
"pre_process_function": "return '{\"length\": ' + params.text_docs.length + ', \"input\": ' + params.text_docs + ' }'",
"post_process_function": "def data = params.outputs[0].data; def n = Math.ceil(data.length / 512);def embeddings = [];for (int i = 0; i < n; i++) {embeddings.add(Arrays.copyOfRange(data, i * 512, Math.min(data.length, 512 * (i + 1))));}return '{\"name\": \"sentence_embedding\", \"data_type\": \"FLOAT32\", \"shape\": [' + n + '], \"data\": ' + embeddings + '}';"
}
]
}
- register and deploy model
POST /_plugins/_ml/models/_register
{
"name": "my_model",
"function_name": "remote",
"model_group_id": "my_model_group_id",
"description": "My Model",
"connector_id": "my_connector_id"
}
POST /_plugins/_ml/models/my_model_id/_deploy
- create ingest pipeline
PUT /_ingest/pipeline/my-ingest-pipeline
{
"description": "My ingest pipeline",
"processors": [
{
"convert": {
"field": "brand.id",
"type": "string"
}
},
{
"convert": {
"field": "category.id",
"type": "string"
}
},
{
"text_embedding": {
"model_id": "my_model_id",
"field_map": {
"name": {
"en": "name_vector"
},
"brand": {
"name": {
"en": "brand_name_vector"
}
},
"category": {
"name": {
"en": "category_name_vector"
}
}
}
}
}
]
}
- simulate pipeline and see error
POST _ingest/pipeline/my-ingest-pipeline/_simulate
{
"docs": [
{
"_index": "my-index",
"_id": "1",
"_source": {
"brand": {
"id": 1,
"name": {
"en": "brand 1"
}
},
"category": {
"id": 1,
"name": {
"en": "category 1"
}
}
}
}
]
}
What is the expected behavior?
Expected generated vector embeddings brand_name_vector and category_name_vector
What is your host/environment?
- Environment: AWS Opensearch - Managed Cluster
- OS: Opensearch
- Version: 2.11
Do you have any screenshots?
{
"docs": [
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Invalid JSON in payload"
}
],
"type": "illegal_argument_exception",
"reason": "Invalid JSON in payload"
}
}
]
}
Do you have any additional context? I am aware of signing the request using AWS SigV4 and providing the correct keys. The issue is not with creating the connector, I am able to create the connector fine.
My issue is when I deploy my model using the connector, and simulate the ingest pipeline, that is when I get the error described.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 15
@toyaokeke Can you try these pre/post process function?
Edit: As you are using OS 2.11, you should add the
escapemethod to pre-process function, the whole pre-process function with escape function will beThe above
pre_process_functionhas Unicode escape sequence , you can also use this@ylwu-amzn this is working as expected!! Thank you very much 🙏🏿
Will be hard to guess what’s wrong. If possible, you can reach out to me on OpenSearch slack and we can jump to a call? You can join the public
mlchannelCan you try this ? Wrap the
params.text_docs.lengthwith\"Just confirm the raw whole output doesn’t contain any key ? For example
If the raw output just [ float[], float[] ], I think you can try default post-process function
No, that’s a function we added in ml-commons (code link). Which added in 2.12 release. If you are using older version, you can manually copy the escape function to your pre/post process function
You can’t use Apache Commons package