elasticsearch-net: Bulk-API | Invalid NEST response built from a successful (200) low level call

NEST/Elasticsearch.Net version: 7.17.0

Elasticsearch version: 8.0.1

.NET runtime version: 6.0.201

Operating system version: Ubuntu 18.04.

Description of the problem including expected versus actual behavior: Successful Bulk-API Request produces invalid NEST Response. See Debug Information.

The property IsValid is false, but there are no errors in the response. Maybe it’s because of the missing _type Property in Elastic v8?

When I execute the query (see Debug-Information) directly in Elasticsearch, no error occurs. My data is in both cases (NEST-Client and direct via Elasticsearch) correctly modified.

Expected behavior Request response should not be invalid.

Provide DebugInformation (if relevant):

C# (also at similar Bulk-API requests with IndexMany and DeleteMany functions -> same error) var updateResponse = await client.BulkAsync(bd => bd .Index(ElasticIndex.MyIndex) .UpdateMany(myData, (descriptor, v) => descriptor .Doc(v)) );

NEST-Response Invalid NEST response built from a successful (200) low level call on POST: /my-index/_bulk Invalid Bulk items: operation[0]: update returned 200 _index: my-index _type: _id: 9f429b8b-dec2-4da1-83b7-e411782920f3 _version: 2 error: Audit trail of this API call: [1] HealthyResponse: Node: http://myserver:9200/ Took: 00:00:00.1292754 Request: {"update":{"_id":"9f429b8b-dec2-4da1-83b7-e411782920f3"}} ...doc information... Response: {"took":6,"errors":false,"items":[{"update":{"_index":"my-index","_id":"9f429b8b-dec2-4da1-83b7-e411782920f3","_version":2,"result":"updated","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":1459,"_primary_term":3,"status":200}}]}

C# - My Connection settings var pool = new SingleNodeConnectionPool(new Uri(_appSettings.Elasticsearch.Url)); var settings = new ConnectionSettings(pool, (builtin, settings) => new JsonNetSerializer(builtin, settings, () => new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All } )); settings.BasicAuthentication(_appSettings.Elasticsearch.Username, _appSettings.Elasticsearch.Password); settings.DisableDirectStreaming(); settings.EnableApiVersioningHeader();

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 3
Comments: 41 (17 by maintainers)

Most upvoted comments

Thanks for checking that @neox2811 and @julbra. I’ll need to investigate this and see if it’s something we can accommodate in the interim before our new client is ready for GA. We do have bulk support in the new client which can be installed side-by-side with NEST. It is however an alpha release and not feature complete.

stevejgordon on Mar 15, 2022

@jeaguilarc As a starting point, please verify that you’ve enabled the compatibility mode on the client settings with the call to EnableApiVersioningHeader.

var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));

var settings = new ConnectionSettings(pool)
    .CertificateFingerprint("FINGERPRINT")
    .BasicAuthentication("elastic", "password")
    .EnableApiVersioningHeader();

var client = new ElasticClient(settings);

If that’s enabled, I’d recommend enabling DebugMode as well and verifying the request and response JSON in DebugInformation on the response. If you can share the response, that would be useful. When using Postman are you including the API versioning required on the Accept header?

stevejgordon on Apr 1, 2022

ElasticSearch 8.1.2 has been released, but I see that the same problem continues, any idea when this issue will be resolved?

jeaguilarc on Apr 1, 2022

did a quick test…and it works! thank you

i did have have to run this with “.EnableApiVersioningHeader()”

mtbitcoin on Apr 11, 2022

@jeaguilarc I just tested a bulk request with 8.1.2 and it appears resolved. I’m seeing _type on the response items.

With Server 8.1.2, BulkAsync inserts are now working correctly: IsValid is true. 🥳

julbra on Apr 1, 2022

@yansklyarenko There are no performance implications in either case, as this simply adds an extra header to indicate the version being used so that it can accept requests and send responses in the 7.x form. It’s not necessary to enable it unless communicating from a 7.17 client to an 8.x server.

stevejgordon on Aug 31, 2023

Thanks for confirming @mtbitcoin. Apologies for any blocks this issue introduced for you and your team. Yes, EnableApiVersioningHeader is required for correct communication with a v8 server as it ensures the server returns v7 compatible responses that the client can deserialise and validate.

stevejgordon on Apr 11, 2022

@SzymonKatra Thanks for raising the new issue. I have investigated this and identified the most likely cause of various issues when using API versioning for compatibility on .NET Framework. I’m testing the fix currently and will release a patch ASAP.

stevejgordon on Apr 11, 2022

@stevejgordon I am having the same issue as @jeaguilarc and @mtbitcoin, but only on .NET Framework. Opened new issue - #6260

SzymonKatra on Apr 7, 2022

Ok, I’ll open another topic with this. Thank you very much.

jeaguilarc on Apr 1, 2022

@stevejgordon the elasticclient x.IsValid is still returning a false (instead of true)

mtbitcoin on Apr 1, 2022

Hi everyone. I wanted to provide an update. This is indeed confirmed as being caused by our check for the type on the response items. We are discussing this with the server team to identify if this can be addressed with the compatibility feature.

stevejgordon on Mar 28, 2022

Hi, @neox2811.

Could you please try enabling our compatibility mode as described in this document to see if that changes the response?

stevejgordon on Mar 14, 2022