OpenSearch: [BUG] version number is incompatible with existing ES clients

Describe the bug The API which provides version includes a field version.number. Clients parse this value to determine what version of ElasticSearch is running, and is used for checking API compatibility.

In ElasticSearch 7.10, this value is reported as 7.10.

In OpenSearch 1.0.0, the value is 1.0.0-SNAPSHOT.

Existing clients which parse this string are unable to communicate with OpenSearch when they rely on the value of the version.

To Reproduce Steps to reproduce the behavior:

Start ElasticSearch 7.10
curl http://localhost:9200
Read version.number.
Start OpenSearch
curl http://localhost:9200
Read version.number.

Expected behavior

Because OpenSearch uses the same API as ElasticSearch 7.10.2, the value of version.number should be 7.10.2.

Expected:

  "version" : {
    "number" : "7.10.2",

Actual:

  "version" : {
    "distribution" : "opensearch",
    "number" : "1.0.0-SNAPSHOT",

Plugins N/A

Screenshots N/A

Host/Environment (please complete the following information): N/A

Additional context

We confirmed that this is the cause of #667. It also appears that this may be causing #669.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 49 (35 by maintainers)

Commits related to this issue

Quick code to dynamically use either the old ElasticSearch version or the current OpenSearch version for REST clients. This is to help test different products. See: https://github.com/opensearch-proje... — committed to dlvenable/OpenSearch by dlvenable 3 years ago

Most upvoted comments

@dlvenable raises a great point that this is almost certainly going to be more than a logstash issue: a lot of clients that interact with OpenSearch are going to check the version # to know the wire protocol to use. The proposed solution seems really elegant: keep version.number at 7.10.2 which reflects the API compatibility and lets current clients work without changing, and have a separate openSearchVersion identifier for new clients.

jcgraybill on May 13, 2021

Are you going to release a new RC in July with the fix? My team can’t use filebeat in production

This will be in 1.0 GA, shortly.

dblock on Jul 7, 2021

A cluster setting was introduced in https://github.com/opensearch-project/OpenSearch/pull/847. We removed the distribution in backwards-compatible mode in https://github.com/opensearch-project/OpenSearch/pull/898. Set compatibility.override_main_response_version to put the cluster into a configuration that returns 7.10.2 as with ES OSS.

I am going to close this because the code is done. I added an issue to document it in https://github.com/opensearch-project/documentation-website/issues/72.

dblock on Jul 7, 2021

Reading through this, I’m wondering if the YML update is the only way we want to support setting versions. Having a REST API would be convenient as it would not require cluster restarts. There would probably need to be some way to durably persist the value set in the event the cluster would restart. Alternatively we could just read off the YML as a baseline after restarts. Just a thought.

elfisher on Jun 2, 2021

@dlvenable,

Just chiming in to say thanks, and this worked in our testing. https://github.com/opensearch-project/OpenSearch/issues/706#issuecomment-841582430

Smithx10 on May 15, 2021