orientdb: Error on creating cluster on distributed nodes: ids are different
Am using Orient DB 2.1.0 and running in clustered environment. I tried with 2.0.13, same issue. Am trying to setup 3 Orient DB servers and create database in one of them
hazelcast.xml looks like this:
<hazelcast xmlns="http://www.hazelcast.com/schema/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.h
azelcast.com/schema/config hazelcast-config-3.0.xsd">
<group>
<name>orient-db</name>
<password>cluster-db</password>
</group>
<network>
<port auto-increment="false">2434</port>
<join>
<multicast enabled="false">
<multicast-group>235.1.1.1</multicast-group>
<multicast-port>2434</multicast-port>
</multicast>
<tcp-ip enabled="true">
<!-- Host names of 3 OrientDB servers and port number -->
<member>orientdb1-prod-dev-brn:2434</member>
<member>orientdb2-prod-dev-brn:2434</member>
<member>orientdb3-prod-dev-brn:2434</member>
</tcp-ip>
</join>
<symmetric-encryption enabled="true">
<algorithm>PBEWithMD5AndDES</algorithm>
<!-- salt value to use when generating the secret key -->
<salt>orientdb-cluster-salt</salt>
<!-- pass phrase to use when generating the secret key -->
<password>orientdb-cluster-pp</password>
<!-- iteration count to use when generating the secret key -->
<iteration-count>19</iteration-count>
</symmetric-encryption>
</network>
<executor-service>
<pool-size>16</pool-size>
</executor-service>
</hazelcast>
Handler OHazelcastPlugin in orientdb-server-config.xml looks like this
<!-- CLUSTER PLUGIN, TO TURN ON SET THE 'ENABLED' PARAMETER TO 'true' -->
<handler class="com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin">
<parameters>
<parameter name="nodeName" value="orientdb3-prod-dev-brn" />
<parameter name="enabled" value="true"/>
<parameter name="configuration.db.default"
value="${ORIENTDB_HOME}/config/default-distributed-db-config.json"/>
<parameter name="configuration.hazelcast" value="${ORIENTDB_HOME}/config/hazelcast.xml"/>
</parameters>
</handler>
default-distributed-db-config.json looks like this
{
"autoDeploy": true,
"hotAlignment": false,
"executionMode": "asynchronous",
"readQuorum": 1,
"writeQuorum": 2,
"failureAvailableNodesLessQuorum": false,
"readYourWrites": true,
"servers": {
"*": "master"
},
"clusters": {
"internal": {},
"index": {},
"*": {
"servers": [
"<NEW_NODE>"
]
}
}
}
ERROR in logs
2015-08-10 21:03:58:208 INFO [product-orientdb3-prod-dev-brn]--[product-orientdb2-prod-dev-brn] update configuration db=product [OHazelcastPlugin]
2015-08-10 21:04:06:407 INFO [product-orientdb3-prod-dev-brn] class 'someEdgeFrom', creation of new local cluster 'someedgefrom_product-orientdb3-prod-dev-brn' (id=-1) [OHazelcastPlugin]
2015-08-10 21:04:09:992 WARNI Error on creating cluster on distributed nodes: ids are different (local=114 and remote=170). Retrying 0/10... [ODistributedStorage]
2015-08-10 21:04:16:949 WARNI Error on creating cluster on distributed nodes: ids are different (local=114 and remote=170). Retrying 1/10... [ODistributedStorage]
2015-08-10 21:04:20:619 WARNI Error on creating cluster on distributed nodes: ids are different (local=114 and remote=170). Retrying 2/10... [ODistributedStorage]
2015-08-10 21:04:22:369 WARNI Error on creating cluster on distributed nodes: ids are different (local=114 and remote=170). Retrying 3/10... [ODistributedStorage]
2015-08-10 21:04:24:767 WARNI Error on creating cluster on distributed nodes: ids are different (local=114 and remote=170). Retrying 4/10... [ODistributedStorage]
2015-08-10 21:04:26:191 WARNI Error on creating cluster on distributed nodes: ids are different (local=114 and remote=170). Retrying 5/10... [ODistributedStorage]
2015-08-10 21:04:27:593 WARNI Error on creating cluster on distributed nodes: ids are different (local=114 and remote=170). Retrying 6/10... [ODistributedStorage]
2015-08-10 21:04:28:956 WARNI Error on creating cluster on distributed nodes: ids are different (local=114 and remote=170). Retrying 7/10... [ODistributedStorage]
2015-08-10 21:04:30:331 WARNI Error on creating cluster on distributed nodes: ids are different (local=114 and remote=170). Retrying 8/10... [ODistributedStorage]
2015-08-10 21:04:31:699 WARNI Error on creating cluster on distributed nodes: ids are different (local=114 and remote=170). Retrying 9/10... [ODistributedStorage]
2015-08-10 21:04:32:799 SEVER [product-orientdb3-prod-dev-brn] error on creating cluster 'someedgefrom_product-orientdb3-prod-dev-brn' in class 'someEdgeFrom': [OHazelcastPlugin]Exception in thread "main" com.orientechnologies.orient.server.distributed.ODistributedException: com.orientechnologies.orient.server.distributed.ODistributedException: Error on creating cluster 'someedgefrom_product-orientdb3-prod-dev-brn' in class 'someEdgeFrom'
at com.orientechnologies.orient.server.hazelcast.OHazelcastDistributedDatabase.configureDatabase(OHazelcastDistributedDatabase.java:224)
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.installDatabaseFromNetwork(OHazelcastPlugin.java:1055)
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.installDatabase(OHazelcastPlugin.java:937)
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.installNewDatabases(OHazelcastPlugin.java:1350)
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.startup(OHazelcastPlugin.java:184)
at com.orientechnologies.orient.server.OServer.registerPlugins(OServer.java:971)
at com.orientechnologies.orient.server.OServer.activate(OServer.java:338)
at com.orientechnologies.orient.server.OServerMain.main(OServerMain.java:41)
Caused by: com.orientechnologies.orient.server.distributed.ODistributedException: Error on creating cluster 'someedgefrom_product-orientdb3-prod-dev-brn' in class 'someEdgeFrom'
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.installLocalClusterPerClass(OHazelcastPlugin.java:1494)
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.installDbClustersForLocalNode(OHazelcastPlugin.java:1207)
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin$1.call(OHazelcastPlugin.java:1058)
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin$1.call(OHazelcastPlugin.java:1055)
at com.orientechnologies.orient.server.hazelcast.OHazelcastDistributedDatabase.configureDatabase(OHazelcastDistributedDatabase.java:222)
... 7 more
Caused by: com.orientechnologies.orient.server.distributed.ODistributedException: Error on creating cluster on distributed nodes: local and remote ids assigned are different
at com.orientechnologies.orient.server.distributed.ODistributedStorage.addCluster(ODistributedStorage.java:1268)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.addCluster(ODatabaseDocumentTx.java:1253)
at com.orientechnologies.orient.core.metadata.schema.OClassImpl.createClusterIfNeeded(OClassImpl.java:2124)
at com.orientechnologies.orient.core.metadata.schema.OClassImpl.addCluster(OClassImpl.java:1105)
at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.installLocalClusterPerClass(OHazelcastPlugin.java:1487)
... 11 more
About this issue
- Original URL
- State: closed
- Created 9 years ago
- Comments: 34 (13 by maintainers)
Commits related to this issue
- Fixed issue #4767 — committed to orientechnologies/orientdb by lvca 9 years ago
- Fixed issue #4767 — committed to orientechnologies/orientdb by lvca 9 years ago
+1 The same issue as others.
I have upgraded my docker image to 2.1.5 version and running it in distributed mode directly from docker-compose file. I can put some startup script to create databases before running it in distributed mode but I don’t think that’s a good option for production.
Do you guys think you can fix this quickly?