orientdb: Server deadlock with multiple concurrent binary clients

Using version 2.1.12, I have multiple concurrent threads, each using its own graph instance, reading, creating and updating records in the DB. After about 20.000 cycles, all clients hang. I debugged the server and noticed that_ all _binary clients threads (at the server) where stuck at:

"OrientDB <- BinaryClient (/10.1.100.22:61097)"
    at com.orientechnologies.common.concur.lock.OReadersWriterSpinLock.acquireWriteLock(OReadersWriterSpinLock.java:132)
    at com.orientechnologies.common.concur.lock.OModificationLock.prohibitModifications(OModificationLock.java:96)
    at com.orientechnologies.common.concur.lock.OModificationLock.prohibitModifications(OModificationLock.java:84)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.synch(OAbstractPaginatedStorage.java:1276)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.checkLowDiskSpaceAndFullCheckpointRequests(OAbstractPaginatedStorage.java:2894)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:1150)
    at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:606)
    at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:156)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2679)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2648)
    at com.tinkerpop.blueprints.impls.orient.OrientTransactionalGraph.commit(OrientTransactionalGraph.java:161)
    at com.orientechnologies.orient.graph.sql.OGraphCommandExecutorSQLFactory.runInConfiguredTxMode(OGraphCommandExecutorSQLFactory.java:262)
    at com.orientechnologies.orient.graph.sql.OCommandExecutorSQLCreateEdge.execute(OCommandExecutorSQLCreateEdge.java:153)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:90)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:1530)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:1511)
    at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:67)
    at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.command(ONetworkProtocolBinary.java:1323)
    at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:400)
    at com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:223)
    at com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:77)

I tried to reduce the transaction sizes to an absolute minimum without luck. In this situation I even cannot shutdown the server cleanly but have to do a kill -9 and restart to re-enable binary clients to connect to the server.

This might relate to #4613

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Comments: 38 (19 by maintainers)

Commits related to this issue

Most upvoted comments

Gentlemen, 2.1.x works with more than 14.000.000 edges and about 200.000 iterations over the above-mentioned code very well. Beforehand, the error occurred after about 20.000 iterations at the latest.

I think it is fixed. Thank you for your efforts!!

@gerdhub we found reason of issue and started to fix it. Thank you very much for bug report , your help and patience.

@gerdhub @TeamXceleratorDev @ygoraly fixed. could you try ?

Quick question: do you have a composite index on edges?