You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Ariel Tubaltsev <tu...@gmail.com> on 2019/03/11 06:41:39 UTC
Transactions stuck after tearing down cluster
I have in-memory cluster of 3 nodes (2.4), replicated mode, transactional
caches.
There is a client sending transactions to the cluster.
1. I bring down all 3 server nodes.
2. Bring all of them back.
3. Client sends some transactions - it's stuck, no visible progress
Logs show that the client is still using an old topology version -
163575049, when servers are using a new one - 5.
Are there any additional steps to take on the client side after reconnect,
waiting period?
DEBUG GridDhtColocatedCache:454 - <cache1> Client topology version mismatch,
need remap lock request [reqTopVer=AffinityTopologyVersion [topVer=5,
minorTopVer=0], locTopVer=AffinityTopologyVersion [topVer=5, minorTopVer=1],
req=GridNearLockRequest [topVer=AffinityTopologyVersion [topVer=5,
minorTopVer=0], miniId=525, dhtVers=[null],
subjId=f00f8691-8dcf-4919-aaff-3c1f25f1b757, taskNameHash=0, createTtl=-1,
accessTtl=-1, flags=3, filter=null, super=GridDistributedLockRequest
[nodeId=f00f8691-8dcf-4919-aaff-3c1f25f1b757, nearXidVer=GridCacheVersion
[topVer=163575049, order=1552095656672, nodeOrder=5], threadId=221,
futId=20fc9106961-a0d19da2-fba4-4805-89dc-ae1195ebb183, timeout=0,
isInTx=true, isInvalidate=false, isRead=false, isolation=SERIALIZABLE,
retVals=[true], txSize=0, flags=0, keysCnt=1,
super=GridDistributedBaseMessage [ver=GridCacheVersion [topVer=163575049,
order=1552095656672, nodeOrder=5], committedVers=null, rolledbackVers=null,
cnt=0, super=GridCacheIdMessage [cacheId=288276891]]]]]
Not sure if it adds anything, there is also bunch of these:
DEBUG GridDhtTxRemote:454 - Invalid transaction state transition
[invalid=PREPARED, cur=PREPARED, tx=GridDhtTxRemote
[nearNodeId=74418337-ff2d-41ea-b93d-0dc371614b68,
rmtFutId=7855bef5961-1f4f7cdf-c7b2-4141-815f-86563df4b23d,
nearXidVer=GridCacheVersion [topVer=163568372, order=1552092449386,
nodeOrder=4], storeWriteThrough=false, super=GridDistributedTxRemoteAdapter
[explicitVers=null, started=true, commitAllowed=0,
txState=IgniteTxRemoteStateImpl [readMap={}, writeMap={IgniteTxKey
[key=com.google.protobuf.ByteString$LiteralByteString [idHash=1280551205,
hash=-847442509, bytes=[101, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0], hash=0],
cacheId=288276891]=IgniteTxEntry
[key=com.google.protobuf.ByteString$LiteralByteString [idHash=1280551205,
hash=-847442509, bytes=[101, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0], hash=0],
cacheId=288276891, txKey=IgniteTxKey
[key=com.google.protobuf.ByteString$LiteralByteString [idHash=1280551205,
hash=-847442509, bytes=[101, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0], hash=0],
cacheId=288276891], val=[op=CREATE,
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Transactions stuck after tearing down cluster
Posted by Ariel Tubaltsev <tu...@gmail.com>.
I see.
Let me get back to you on that - it may take some time employ 2.7.
Thank you
Ariel
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Transactions stuck after tearing down cluster
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
This sounds scary, but I doubt that anyone will investigate 2.4 behavior.
Can you try to get that behavior on 2.7?
Regards,
--
Ilya Kasnacheev
пн, 11 мар. 2019 г. в 09:41, Ariel Tubaltsev <tu...@gmail.com>:
> I have in-memory cluster of 3 nodes (2.4), replicated mode, transactional
> caches.
> There is a client sending transactions to the cluster.
>
> 1. I bring down all 3 server nodes.
> 2. Bring all of them back.
> 3. Client sends some transactions - it's stuck, no visible progress
>
> Logs show that the client is still using an old topology version -
> 163575049, when servers are using a new one - 5.
> Are there any additional steps to take on the client side after reconnect,
> waiting period?
>
> DEBUG GridDhtColocatedCache:454 - <cache1> Client topology version
> mismatch,
> need remap lock request [reqTopVer=AffinityTopologyVersion [topVer=5,
> minorTopVer=0], locTopVer=AffinityTopologyVersion [topVer=5,
> minorTopVer=1],
> req=GridNearLockRequest [topVer=AffinityTopologyVersion [topVer=5,
> minorTopVer=0], miniId=525, dhtVers=[null],
> subjId=f00f8691-8dcf-4919-aaff-3c1f25f1b757, taskNameHash=0, createTtl=-1,
> accessTtl=-1, flags=3, filter=null, super=GridDistributedLockRequest
> [nodeId=f00f8691-8dcf-4919-aaff-3c1f25f1b757, nearXidVer=GridCacheVersion
> [topVer=163575049, order=1552095656672, nodeOrder=5], threadId=221,
> futId=20fc9106961-a0d19da2-fba4-4805-89dc-ae1195ebb183, timeout=0,
> isInTx=true, isInvalidate=false, isRead=false, isolation=SERIALIZABLE,
> retVals=[true], txSize=0, flags=0, keysCnt=1,
> super=GridDistributedBaseMessage [ver=GridCacheVersion [topVer=163575049,
> order=1552095656672, nodeOrder=5], committedVers=null, rolledbackVers=null,
> cnt=0, super=GridCacheIdMessage [cacheId=288276891]]]]]
>
> Not sure if it adds anything, there is also bunch of these:
> DEBUG GridDhtTxRemote:454 - Invalid transaction state transition
> [invalid=PREPARED, cur=PREPARED, tx=GridDhtTxRemote
> [nearNodeId=74418337-ff2d-41ea-b93d-0dc371614b68,
> rmtFutId=7855bef5961-1f4f7cdf-c7b2-4141-815f-86563df4b23d,
> nearXidVer=GridCacheVersion [topVer=163568372, order=1552092449386,
> nodeOrder=4], storeWriteThrough=false, super=GridDistributedTxRemoteAdapter
> [explicitVers=null, started=true, commitAllowed=0,
> txState=IgniteTxRemoteStateImpl [readMap={}, writeMap={IgniteTxKey
> [key=com.google.protobuf.ByteString$LiteralByteString [idHash=1280551205,
> hash=-847442509, bytes=[101, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0], hash=0],
> cacheId=288276891]=IgniteTxEntry
> [key=com.google.protobuf.ByteString$LiteralByteString [idHash=1280551205,
> hash=-847442509, bytes=[101, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0], hash=0],
> cacheId=288276891, txKey=IgniteTxKey
> [key=com.google.protobuf.ByteString$LiteralByteString [idHash=1280551205,
> hash=-847442509, bytes=[101, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0], hash=0],
> cacheId=288276891], val=[op=CREATE,
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>