You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Semen Boikov (JIRA)" <ji...@apache.org> on 2016/03/28 13:05:25 UTC
[jira] [Comment Edited] (IGNITE-2835) BinaryObjectOffHeapImpl
leaked to public code
[ https://issues.apache.org/jira/browse/IGNITE-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214088#comment-15214088 ]
Semen Boikov edited comment on IGNITE-2835 at 3/28/16 11:04 AM:
----------------------------------------------------------------
Fix looks good, but I think we need better test coverage. Please make sure we have tests for all cache operations which return values:
- get/getAll/getEntries/getAllOutTx
- invoke/invokeAll
- getAndPut
- getAndRemove
- getAndReplace
- getAndPutIfAbsent
- localPeek
For transactional cache need to tests these operations inside explicit transaction (with all tx modes) and with implicit transactions (i.e. just cache.put outside of explicit transaction).
Also please make sure these tests run for both OFFHEAP_TIERED and OFFHEAP_VALUES modes.
was (Author: sboikov):
Fix looks good, but I think we need better test coverage. Please make sure we have tests for all cache operations which return values:
- get/getAll/getEntries/getAllOutTx
- invoke/invokeAll
- getAndPut
- getAndRemove
- getAndReplace
- getAndPutIfAbsent
- localPeek
For transactional cache need to tests these operations inside explicit transaction (with all tx modes) and with implicit transactions (i.e. just cache.put outside of explicit transaction).
> BinaryObjectOffHeapImpl leaked to public code
> ---------------------------------------------
>
> Key: IGNITE-2835
> URL: https://issues.apache.org/jira/browse/IGNITE-2835
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 1.5.0.final
> Reporter: Denis Magda
> Assignee: Artem Shutak
> Priority: Critical
> Labels: community, important
> Fix For: 1.6
>
> Attachments: BinaryObjectOffHeapIssue.java
>
>
> To my knowledge {{BinaryObjectOffHeapImpl}} is considered to be used under some internal lock only to prevent possible offheap pointer movement.
> However seems that we made it available to public code. If to start a partitioned cache in {{OFFHEAP_TIRED}} mode, get {{BinaryObject}} from the cache inside of a TX and put the same object back we will get exception like below
> {noformat}
> [15:00:00,892][WARN ][main][GridNearTxLocal] Set transaction invalidation flag to true due to error [tx=GridNearTxLocal [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMapped=true, needCheckBackup=null, hasRemoteLocks=false, mappings=IgniteTxMappingsImpl [], super=GridDhtTxLocalAdapter [nearOnOriginatingNode=false, nearNodes=[], dhtNodes=[], explicitLock=false, super=IgniteTxLocalAdapter [completedBase=null, sndTransformedVals=false, depEnabled=false, txState=IgniteTxStateImpl [activeCacheIds=GridLongList [idx=1, arr=[-1206548976]], txMap={IgniteTxKey [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976]=IgniteTxEntry [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976, txKey=IgniteTxKey [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976], val=[op=UPDATE, val=SomeType [idHash=1337835760, hash=0, field2=name_0, field1=0]], prevVal=[op=UPDATE, val=SomeType [idHash=1337835760, hash=0, field2=name_0, field1=0]], entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200871, order=1458043167489], filters=[], filtersPassed=false, filtersSet=true, entry=GridDhtColocatedCacheEntry [super=GridDhtCacheEntry [rdrs=[], locPart=GridDhtLocalPartition [id=0, mapPubSize=0, rmvQueue=GridCircularBuffer [sizeMask=255, idxGen=1], cntr=1, state=OWNING, reservations=0, empty=true, createTime=03/15/2016 15:00:00, mapPubSize=0], super=GridDistributedCacheEntry [super=GridCacheMapEntry [key=KeyCacheObjectImpl [val=0, hasValBytes=true], val=null, startVer=1458043167488, ver=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200890, order=1458043167490], hash=-1484017934, extras=GridCacheObsoleteEntryExtras [obsoleteVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200890, order=1458043167491]], flags=7]]]], prepared=false, locked=true, nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, locMapped=false, expiryPlc=null, transferExpiryPlc=false, flags=2, partUpdateCntr=0, serReadVer=null, xidVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487]]}], super=IgniteTxAdapter [xidVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], writeVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200871, order=1458043167489], implicit=false, loc=true, threadId=1, startTime=1458043200850, nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, startVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], endVer=null, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, timeout=0, sysInvalidate=true, sys=false, plc=2, commitVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], finalizing=NONE, preparing=false, invalidParts=null, state=UNKNOWN, timedOut=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], duration=40ms, onePhaseCommit=true], size=1]]], err=class o.a.i.i.transactions.IgniteTxHeuristicCheckedException: Failed to locally write to cache (all transaction entries will be invalidated, however there was a window when entries for this transaction were visible to others): GridNearTxLocal [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMapped=true, needCheckBackup=null, hasRemoteLocks=false, mappings=IgniteTxMappingsImpl [], super=GridDhtTxLocalAdapter [nearOnOriginatingNode=false, nearNodes=[], dhtNodes=[], explicitLock=false, super=IgniteTxLocalAdapter [completedBase=null, sndTransformedVals=false, depEnabled=false, txState=IgniteTxStateImpl [activeCacheIds=GridLongList [idx=1, arr=[-1206548976]], txMap={IgniteTxKey [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976]=IgniteTxEntry [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976, txKey=IgniteTxKey [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976], val=[op=UPDATE, val=SomeType [idHash=1337835760, hash=0, field2=name_0, field1=0]], prevVal=[op=UPDATE, val=SomeType [idHash=1337835760, hash=0, field2=name_0, field1=0]], entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200871, order=1458043167489], filters=[], filtersPassed=false, filtersSet=true, entry=GridDhtColocatedCacheEntry [super=GridDhtCacheEntry [rdrs=[], locPart=GridDhtLocalPartition [id=0, mapPubSize=1, rmvQueue=GridCircularBuffer [sizeMask=255, idxGen=0], cntr=1, state=OWNING, reservations=0, empty=false, createTime=03/15/2016 15:00:00, mapPubSize=1], super=GridDistributedCacheEntry [super=GridCacheMapEntry [key=KeyCacheObjectImpl [val=0, hasValBytes=true], val=null, startVer=1458043167488, ver=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200317, order=1458043162489], hash=-1484017934, extras=GridCacheMvccEntryExtras [mvcc=GridCacheMvcc [locs=[GridCacheMvccCandidate [nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, ver=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], timeout=0, ts=1458043200860, threadId=1, id=1003, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], reentry=null, otherNodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, otherVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], mappedDhtNodes=null, mappedNearNodes=null, ownerVer=null, serOrder=null, key=KeyCacheObjectImpl [val=0, hasValBytes=true], masks=local=1|owner=1|ready=1|reentry=0|used=0|tx=1|single_implicit=0|dht_local=1|near_local=0|removed=0, prevVer=null, nextVer=null]], rmts=null]], flags=6]]]], prepared=false, locked=true, nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, locMapped=false, expiryPlc=null, transferExpiryPlc=false, flags=2, partUpdateCntr=0, serReadVer=null, xidVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487]]}], super=IgniteTxAdapter [xidVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], writeVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200871, order=1458043167489], implicit=false, loc=true, threadId=1, startTime=1458043200850, nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, startVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], endVer=null, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, timeout=0, sysInvalidate=false, sys=false, plc=2, commitVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], finalizing=NONE, preparing=false, invalidParts=null, state=COMMITTING, timedOut=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], duration=20ms, onePhaseCommit=true], size=1]]]]
> Exception in thread "main" class org.apache.ignite.transactions.TransactionHeuristicException: Failed to locally write to cache (all transaction entries will be invalidated, however there was a window when entries for this transaction were visible to others): GridNearTxLocal [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMapped=true, needCheckBackup=null, hasRemoteLocks=false, mappings=IgniteTxMappingsImpl [], super=GridDhtTxLocalAdapter [nearOnOriginatingNode=false, nearNodes=[], dhtNodes=[], explicitLock=false, super=IgniteTxLocalAdapter [completedBase=null, sndTransformedVals=false, depEnabled=false, txState=IgniteTxStateImpl [activeCacheIds=GridLongList [idx=1, arr=[-1206548976]], txMap={IgniteTxKey [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976]=IgniteTxEntry [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976, txKey=IgniteTxKey [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976], val=[op=UPDATE, val=SomeType [idHash=1337835760, hash=0, field2=name_0, field1=0]], prevVal=[op=UPDATE, val=SomeType [idHash=1337835760, hash=0, field2=name_0, field1=0]], entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200871, order=1458043167489], filters=[], filtersPassed=false, filtersSet=true, entry=GridDhtColocatedCacheEntry [super=GridDhtCacheEntry [rdrs=[], locPart=GridDhtLocalPartition [id=0, mapPubSize=1, rmvQueue=GridCircularBuffer [sizeMask=255, idxGen=0], cntr=1, state=OWNING, reservations=0, empty=false, createTime=03/15/2016 15:00:00, mapPubSize=1], super=GridDistributedCacheEntry [super=GridCacheMapEntry [key=KeyCacheObjectImpl [val=0, hasValBytes=true], val=null, startVer=1458043167488, ver=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200317, order=1458043162489], hash=-1484017934, extras=GridCacheMvccEntryExtras [mvcc=GridCacheMvcc [locs=[GridCacheMvccCandidate [nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, ver=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], timeout=0, ts=1458043200860, threadId=1, id=1003, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], reentry=null, otherNodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, otherVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], mappedDhtNodes=null, mappedNearNodes=null, ownerVer=null, serOrder=null, key=KeyCacheObjectImpl [val=0, hasValBytes=true], masks=local=1|owner=1|ready=1|reentry=0|used=0|tx=1|single_implicit=0|dht_local=1|near_local=0|removed=0, prevVer=null, nextVer=null]], rmts=null]], flags=6]]]], prepared=false, locked=true, nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, locMapped=false, expiryPlc=null, transferExpiryPlc=false, flags=2, partUpdateCntr=0, serReadVer=null, xidVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487]]}], super=IgniteTxAdapter [xidVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], writeVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200871, order=1458043167489], implicit=false, loc=true, threadId=1, startTime=1458043200850, nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, startVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], endVer=null, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, timeout=0, sysInvalidate=false, sys=false, plc=2, commitVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], finalizing=NONE, preparing=false, invalidParts=null, state=COMMITTING, timedOut=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], duration=20ms, onePhaseCommit=true], size=1]]]
> at org.apache.ignite.internal.util.IgniteUtils$11.apply(IgniteUtils.java:785)
> at org.apache.ignite.internal.util.IgniteUtils$11.apply(IgniteUtils.java:783)
> at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:877)
> at org.apache.ignite.internal.processors.cache.transactions.TransactionProxyImpl.commit(TransactionProxyImpl.java:264)
> at org.apache.ignite.examples.sber.BinaryObjectOffHeapIssue.main(BinaryObjectOffHeapIssue.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
> Caused by: class org.apache.ignite.internal.transactions.IgniteTxHeuristicCheckedException: Failed to locally write to cache (all transaction entries will be invalidated, however there was a window when entries for this transaction were visible to others): GridNearTxLocal [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMapped=true, needCheckBackup=null, hasRemoteLocks=false, mappings=IgniteTxMappingsImpl [], super=GridDhtTxLocalAdapter [nearOnOriginatingNode=false, nearNodes=[], dhtNodes=[], explicitLock=false, super=IgniteTxLocalAdapter [completedBase=null, sndTransformedVals=false, depEnabled=false, txState=IgniteTxStateImpl [activeCacheIds=GridLongList [idx=1, arr=[-1206548976]], txMap={IgniteTxKey [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976]=IgniteTxEntry [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976, txKey=IgniteTxKey [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976], val=[op=UPDATE, val=SomeType [idHash=1337835760, hash=0, field2=name_0, field1=0]], prevVal=[op=UPDATE, val=SomeType [idHash=1337835760, hash=0, field2=name_0, field1=0]], entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200871, order=1458043167489], filters=[], filtersPassed=false, filtersSet=true, entry=GridDhtColocatedCacheEntry [super=GridDhtCacheEntry [rdrs=[], locPart=GridDhtLocalPartition [id=0, mapPubSize=1, rmvQueue=GridCircularBuffer [sizeMask=255, idxGen=0], cntr=1, state=OWNING, reservations=0, empty=false, createTime=03/15/2016 15:00:00, mapPubSize=1], super=GridDistributedCacheEntry [super=GridCacheMapEntry [key=KeyCacheObjectImpl [val=0, hasValBytes=true], val=null, startVer=1458043167488, ver=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200317, order=1458043162489], hash=-1484017934, extras=GridCacheMvccEntryExtras [mvcc=GridCacheMvcc [locs=[GridCacheMvccCandidate [nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, ver=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], timeout=0, ts=1458043200860, threadId=1, id=1003, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], reentry=null, otherNodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, otherVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], mappedDhtNodes=null, mappedNearNodes=null, ownerVer=null, serOrder=null, key=KeyCacheObjectImpl [val=0, hasValBytes=true], masks=local=1|owner=1|ready=1|reentry=0|used=0|tx=1|single_implicit=0|dht_local=1|near_local=0|removed=0, prevVer=null, nextVer=null]], rmts=null]], flags=6]]]], prepared=false, locked=true, nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, locMapped=false, expiryPlc=null, transferExpiryPlc=false, flags=2, partUpdateCntr=0, serReadVer=null, xidVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487]]}], super=IgniteTxAdapter [xidVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], writeVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200871, order=1458043167489], implicit=false, loc=true, threadId=1, startTime=1458043200850, nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, startVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], endVer=null, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, timeout=0, sysInvalidate=false, sys=false, plc=2, commitVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, order=1458043167487], finalizing=NONE, preparing=false, invalidParts=null, state=COMMITTING, timedOut=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], duration=20ms, onePhaseCommit=true], size=1]]]
> at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:1162)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.finish(GridNearTxLocal.java:738)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:331)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$4.apply(GridNearTxLocal.java:838)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$4.apply(GridNearTxLocal.java:830)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:262)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:225)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commitAsync(GridNearTxLocal.java:830)
> at org.apache.ignite.internal.processors.cache.GridCacheAdapter.commitTxAsync(GridCacheAdapter.java:4220)
> at org.apache.ignite.internal.processors.cache.GridCacheSharedContext.commitTxAsync(GridCacheSharedContext.java:619)
> at org.apache.ignite.internal.processors.cache.transactions.TransactionProxyImpl.commit(TransactionProxyImpl.java:256)
> ... 6 more
> Caused by: java.lang.UnsupportedOperationException
> at org.apache.ignite.internal.binary.BinaryObjectOffheapImpl.prepareForCache(BinaryObjectOffheapImpl.java:358)
> at org.apache.ignite.internal.processors.cacheobject.IgniteCacheObjectProcessorImpl.prepareForCache(IgniteCacheObjectProcessorImpl.java:98)
> at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerSet(GridCacheMapEntry.java:1182)
> at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:1004)
> ... 16 more
> {noformat}
> The test is attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)