You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by rick_tem <rv...@temenos.com> on 2017/04/06 09:05:53 UTC

IgniteCacheProxy connection failure in REPLICATAED mode

Hi,

I have a cache configuration that looks like this:

				<bean class="org.apache.ignite.configuration.CacheConfiguration">
					<property name="name" value="MCache" />
					<property name="cacheMode" value="REPLICATED" />
					<property name="memoryMode" value="ONHEAP_TIERED" />
					
					<property name="offHeapMaxMemory" value="#{10 * 1024L * 1024L * 1024L}"
/>
					<property name="evictionPolicy">
						
						<bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
							
							<property name="maxSize" value="1000000" />
						</bean>
					</property>
					<property name="atomicityMode" value="TRANSACTIONAL" />
					<property name="backups" value="1" />
					<property name="writeSynchronizationMode" value="FULL_SYNC" />
					<property name="statisticsEnabled" value="true" />
				</bean>

I've started two Ignite servers and within JBOSS application server set
Ignition.setClientMode(true); such that I have 2 servers and 1 client
(JBOSS) shown in my topology.  When I then terminate one of the Ignite
servers, I receive the following exception below when my JBOSS app is
manipulating the cache.  If I am in REPLICATED mode, why doesn't the Ignite
proxy try the other server automatically on connection failure?  Is there a
mechanism I should use to protect against connection failure, or how should
I ensure (in this case a remove operation on the cache) my actions complete
successfully?

Thanks!
Rick

Caused by: org.apache.ignite.IgniteClientDisconnectedException: Client node
disconnected: TGrid
	... 88 more
[ERROR] 2017-04-05 14:11:35,675 [tSA 4_1037028145] LOCKING - IgniteCache
Error 
javax.cache.CacheException: class
org.apache.ignite.IgniteClientDisconnectedException: Client node
disconnected: TGrid
	at
org.apache.ignite.internal.processors.cache.GridCacheGateway.checkState(GridCacheGateway.java:92)
~[ignite-core-1.9.0.jar!/:1.9.0]
	at
org.apache.ignite.internal.processors.cache.GridCacheGateway.enter(GridCacheGateway.java:173)
~[ignite-core-1.9.0.jar!/:1.9.0]
	at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.onEnter(IgniteCacheProxy.java:2264)
~[ignite-core-1.9.0.jar!/:1.9.0]
	at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.remove(IgniteCacheProxy.java:1463)
~[ignite-core-1.9.0.jar!/:1.9.0]




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteCacheProxy-connection-failure-in-REPLICATAED-mode-tp11769.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IgniteCacheProxy connection failure in REPLICATAED mode

Posted by rick_tem <rv...@temenos.com>.
Thanks Andrey.  I'll see how it behaves when 2.0 comes out in a few weeks.

Thanks again and kind regards,
Rick




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteCacheProxy-connection-failure-in-REPLICATAED-mode-tp11769p12235.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IgniteCacheProxy connection failure in REPLICATAED mode

Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Rick,

It 100% reproduces on ignite-1.9, but ignite-2.0 looks fine.
Looks like put and remove can fails regardless of cache modes. PFA the
reproducer I've made.

Seems, it can failed when node tries to register key class in marshaller
cache while other node leaves topology.
System marshaller cache was removed in Ignite-2.0, that is why issue is not
reproducable on 2.0-snapshot.

The only workaround is to get a value for some key in cache once at startup
before any update operation will be performed. And repeat this for every
key class that will be used.


On Mon, Apr 24, 2017 at 2:22 PM, rick_tem <rv...@temenos.com> wrote:

> Hi Andrey,
>
> Yes, it is directly after I stop the Ignite node.  The output is from the
> Jboss console (where the other Ignite server node is running in the same
> JVM) and is the full stack trace before our proprietary code calling
> remove().  We are not calling any transaction within our proprietary code,
> simply cache.remove(key).  Would it help if I try to reproduce with a test
> case?  Should I try changing anything with the below config?
>
> Thanks,
> Rick
>
>                                 <bean class="org.apache.ignite.
> configuration.CacheConfiguration">
>                                         <property name="name"
> value="MyCache" />
>                                         <property name="cacheMode"
> value="REPLICATED" />
>                                         <property name="memoryMode"
> value="ONHEAP_TIERED" />
>
>                                         <property name="offHeapMaxMemory"
> value="#{10 * 1024L * 1024L * 1024L}"
> />
>                                         <property name="evictionPolicy">
>
>                                                 <bean
> class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
>
>                                                         <property
> name="maxSize" value="1000000" />
>                                                 </bean>
>                                         </property>
>                                         <property name="atomicityMode"
> value="TRANSACTIONAL" />
>                                         <property name="backups" value="1"
> />
>                                         <property name="writeSynchronizationMode"
> value="FULL_SYNC" />
>                                         <property name="statisticsEnabled"
> value="true" />
>                                 </bean>
>                         </list>
>                 </property>
>                 <property name="transactionConfiguration">
>                         <bean class="org.apache.ignite.configuration.
> TransactionConfiguration">
>
>                         </bean>
>                 </property>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/IgniteCacheProxy-connection-
> failure-in-REPLICATAED-mode-tp11769p12191.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Best regards,
Andrey V. Mashenkov

Re: IgniteCacheProxy connection failure in REPLICATAED mode

Posted by rick_tem <rv...@temenos.com>.
Hi Andrey,

Yes, it is directly after I stop the Ignite node.  The output is from the
Jboss console (where the other Ignite server node is running in the same
JVM) and is the full stack trace before our proprietary code calling
remove().  We are not calling any transaction within our proprietary code,
simply cache.remove(key).  Would it help if I try to reproduce with a test
case?  Should I try changing anything with the below config?

Thanks,
Rick

				<bean class="org.apache.ignite.configuration.CacheConfiguration">
					<property name="name" value="MyCache" />
					<property name="cacheMode" value="REPLICATED" />
					<property name="memoryMode" value="ONHEAP_TIERED" />
					
					<property name="offHeapMaxMemory" value="#{10 * 1024L * 1024L * 1024L}"
/>
					<property name="evictionPolicy">
						
						<bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
							
							<property name="maxSize" value="1000000" />
						</bean>
					</property>
					<property name="atomicityMode" value="TRANSACTIONAL" />
					<property name="backups" value="1" />
					<property name="writeSynchronizationMode" value="FULL_SYNC" />
					<property name="statisticsEnabled" value="true" />
				</bean>
			</list>
		</property>
		<property name="transactionConfiguration">
			<bean class="org.apache.ignite.configuration.TransactionConfiguration">
				
			</bean>
		</property>




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteCacheProxy-connection-failure-in-REPLICATAED-mode-tp11769p12191.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IgniteCacheProxy connection failure in REPLICATAED mode

Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Rick,


Is it full stacktrace? Looks like transaction commit was failed when
primary node has left the grid.
Looks weird. Could you check this transaction was successfully restarted
and commited at last?


On Fri, Apr 21, 2017 at 6:21 PM, rick_tem <rv...@temenos.com> wrote:

> Hey Andre,
>
> Here's what the stack trace looks like inside jboss.
>
> Topology snapshot [ver=46, servers=1, clients=2, CPUs=8, heap=7.1GB]
>
> 2017-04-20 11:23:41,575 ERROR [stderr] (tSA 6_1968933222) [11:23:41] (err)
> *Failed to execute compound future reducer: GridNearTxFinishFuture*
> [futId=bcd3eba8b51-55bc7836-f114-41d7-b0d0-44a14beb441f,
> tx=GridNearTxLocal
> [mappings=IgniteTxMappingsSingleImpl [mapping=GridDistributedTxMapping
> [entries=[IgniteTxEntry [key=KeyCacheObjectImpl
> [val=FBNK_STATIC_CHANGE_000*360494.0008100.01, hasValBytes=true],
> cacheId=-1750629716, partId=-1, txKey=IgniteTxKey [key=KeyCacheObjectImpl
> [val=FBNK_STATIC_CHANGE_000*360494.0008100.01, hasValBytes=true],
> cacheId=-1750629716], val=[op=DELETE, val=null], prevVal=[op=NOOP,
> val=null], oldVal=[op=NOOP, val=null], entryProcessorsCol=null, ttl=-1,
> conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=null,
> filters=[], filtersPassed=false, filtersSet=true,
> entry=GridDhtDetachedCacheEntry [super=GridDistributedCacheEntry
> [super=GridCacheMapEntry [key=KeyCacheObjectImpl
> [val=FBNK_STATIC_CHANGE_000*360494.0008100.01, hasValBytes=true],
> val=null,
> startVer=1492681260390, ver=GridCacheVersion [topVer=104082481,
> time=1492680221503, order=1492681260390, nodeOrder=44], hash=-852679692,
> extras=null, flags=0]]], prepared=0, locked=false,
> nodeId=b992bdfd-1f6e-45d0-ac6e-de4c1355eb34, locMapped=false,
> expiryPlc=null, transferExpiryPlc=false, flags=0, partUpdateCntr=0,
> serReadVer=null, xidVer=GridCacheVersion [topVer=104082481,
> time=1492680221502, order=1492681260388, nodeOrder=44]]],
> explicitLock=false, dhtVer=null, last=false, near=false, clientFirst=false,
> node=b992bdfd-1f6e-45d0-ac6e-de4c1355eb34]], nearLocallyMapped=false,
> colocatedLocallyMapped=false, needCheckBackup=false, hasRemoteLocks=false,
> thread=tSA 6_1968933222, mappings=IgniteTxMappingsSingleImpl
> [mapping=GridDistributedTxMapping [entries=[IgniteTxEntry
> [key=KeyCacheObjectImpl [val=FBNK_STATIC_CHANGE_000*360494.0008100.01,
> hasValBytes=true], cacheId=-1750629716, partId=-1, txKey=IgniteTxKey
> [key=KeyCacheObjectImpl [val=FBNK_STATIC_CHANGE_000*360494.0008100.01,
> hasValBytes=true], cacheId=-1750629716], val=[op=DELETE, val=null],
> prevVal=[op=NOOP, val=null], oldVal=[op=NOOP, val=null],
> entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null,
> explicitVer=null, dhtVer=null, filters=[], filtersPassed=false,
> filtersSet=true, entry=GridDhtDetachedCacheEntry
> [super=GridDistributedCacheEntry [super=GridCacheMapEntry
> [key=KeyCacheObjectImpl [val=FBNK_STATIC_CHANGE_000*360494.0008100.01,
> hasValBytes=true], val=null, startVer=1492681260390, ver=GridCacheVersion
> [topVer=104082481, time=1492680221503, order=1492681260390, nodeOrder=44],
> hash=-852679692, extras=null, flags=0]]], prepared=0, locked=false,
> nodeId=b992bdfd-1f6e-45d0-ac6e-de4c1355eb34, locMapped=false,
> expiryPlc=null, transferExpiryPlc=false, flags=0, partUpdateCntr=0,
> serReadVer=null, xidVer=GridCacheVersion [topVer=104082481,
> time=1492680221502, order=1492681260388, nodeOrder=44]]],
> explicitLock=false, dhtVer=null, last=false, near=false, clientFirst=false,
> node=b992bdfd-1f6e-45d0-ac6e-de4c1355eb34]], super=GridDhtTxLocalAdapter
> [nearOnOriginatingNode=false, nearNodes=[], dhtNodes=[],
> explicitLock=false,
> super=IgniteTxLocalAdapter [completedBase=null, sndTransformedVals=false,
> depEnabled=false, txState=IgniteTxImplicitSingleStateImpl [init=true],
> super=IgniteTxAdapter [xidVer=GridCacheVersion [topVer=104082481,
> time=1492680221502, order=1492681260388, nodeOrder=44], writeVer=null,
> implicit=true, loc=true, threadId=391, startTime=1492680221473,
> nodeId=9e9758d1-8dbf-44e2-8181-6f9b454cf61c, startVer=GridCacheVersion
> [topVer=104082481, time=1492680221502, order=1492681260388, nodeOrder=44],
> endVer=null, isolation=READ_COMMITTED, concurrency=OPTIMISTIC, timeout=0,
> sysInvalidate=false, sys=false, plc=2, commitVer=GridCacheVersion
> [topVer=104082481, time=1492680221502, order=1492681260388, nodeOrder=44],
> finalizing=NONE, preparing=false, invalidParts=null, state=PREPARED,
> timedOut=false, topVer=AffinityTopologyVersion [topVer=45, minorTopVer=0],
> duration=95ms, onePhaseCommit=true], size=1]]], commit=true,
> mappings=IgniteTxMappingsSingleImpl [mapping=GridDistributedTxMapping
> [entries=[IgniteTxEntry [key=KeyCacheObjectImpl
> [val=FBNK_STATIC_CHANGE_000*360494.0008100.01, hasValBytes=true],
> cacheId=-1750629716, partId=-1, txKey=IgniteTxKey [key=KeyCacheObjectImpl
> [val=FBNK_STATIC_CHANGE_000*360494.0008100.01, hasValBytes=true],
> cacheId=-1750629716], val=[op=DELETE, val=null], prevVal=[op=NOOP,
> val=null], oldVal=[op=NOOP, val=null], entryProcessorsCol=null, ttl=-1,
> conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=null,
> filters=[], filtersPassed=false, filtersSet=true,
> entry=GridDhtDetachedCacheEntry [super=GridDistributedCacheEntry
> [super=GridCacheMapEntry [key=KeyCacheObjectImpl
> [val=FBNK_STATIC_CHANGE_000*360494.0008100.01, hasValBytes=true],
> val=null,
> startVer=1492681260390, ver=GridCacheVersion [topVer=104082481,
> time=1492680221503, order=1492681260390, nodeOrder=44], hash=-852679692,
> extras=null, flags=0]]], prepared=0, locked=false,
> nodeId=b992bdfd-1f6e-45d0-ac6e-de4c1355eb34, locMapped=false,
> expiryPlc=null, transferExpiryPlc=false, flags=0, partUpdateCntr=0,
> serReadVer=null, xidVer=GridCacheVersion [topVer=104082481,
> time=1492680221502, order=1492681260388, nodeOrder=44]]],
> explicitLock=false, dhtVer=null, last=false, near=false, clientFirst=false,
> node=b992bdfd-1f6e-45d0-ac6e-de4c1355eb34]], trackable=true,
> finishOnePhaseCalled=false,
> innerFuts=[CheckBackupFuture[node=9e9758d1-8dbf-44e2-8181-6f9b454cf61c,
> loc=true, done=true]], super=GridCompoundIdentityFuture
> [super=GridCompoundFuture [rdc=o.a.i.i.util.lang.GridFunc$27@45c0b702,
> initFlag=0, lsnrCalls=0, done=false, cancelled=false, err=null,
> futs=[true]]]]class
> org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
> Failed to commit transaction (transaction has been rolled back on backup
> node): GridCacheVersion [topVer=104082481, time=1492680221502,
> order=1492681260388, nodeOrder=44]
>
> 2017-04-20 11:23:41,600 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.distributed.
> near.GridNearTxFinishFuture.checkBackup(GridNearTxFinishFuture.java:571)
>
> 2017-04-20 11:23:41,602 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.distributed.
> near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:400)
>
> 2017-04-20 11:23:41,604 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.distributed.
> near.GridNearTxLocal$4.apply(GridNearTxLocal.java:874)
>
> 2017-04-20 11:23:41,605 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.distributed.
> near.GridNearTxLocal$4.apply(GridNearTxLocal.java:866)
>
> 2017-04-20 11:23:41,609 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(
> GridFutureAdapter.java:271)
>
> 2017-04-20 11:23:41,611 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.util.future.GridFutureAdapter.
> listen(GridFutureAdapter.java:228)
>
> 2017-04-20 11:23:41,612 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.distributed.
> near.GridNearTxLocal.commitAsync(GridNearTxLocal.java:866)
>
> 2017-04-20 11:23:41,615 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.transactions.
> IgniteTxLocalAdapter.removeAllAsync0(IgniteTxLocalAdapter.java:3527)
>
> 2017-04-20 11:23:41,619 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.transactions.
> IgniteTxLocalAdapter.removeAllAsync(IgniteTxLocalAdapter.java:3320)
>
> 2017-04-20 11:23:41,622 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter$41.op(
> GridCacheAdapter.java:2994)
>
> 2017-04-20 11:23:41,626 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter$41.op(
> GridCacheAdapter.java:2992)
>
> 2017-04-20 11:23:41,628 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(
> GridCacheAdapter.java:4235)
>
> 2017-04-20 11:23:41,630 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.remove0(
> GridCacheAdapter.java:2992)
>
> 2017-04-20 11:23:41,632 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.remove(
> GridCacheAdapter.java:2977)
>
> 2017-04-20 11:23:41,633 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.remove(
> GridCacheAdapter.java:2958)
>
> 2017-04-20 11:23:41,635 ERROR [stderr] (tSA 6_1968933222)       at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.remove(
> IgniteCacheProxy.java:1472)
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/IgniteCacheProxy-connection-
> failure-in-REPLICATAED-mode-tp11769p12160.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Best regards,
Andrey V. Mashenkov

Re: IgniteCacheProxy connection failure in REPLICATAED mode

Posted by rick_tem <rv...@temenos.com>.
Hey Andre,

Here's what the stack trace looks like inside jboss.

Topology snapshot [ver=46, servers=1, clients=2, CPUs=8, heap=7.1GB]

2017-04-20 11:23:41,575 ERROR [stderr] (tSA 6_1968933222) [11:23:41] (err)
*Failed to execute compound future reducer: GridNearTxFinishFuture*
[futId=bcd3eba8b51-55bc7836-f114-41d7-b0d0-44a14beb441f, tx=GridNearTxLocal
[mappings=IgniteTxMappingsSingleImpl [mapping=GridDistributedTxMapping
[entries=[IgniteTxEntry [key=KeyCacheObjectImpl
[val=FBNK_STATIC_CHANGE_000*360494.0008100.01, hasValBytes=true],
cacheId=-1750629716, partId=-1, txKey=IgniteTxKey [key=KeyCacheObjectImpl
[val=FBNK_STATIC_CHANGE_000*360494.0008100.01, hasValBytes=true],
cacheId=-1750629716], val=[op=DELETE, val=null], prevVal=[op=NOOP,
val=null], oldVal=[op=NOOP, val=null], entryProcessorsCol=null, ttl=-1,
conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=null,
filters=[], filtersPassed=false, filtersSet=true,
entry=GridDhtDetachedCacheEntry [super=GridDistributedCacheEntry
[super=GridCacheMapEntry [key=KeyCacheObjectImpl
[val=FBNK_STATIC_CHANGE_000*360494.0008100.01, hasValBytes=true], val=null,
startVer=1492681260390, ver=GridCacheVersion [topVer=104082481,
time=1492680221503, order=1492681260390, nodeOrder=44], hash=-852679692,
extras=null, flags=0]]], prepared=0, locked=false,
nodeId=b992bdfd-1f6e-45d0-ac6e-de4c1355eb34, locMapped=false,
expiryPlc=null, transferExpiryPlc=false, flags=0, partUpdateCntr=0,
serReadVer=null, xidVer=GridCacheVersion [topVer=104082481,
time=1492680221502, order=1492681260388, nodeOrder=44]]],
explicitLock=false, dhtVer=null, last=false, near=false, clientFirst=false,
node=b992bdfd-1f6e-45d0-ac6e-de4c1355eb34]], nearLocallyMapped=false,
colocatedLocallyMapped=false, needCheckBackup=false, hasRemoteLocks=false,
thread=tSA 6_1968933222, mappings=IgniteTxMappingsSingleImpl
[mapping=GridDistributedTxMapping [entries=[IgniteTxEntry
[key=KeyCacheObjectImpl [val=FBNK_STATIC_CHANGE_000*360494.0008100.01,
hasValBytes=true], cacheId=-1750629716, partId=-1, txKey=IgniteTxKey
[key=KeyCacheObjectImpl [val=FBNK_STATIC_CHANGE_000*360494.0008100.01,
hasValBytes=true], cacheId=-1750629716], val=[op=DELETE, val=null],
prevVal=[op=NOOP, val=null], oldVal=[op=NOOP, val=null],
entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null,
explicitVer=null, dhtVer=null, filters=[], filtersPassed=false,
filtersSet=true, entry=GridDhtDetachedCacheEntry
[super=GridDistributedCacheEntry [super=GridCacheMapEntry
[key=KeyCacheObjectImpl [val=FBNK_STATIC_CHANGE_000*360494.0008100.01,
hasValBytes=true], val=null, startVer=1492681260390, ver=GridCacheVersion
[topVer=104082481, time=1492680221503, order=1492681260390, nodeOrder=44],
hash=-852679692, extras=null, flags=0]]], prepared=0, locked=false,
nodeId=b992bdfd-1f6e-45d0-ac6e-de4c1355eb34, locMapped=false,
expiryPlc=null, transferExpiryPlc=false, flags=0, partUpdateCntr=0,
serReadVer=null, xidVer=GridCacheVersion [topVer=104082481,
time=1492680221502, order=1492681260388, nodeOrder=44]]],
explicitLock=false, dhtVer=null, last=false, near=false, clientFirst=false,
node=b992bdfd-1f6e-45d0-ac6e-de4c1355eb34]], super=GridDhtTxLocalAdapter
[nearOnOriginatingNode=false, nearNodes=[], dhtNodes=[], explicitLock=false,
super=IgniteTxLocalAdapter [completedBase=null, sndTransformedVals=false,
depEnabled=false, txState=IgniteTxImplicitSingleStateImpl [init=true],
super=IgniteTxAdapter [xidVer=GridCacheVersion [topVer=104082481,
time=1492680221502, order=1492681260388, nodeOrder=44], writeVer=null,
implicit=true, loc=true, threadId=391, startTime=1492680221473,
nodeId=9e9758d1-8dbf-44e2-8181-6f9b454cf61c, startVer=GridCacheVersion
[topVer=104082481, time=1492680221502, order=1492681260388, nodeOrder=44],
endVer=null, isolation=READ_COMMITTED, concurrency=OPTIMISTIC, timeout=0,
sysInvalidate=false, sys=false, plc=2, commitVer=GridCacheVersion
[topVer=104082481, time=1492680221502, order=1492681260388, nodeOrder=44],
finalizing=NONE, preparing=false, invalidParts=null, state=PREPARED,
timedOut=false, topVer=AffinityTopologyVersion [topVer=45, minorTopVer=0],
duration=95ms, onePhaseCommit=true], size=1]]], commit=true,
mappings=IgniteTxMappingsSingleImpl [mapping=GridDistributedTxMapping
[entries=[IgniteTxEntry [key=KeyCacheObjectImpl
[val=FBNK_STATIC_CHANGE_000*360494.0008100.01, hasValBytes=true],
cacheId=-1750629716, partId=-1, txKey=IgniteTxKey [key=KeyCacheObjectImpl
[val=FBNK_STATIC_CHANGE_000*360494.0008100.01, hasValBytes=true],
cacheId=-1750629716], val=[op=DELETE, val=null], prevVal=[op=NOOP,
val=null], oldVal=[op=NOOP, val=null], entryProcessorsCol=null, ttl=-1,
conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=null,
filters=[], filtersPassed=false, filtersSet=true,
entry=GridDhtDetachedCacheEntry [super=GridDistributedCacheEntry
[super=GridCacheMapEntry [key=KeyCacheObjectImpl
[val=FBNK_STATIC_CHANGE_000*360494.0008100.01, hasValBytes=true], val=null,
startVer=1492681260390, ver=GridCacheVersion [topVer=104082481,
time=1492680221503, order=1492681260390, nodeOrder=44], hash=-852679692,
extras=null, flags=0]]], prepared=0, locked=false,
nodeId=b992bdfd-1f6e-45d0-ac6e-de4c1355eb34, locMapped=false,
expiryPlc=null, transferExpiryPlc=false, flags=0, partUpdateCntr=0,
serReadVer=null, xidVer=GridCacheVersion [topVer=104082481,
time=1492680221502, order=1492681260388, nodeOrder=44]]],
explicitLock=false, dhtVer=null, last=false, near=false, clientFirst=false,
node=b992bdfd-1f6e-45d0-ac6e-de4c1355eb34]], trackable=true,
finishOnePhaseCalled=false,
innerFuts=[CheckBackupFuture[node=9e9758d1-8dbf-44e2-8181-6f9b454cf61c,
loc=true, done=true]], super=GridCompoundIdentityFuture
[super=GridCompoundFuture [rdc=o.a.i.i.util.lang.GridFunc$27@45c0b702,
initFlag=0, lsnrCalls=0, done=false, cancelled=false, err=null,
futs=[true]]]]class
org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
Failed to commit transaction (transaction has been rolled back on backup
node): GridCacheVersion [topVer=104082481, time=1492680221502,
order=1492681260388, nodeOrder=44]

2017-04-20 11:23:41,600 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.checkBackup(GridNearTxFinishFuture.java:571)

2017-04-20 11:23:41,602 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:400)

2017-04-20 11:23:41,604 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$4.apply(GridNearTxLocal.java:874)

2017-04-20 11:23:41,605 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$4.apply(GridNearTxLocal.java:866)

2017-04-20 11:23:41,609 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:271)

2017-04-20 11:23:41,611 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:228)

2017-04-20 11:23:41,612 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commitAsync(GridNearTxLocal.java:866)

2017-04-20 11:23:41,615 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.removeAllAsync0(IgniteTxLocalAdapter.java:3527)

2017-04-20 11:23:41,619 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.removeAllAsync(IgniteTxLocalAdapter.java:3320)

2017-04-20 11:23:41,622 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$41.op(GridCacheAdapter.java:2994)

2017-04-20 11:23:41,626 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$41.op(GridCacheAdapter.java:2992)

2017-04-20 11:23:41,628 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4235)

2017-04-20 11:23:41,630 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.remove0(GridCacheAdapter.java:2992)

2017-04-20 11:23:41,632 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.remove(GridCacheAdapter.java:2977)

2017-04-20 11:23:41,633 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.remove(GridCacheAdapter.java:2958)

2017-04-20 11:23:41,635 ERROR [stderr] (tSA 6_1968933222) 	at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.remove(IgniteCacheProxy.java:1472)




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteCacheProxy-connection-failure-in-REPLICATAED-mode-tp11769p12160.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IgniteCacheProxy connection failure in REPLICATAED mode

Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Rick,

Looks weird. Cache operation on replicated cache on server node should not
failed when other node leaved.
Would you please attach logs?

Ignite 2.0 ETA should be release on next week.
But seems, it may delays for 1-2 weeks as there is neither node freeze
happened nor vote started.



On Fri, Apr 21, 2017 at 12:10 PM, rick_tem <rv...@temenos.com> wrote:

> Hi Andrey,
>
> Thanks for you response.  Still seems to be problematic.  See below.  Any
> idea when 2.0 will be available?  Recall I had one Ignite as a client in
> Jboss and two Ignite servers.  What I've done is force the Jboss into a
> server, and start one other Ignite instance (two Ignite server nodes, 0
> clients).  Now when I kill the Ignite node, I did get some exceptions in
> the
> Jboss console and at least once I had some remove cache operations fail.  I
> don't believe this should happen in replicated mode and assume this is the
> same bug.  So to get around that, I catch any exception around the
> cache.remove(key) and retry, but since, I haven't been able to reproduce
> it.
> I'll try to see if I can write a test case for it.  Regardless, whether
> client or server, I think we shouldn't expect failures in replicated mode
> with the below config, correct?  Oracle Coherence doesn't fail when you
> take
> down a node in replicated mode.
>
> Thanks,
> Rick
>
>                                 <bean class="org.apache.ignite.
> configuration.CacheConfiguration">
>                                         <property name="name"
> value="MyCache" />
>                                         <property name="cacheMode"
> value="REPLICATED" />
>                                         <property name="memoryMode"
> value="ONHEAP_TIERED" />
>
>                                         <property name="offHeapMaxMemory"
> value="#{10 * 1024L * 1024L * 1024L}"
> />
>                                         <property name="evictionPolicy">
>
>                                                 <bean
> class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
>
>                                                         <property
> name="maxSize" value="1000000" />
>                                                 </bean>
>                                         </property>
>                                         <property name="atomicityMode"
> value="TRANSACTIONAL" />
>                                         <property name="backups" value="1"
> />
>                                         <property name="writeSynchronizationMode"
> value="FULL_SYNC" />
>                                         <property name="statisticsEnabled"
> value="true" />
>                                 </bean>
>                         </list>
>                 </property>
>
>
> C:\java\GitHub\ignite1.9\ignite-1.9\modules [ignite-1.9 ≡ +20 ~14 -0 !]>
> git
> apply --verbose --check 4473.patch
> Checking patch
> modules/core/src/main/java/org/apache/ignite/internal/
> IgniteNeedReconnectException.java...
> error: while searching for:
>
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/internal/
> IgniteNeedReconnectException.java:0
> error:
> modules/core/src/main/java/org/apache/ignite/internal/
> IgniteNeedReconnectException.java:
> patch does not apply
> Checking patch
> modules/core/src/main/java/org/apache/ignite/spi/
> discovery/tcp/TcpDiscoverySpi.java...
> error: while searching for:
>         return ignite().configuration().getSslContextFactory() != null;
>     }
>
>     /**
>      * <strong>FOR TEST ONLY!!!</strong>
>      */
>
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/spi/
> discovery/tcp/TcpDiscoverySpi.java:1933
> error:
> modules/core/src/main/java/org/apache/ignite/spi/
> discovery/tcp/TcpDiscoverySpi.java:
> patch does not apply
> Checking patch
> modules/core/src/test/java/org/apache/ignite/testsuites/
> IgniteClientReconnectTestSuite.java...
> error: while searching for:
> import org.apache.ignite.internal.IgniteClientReconnectServicesTest;
> import org.apache.ignite.internal.IgniteClientReconnectStopTest;
> import org.apache.ignite.internal.IgniteClientReconnectStreamerTest;
>
> /**
>  *
>
> error: patch failed:
> modules/core/src/test/java/org/apache/ignite/testsuites/
> IgniteClientReconnectTestSuite.java:29
> error:
> modules/core/src/test/java/org/apache/ignite/testsuites/
> IgniteClientReconnectTestSuite.java:
> patch does not apply
> Checking patch
> modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java...
> error: while searching for:
>     /** Periodic starvation check interval. */
>     private static final long PERIODIC_STARVATION_CHECK_FREQ = 1000 * 30;
>
>     /** */
>     @GridToStringExclude
>     private GridKernalContextImpl ctx;
>
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/internal/
> IgniteKernal.java:253
> error:
> modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java:
> patch does not apply
> Checking patch
> modules/core/src/main/java/org/apache/ignite/spi/
> discovery/tcp/ServerImpl.java...
> error: while searching for:
>         throw new UnsupportedOperationException();
>     }
>
>     /** {@inheritDoc} */
>     @Override protected IgniteSpiThread workerThread() {
>         return msgWorker;
>
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/spi/
> discovery/tcp/ServerImpl.java:1589
> error:
> modules/core/src/main/java/org/apache/ignite/spi/
> discovery/tcp/ServerImpl.java:
> patch does not apply
> Checking patch
> modules/core/src/test/java/org/apache/ignite/internal/
> IgniteClientReconnectCacheTest.java...
> error: while searching for:
>                 try {
>
> Ignition.start(optimize(getConfiguration(getTestGridName(SRV_CNT))));
>
>                     fail();
>
>                     return false;
>                 }
>                 catch (IgniteClientDisconnectedException e) {
>                     log.info("Expected start error: " + e);
>
> error: patch failed:
> modules/core/src/test/java/org/apache/ignite/internal/
> IgniteClientReconnectCacheTest.java:700
> error:
> modules/core/src/test/java/org/apache/ignite/internal/
> IgniteClientReconnectCacheTest.java:
> patch does not apply
> Checking patch
> modules/core/src/test/java/org/apache/ignite/internal/
> IgniteClientRejoinTest.java...
> error: while searching for:
>
> error: patch failed:
> modules/core/src/test/java/org/apache/ignite/internal/
> IgniteClientRejoinTest.java:0
> error:
> modules/core/src/test/java/org/apache/ignite/internal/
> IgniteClientRejoinTest.java:
> patch does not apply
> Checking patch
> modules/core/src/main/java/org/apache/ignite/spi/
> discovery/tcp/ClientImpl.java...
> error: while searching for:
>     /** */
>     private static final Object SPI_RECONNECT_FAILED =
> "SPI_RECONNECT_FAILED";
>
>     /** Remote nodes. */
>     private final ConcurrentMap<UUID, TcpDiscoveryNode> rmtNodes = new
> ConcurrentHashMap8<>();
>
>
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/spi/
> discovery/tcp/ClientImpl.java:127
> error:
> modules/core/src/main/java/org/apache/ignite/spi/
> discovery/tcp/ClientImpl.java:
> patch does not apply
> Checking patch
> modules/core/src/main/java/org/apache/ignite/internal/processors/cache/
> GridCachePartitionExchangeManager.java...
> error: while searching for:
> import java.util.concurrent.locks.ReadWriteLock;
> import java.util.concurrent.locks.ReentrantReadWriteLock;
> import org.apache.ignite.IgniteCheckedException;
> import org.apache.ignite.IgniteSystemProperties;
> import org.apache.ignite.cache.affinity.AffinityFunction;
> import org.apache.ignite.cluster.ClusterNode;
>
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/internal/processors/cache/
> GridCachePartitionExchangeManager.java:43
> error:
> modules/core/src/main/java/org/apache/ignite/internal/processors/cache/
> GridCachePartitionExchangeManager.java:
> patch does not apply
> Checking patch
> modules/core/src/main/java/org/apache/ignite/internal/
> GridKernalGatewayImpl.java...
> error: while searching for:
>
>     /** */
>     @GridToStringExclude
>     private IgniteFutureImpl<?> reconnectFut;
>
>     /** */
>     private final AtomicReference<GridKernalState> state = new
> AtomicReference<>(GridKernalState.STOPPED);
>
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/internal/
> GridKernalGatewayImpl.java:44
> error:
> modules/core/src/main/java/org/apache/ignite/internal/
> GridKernalGatewayImpl.java:
> patch does not apply
> Checking patch
> modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/
> TcpDiscoveryImpl.java...
> error: while searching for:
> import org.apache.ignite.IgniteException;
> import org.apache.ignite.IgniteLogger;
> import org.apache.ignite.cluster.ClusterNode;
> import org.apache.ignite.internal.IgniteInterruptedCheckedException;
> import org.apache.ignite.internal.util.typedef.internal.LT;
> import org.apache.ignite.internal.util.typedef.internal.U;
>
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/
> TcpDiscoveryImpl.java:29
> error:
> modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/
> TcpDiscoveryImpl.java:
> patch does not apply
> Checking patch
> modules/core/src/main/java/org/apache/ignite/internal/
> processors/cache/distributed/dht/GridDhtAssignmentFetchFuture.java...
> error: while searching for:
> import org.apache.ignite.internal.util.future.GridFutureAdapter;
> import org.apache.ignite.internal.util.tostring.GridToStringInclude;
> import org.apache.ignite.internal.util.typedef.T2;
> import org.apache.ignite.internal.util.typedef.internal.CU;
> import org.apache.ignite.internal.util.typedef.internal.S;
> import org.apache.ignite.internal.util.typedef.internal.U;
>
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/internal/
> processors/cache/distributed/dht/GridDhtAssignmentFetchFuture.java:33
> error:
> modules/core/src/main/java/org/apache/ignite/internal/
> processors/cache/distributed/dht/GridDhtAssignmentFetchFuture.java:
> patch does not apply
> Checking patch
> modules/core/src/main/java/org/apache/ignite/internal/
> processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFutur
> e.java...
> error: while searching for:
>
> package
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader;
>
> import java.util.ArrayList;
> import java.util.Collection;
> import java.util.Collections;
>
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/internal/
> processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFutur
> e.java:17
> error:
> modules/core/src/main/java/org/apache/ignite/internal/
> processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFutur
> e.java:
> patch does not apply
> Checking patch
> modules/core/src/main/java/org/apache/ignite/internal/processors/service/
> GridServiceProcessor.java...
> error: while searching for:
>         }
>     }
>
>         /**
>          * Deployment callback.
>          *
>          * @param dep Service deployment.
>          * @param topVer Topology version.
>          */
>         private void onDeployment(final GridServiceDeployment dep, final
> AffinityTopologyVersion topVer) {
>             // Retry forever.
>             try {
>                 AffinityTopologyVersion newTopVer =
> ctx.discovery().topologyVersionEx();
>
>                 // If topology version changed, reassignment will happen
> from topology event.
>                 if (newTopVer.equals(topVer))
>                     reassign(dep, topVer);
>             }
>             catch (IgniteCheckedException e) {
>                 if (!(e instanceof ClusterTopologyCheckedException))
>                     log.error("Failed to do service reassignment (will
> retry): " + dep.configuration().getName(), e);
>
>                 AffinityTopologyVersion newTopVer =
> ctx.discovery().topologyVersionEx();
>
>                 if (!newTopVer.equals(topVer)) {
>                     assert newTopVer.compareTo(topVer) > 0;
>
>                     // Reassignment will happen from topology event.
>                     return;
>                 }
>
>                 ctx.timeout().addTimeoutObject(new GridTimeoutObject() {
>                     private IgniteUuid id = IgniteUuid.randomUuid();
>
>                     private long start = System.currentTimeMillis();
>
>                     @Override public IgniteUuid timeoutId() {
>                         return id;
>                     }
>
>                     @Override public long endTime() {
>                         return start + RETRY_TIMEOUT;
>                     }
>
>                     @Override public void onTimeout() {
>                         if (!busyLock.enterBusy())
>                             return;
>
>                         try {
>                             // Try again.
>                             onDeployment(dep, topVer);
>                         }
>                         finally {
>                             busyLock.leaveBusy();
>                         }
>                     }
>                 });
>         }
>     }
>
>
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/internal/processors/service/
> GridServiceProcessor.java:1508
> error:
> modules/core/src/main/java/org/apache/ignite/internal/processors/service/
> GridServiceProcessor.java:
> patch does not apply
> Checking patch
> modules/core/src/main/java/org/apache/ignite/internal/managers/discovery/
> GridDiscoveryManager.java...
> error: while searching for:
> import org.apache.ignite.spi.discovery.DiscoverySpiListener;
> import org.apache.ignite.spi.discovery.DiscoverySpiNodeAuthenticator;
> import org.apache.ignite.spi.discovery.DiscoverySpiOrderSupport;
> import org.apache.ignite.thread.IgniteThread;
> import org.jetbrains.annotations.Nullable;
> import org.jsr166.ConcurrentHashMap8;
>
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/internal/managers/discovery/
> GridDiscoveryManager.java:112
> error:
> modules/core/src/main/java/org/apache/ignite/internal/managers/discovery/
> GridDiscoveryManager.java:
> patch does not apply
> Checking patch
> modules/core/src/test/java/org/apache/ignite/spi/discovery/tcp/
> TcpClientDiscoverySpiSelfTest.java...
> error: while searching for:
> import java.util.concurrent.atomic.AtomicInteger;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteCheckedException;
> import org.apache.ignite.IgniteInterruptedException;
> import org.apache.ignite.IgniteMessaging;
> import org.apache.ignite.IgniteState;
>
> error: patch failed:
> modules/core/src/test/java/org/apache/ignite/spi/discovery/tcp/
> TcpClientDiscoverySpiSelfTest.java:32
> error:
> modules/core/src/test/java/org/apache/ignite/spi/discovery/tcp/
> TcpClientDiscoverySpiSelfTest.java:
> patch does not apply
> C:\java\GitHub\ignite1.9\ignite-1.9\modules [ignite-1.9 ≡ +20 ~14 -0 !]>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/IgniteCacheProxy-connection-
> failure-in-REPLICATAED-mode-tp11769p12140.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Best regards,
Andrey V. Mashenkov

Re: IgniteCacheProxy connection failure in REPLICATAED mode

Posted by rick_tem <rv...@temenos.com>.
Hi Andrey,

Thanks for you response.  Still seems to be problematic.  See below.  Any
idea when 2.0 will be available?  Recall I had one Ignite as a client in
Jboss and two Ignite servers.  What I've done is force the Jboss into a
server, and start one other Ignite instance (two Ignite server nodes, 0
clients).  Now when I kill the Ignite node, I did get some exceptions in the
Jboss console and at least once I had some remove cache operations fail.  I
don't believe this should happen in replicated mode and assume this is the
same bug.  So to get around that, I catch any exception around the
cache.remove(key) and retry, but since, I haven't been able to reproduce it. 
I'll try to see if I can write a test case for it.  Regardless, whether
client or server, I think we shouldn't expect failures in replicated mode
with the below config, correct?  Oracle Coherence doesn't fail when you take
down a node in replicated mode.

Thanks,
Rick

				<bean class="org.apache.ignite.configuration.CacheConfiguration">
					<property name="name" value="MyCache" />
					<property name="cacheMode" value="REPLICATED" />
					<property name="memoryMode" value="ONHEAP_TIERED" />
					
					<property name="offHeapMaxMemory" value="#{10 * 1024L * 1024L * 1024L}"
/>
					<property name="evictionPolicy">
						
						<bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
							
							<property name="maxSize" value="1000000" />
						</bean>
					</property>
					<property name="atomicityMode" value="TRANSACTIONAL" />
					<property name="backups" value="1" />
					<property name="writeSynchronizationMode" value="FULL_SYNC" />
					<property name="statisticsEnabled" value="true" />
				</bean>
			</list>
		</property>
	

C:\java\GitHub\ignite1.9\ignite-1.9\modules [ignite-1.9 ≡ +20 ~14 -0 !]> git
apply --verbose --check 4473.patch
Checking patch
modules/core/src/main/java/org/apache/ignite/internal/IgniteNeedReconnectException.java...
error: while searching for:

error: patch failed:
modules/core/src/main/java/org/apache/ignite/internal/IgniteNeedReconnectException.java:0
error:
modules/core/src/main/java/org/apache/ignite/internal/IgniteNeedReconnectException.java:
patch does not apply
Checking patch
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java...
error: while searching for:
        return ignite().configuration().getSslContextFactory() != null;
    }

    /**
     * <strong>FOR TEST ONLY!!!</strong>
     */

error: patch failed:
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java:1933
error:
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java:
patch does not apply
Checking patch
modules/core/src/test/java/org/apache/ignite/testsuites/IgniteClientReconnectTestSuite.java...
error: while searching for:
import org.apache.ignite.internal.IgniteClientReconnectServicesTest;
import org.apache.ignite.internal.IgniteClientReconnectStopTest;
import org.apache.ignite.internal.IgniteClientReconnectStreamerTest;

/**
 *

error: patch failed:
modules/core/src/test/java/org/apache/ignite/testsuites/IgniteClientReconnectTestSuite.java:29
error:
modules/core/src/test/java/org/apache/ignite/testsuites/IgniteClientReconnectTestSuite.java:
patch does not apply
Checking patch
modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java...
error: while searching for:
    /** Periodic starvation check interval. */
    private static final long PERIODIC_STARVATION_CHECK_FREQ = 1000 * 30;

    /** */
    @GridToStringExclude
    private GridKernalContextImpl ctx;

error: patch failed:
modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java:253
error:
modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java:
patch does not apply
Checking patch
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/ServerImpl.java...
error: while searching for:
        throw new UnsupportedOperationException();
    }

    /** {@inheritDoc} */
    @Override protected IgniteSpiThread workerThread() {
        return msgWorker;

error: patch failed:
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/ServerImpl.java:1589
error:
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/ServerImpl.java:
patch does not apply
Checking patch
modules/core/src/test/java/org/apache/ignite/internal/IgniteClientReconnectCacheTest.java...
error: while searching for:
                try {
                   
Ignition.start(optimize(getConfiguration(getTestGridName(SRV_CNT))));

                    fail();

                    return false;
                }
                catch (IgniteClientDisconnectedException e) {
                    log.info("Expected start error: " + e);

error: patch failed:
modules/core/src/test/java/org/apache/ignite/internal/IgniteClientReconnectCacheTest.java:700
error:
modules/core/src/test/java/org/apache/ignite/internal/IgniteClientReconnectCacheTest.java:
patch does not apply
Checking patch
modules/core/src/test/java/org/apache/ignite/internal/IgniteClientRejoinTest.java...
error: while searching for:

error: patch failed:
modules/core/src/test/java/org/apache/ignite/internal/IgniteClientRejoinTest.java:0
error:
modules/core/src/test/java/org/apache/ignite/internal/IgniteClientRejoinTest.java:
patch does not apply
Checking patch
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/ClientImpl.java...
error: while searching for:
    /** */
    private static final Object SPI_RECONNECT_FAILED =
"SPI_RECONNECT_FAILED";

    /** Remote nodes. */
    private final ConcurrentMap<UUID, TcpDiscoveryNode> rmtNodes = new
ConcurrentHashMap8<>();


error: patch failed:
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/ClientImpl.java:127
error:
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/ClientImpl.java:
patch does not apply
Checking patch
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java...
error: while searching for:
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import org.apache.ignite.IgniteCheckedException;
import org.apache.ignite.IgniteSystemProperties;
import org.apache.ignite.cache.affinity.AffinityFunction;
import org.apache.ignite.cluster.ClusterNode;

error: patch failed:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java:43
error:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java:
patch does not apply
Checking patch
modules/core/src/main/java/org/apache/ignite/internal/GridKernalGatewayImpl.java...
error: while searching for:

    /** */
    @GridToStringExclude
    private IgniteFutureImpl<?> reconnectFut;

    /** */
    private final AtomicReference<GridKernalState> state = new
AtomicReference<>(GridKernalState.STOPPED);

error: patch failed:
modules/core/src/main/java/org/apache/ignite/internal/GridKernalGatewayImpl.java:44
error:
modules/core/src/main/java/org/apache/ignite/internal/GridKernalGatewayImpl.java:
patch does not apply
Checking patch
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoveryImpl.java...
error: while searching for:
import org.apache.ignite.IgniteException;
import org.apache.ignite.IgniteLogger;
import org.apache.ignite.cluster.ClusterNode;
import org.apache.ignite.internal.IgniteInterruptedCheckedException;
import org.apache.ignite.internal.util.typedef.internal.LT;
import org.apache.ignite.internal.util.typedef.internal.U;

error: patch failed:
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoveryImpl.java:29
error:
modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoveryImpl.java:
patch does not apply
Checking patch
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtAssignmentFetchFuture.java...
error: while searching for:
import org.apache.ignite.internal.util.future.GridFutureAdapter;
import org.apache.ignite.internal.util.tostring.GridToStringInclude;
import org.apache.ignite.internal.util.typedef.T2;
import org.apache.ignite.internal.util.typedef.internal.CU;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.internal.util.typedef.internal.U;

error: patch failed:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtAssignmentFetchFuture.java:33
error:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtAssignmentFetchFuture.java:
patch does not apply
Checking patch
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java...
error: while searching for:

package
org.apache.ignite.internal.processors.cache.distributed.dht.preloader;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;

error: patch failed:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java:17
error:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java:
patch does not apply
Checking patch
modules/core/src/main/java/org/apache/ignite/internal/processors/service/GridServiceProcessor.java...
error: while searching for:
        }
    }

        /**
         * Deployment callback.
         *
         * @param dep Service deployment.
         * @param topVer Topology version.
         */
        private void onDeployment(final GridServiceDeployment dep, final
AffinityTopologyVersion topVer) {
            // Retry forever.
            try {
                AffinityTopologyVersion newTopVer =
ctx.discovery().topologyVersionEx();

                // If topology version changed, reassignment will happen
from topology event.
                if (newTopVer.equals(topVer))
                    reassign(dep, topVer);
            }
            catch (IgniteCheckedException e) {
                if (!(e instanceof ClusterTopologyCheckedException))
                    log.error("Failed to do service reassignment (will
retry): " + dep.configuration().getName(), e);

                AffinityTopologyVersion newTopVer =
ctx.discovery().topologyVersionEx();

                if (!newTopVer.equals(topVer)) {
                    assert newTopVer.compareTo(topVer) > 0;

                    // Reassignment will happen from topology event.
                    return;
                }

                ctx.timeout().addTimeoutObject(new GridTimeoutObject() {
                    private IgniteUuid id = IgniteUuid.randomUuid();

                    private long start = System.currentTimeMillis();

                    @Override public IgniteUuid timeoutId() {
                        return id;
                    }

                    @Override public long endTime() {
                        return start + RETRY_TIMEOUT;
                    }

                    @Override public void onTimeout() {
                        if (!busyLock.enterBusy())
                            return;

                        try {
                            // Try again.
                            onDeployment(dep, topVer);
                        }
                        finally {
                            busyLock.leaveBusy();
                        }
                    }
                });
        }
    }


error: patch failed:
modules/core/src/main/java/org/apache/ignite/internal/processors/service/GridServiceProcessor.java:1508
error:
modules/core/src/main/java/org/apache/ignite/internal/processors/service/GridServiceProcessor.java:
patch does not apply
Checking patch
modules/core/src/main/java/org/apache/ignite/internal/managers/discovery/GridDiscoveryManager.java...
error: while searching for:
import org.apache.ignite.spi.discovery.DiscoverySpiListener;
import org.apache.ignite.spi.discovery.DiscoverySpiNodeAuthenticator;
import org.apache.ignite.spi.discovery.DiscoverySpiOrderSupport;
import org.apache.ignite.thread.IgniteThread;
import org.jetbrains.annotations.Nullable;
import org.jsr166.ConcurrentHashMap8;

error: patch failed:
modules/core/src/main/java/org/apache/ignite/internal/managers/discovery/GridDiscoveryManager.java:112
error:
modules/core/src/main/java/org/apache/ignite/internal/managers/discovery/GridDiscoveryManager.java:
patch does not apply
Checking patch
modules/core/src/test/java/org/apache/ignite/spi/discovery/tcp/TcpClientDiscoverySpiSelfTest.java...
error: while searching for:
import java.util.concurrent.atomic.AtomicInteger;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCheckedException;
import org.apache.ignite.IgniteInterruptedException;
import org.apache.ignite.IgniteMessaging;
import org.apache.ignite.IgniteState;

error: patch failed:
modules/core/src/test/java/org/apache/ignite/spi/discovery/tcp/TcpClientDiscoverySpiSelfTest.java:32
error:
modules/core/src/test/java/org/apache/ignite/spi/discovery/tcp/TcpClientDiscoverySpiSelfTest.java:
patch does not apply
C:\java\GitHub\ignite1.9\ignite-1.9\modules [ignite-1.9 ≡ +20 ~14 -0 !]>



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteCacheProxy-connection-failure-in-REPLICATAED-mode-tp11769p12140.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IgniteCacheProxy connection failure in REPLICATAED mode

Posted by Andrey Mashenkov <an...@gmail.com>.
Hi

Looks like master has gone ahead. Try this one.



On Tue, Apr 18, 2017 at 4:51 PM, rick_tem <rv...@temenos.com> wrote:

> Hi,
>
> I appreciate the last response to this.  I've checked out the 1.9 branch
> (ie. git clone -b ignite-1.9 --single-branch
> https://github.com/apache/ignite.git ignite-1.9)...I'm new to github so
> bear
> with me.  In seeing if this patch will apply, I get the following error.
> I'm not exactly sure why.  Is there a procedure somewhere of how to apply
> patches?  As well, any documentation links for beginner contributors would
> be appreciated.
>
> Thanks!
>
> C:\java\GitHub\ignite1.9\ignite-1.9\modules [ignite-1.9 ≡ +1 ~0 -0 !]> git
> apply --check IGNITE_4473___Client_should_re_
> try_connection_attempt_in_case_of_concurrent_network_failur.patch
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/internal/processors/cache/
> GridCachePartitionExchangeMa
> nager.java:1836
> error:
> modules/core/src/main/java/org/apache/ignite/internal/processors/cache/
> GridCachePartitionExchangeManager.java:
> pa
> tch does not apply
> error: patch failed:
> modules/core/src/main/java/org/apache/ignite/internal/
> processors/cache/distributed/dht/GridDhtAssig
> nmentFetchFuture.java:17
> error:
> modules/core/src/main/java/org/apache/ignite/internal/
> processors/cache/distributed/dht/GridDhtAssignmentFetchFutu
> re.java: patch does not apply
> *error: patch failed:
> modules/core/src/main/java/org/apache/ignite/internal/
> processors/cache/distributed/dht/preloader/Gr
> idDhtPartitionsExchangeFuture.java:54*
> error:
> modules/core/src/main/java/org/apache/ignite/internal/
> processors/cache/distributed/dht/preloader/GridDhtPartition
> sExchangeFuture.java: patch does not apply
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/IgniteCacheProxy-connection-
> failure-in-REPLICATAED-mode-tp11769p12030.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Best regards,
Andrey V. Mashenkov

Re: IgniteCacheProxy connection failure in REPLICATAED mode

Posted by rick_tem <rv...@temenos.com>.
Hi,

I appreciate the last response to this.  I've checked out the 1.9 branch
(ie. git clone -b ignite-1.9 --single-branch
https://github.com/apache/ignite.git ignite-1.9)...I'm new to github so bear
with me.  In seeing if this patch will apply, I get the following error. 
I'm not exactly sure why.  Is there a procedure somewhere of how to apply
patches?  As well, any documentation links for beginner contributors would
be appreciated.

Thanks!

C:\java\GitHub\ignite1.9\ignite-1.9\modules [ignite-1.9 ≡ +1 ~0 -0 !]> git
apply --check IGNITE_4473___Client_should_re_
try_connection_attempt_in_case_of_concurrent_network_failur.patch
error: patch failed:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeMa
nager.java:1836
error:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java:
pa
tch does not apply
error: patch failed:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtAssig
nmentFetchFuture.java:17
error:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtAssignmentFetchFutu
re.java: patch does not apply
*error: patch failed:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/Gr
idDhtPartitionsExchangeFuture.java:54*
error:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartition
sExchangeFuture.java: patch does not apply



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteCacheProxy-connection-failure-in-REPLICATAED-mode-tp11769p12030.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IgniteCacheProxy connection failure in REPLICATAED mode

Posted by rick_tem <rv...@temenos.com>.
Thanks Andrey.  I'll figure out how to patch the branch and give it a try.

Appreciate it.

Best,
Rick




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteCacheProxy-connection-failure-in-REPLICATAED-mode-tp11769p11819.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IgniteCacheProxy connection failure in REPLICATAED mode

Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Rick,

Looks like it is fixed [1] in master and will be available in 2.0 version.
You can try to apply a patch to 1.9 version. PFA patch.

[1] https://issues.apache.org/jira/browse/IGNITE-4473

On Thu, Apr 6, 2017 at 12:05 PM, rick_tem <rv...@temenos.com> wrote:

> Hi,
>
> I have a cache configuration that looks like this:
>
>                                 <bean class="org.apache.ignite.
> configuration.CacheConfiguration">
>                                         <property name="name"
> value="MCache" />
>                                         <property name="cacheMode"
> value="REPLICATED" />
>                                         <property name="memoryMode"
> value="ONHEAP_TIERED" />
>
>                                         <property name="offHeapMaxMemory"
> value="#{10 * 1024L * 1024L * 1024L}"
> />
>                                         <property name="evictionPolicy">
>
>                                                 <bean
> class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
>
>                                                         <property
> name="maxSize" value="1000000" />
>                                                 </bean>
>                                         </property>
>                                         <property name="atomicityMode"
> value="TRANSACTIONAL" />
>                                         <property name="backups" value="1"
> />
>                                         <property name="writeSynchronizationMode"
> value="FULL_SYNC" />
>                                         <property name="statisticsEnabled"
> value="true" />
>                                 </bean>
>
> I've started two Ignite servers and within JBOSS application server set
> Ignition.setClientMode(true); such that I have 2 servers and 1 client
> (JBOSS) shown in my topology.  When I then terminate one of the Ignite
> servers, I receive the following exception below when my JBOSS app is
> manipulating the cache.  If I am in REPLICATED mode, why doesn't the Ignite
> proxy try the other server automatically on connection failure?  Is there a
> mechanism I should use to protect against connection failure, or how should
> I ensure (in this case a remove operation on the cache) my actions complete
> successfully?
>
> Thanks!
> Rick
>
> Caused by: org.apache.ignite.IgniteClientDisconnectedException: Client
> node
> disconnected: TGrid
>         ... 88 more
> [ERROR] 2017-04-05 14:11:35,675 [tSA 4_1037028145] LOCKING - IgniteCache
> Error
> javax.cache.CacheException: class
> org.apache.ignite.IgniteClientDisconnectedException: Client node
> disconnected: TGrid
>         at
> org.apache.ignite.internal.processors.cache.GridCacheGateway.checkState(
> GridCacheGateway.java:92)
> ~[ignite-core-1.9.0.jar!/:1.9.0]
>         at
> org.apache.ignite.internal.processors.cache.GridCacheGateway.enter(
> GridCacheGateway.java:173)
> ~[ignite-core-1.9.0.jar!/:1.9.0]
>         at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.onEnter(
> IgniteCacheProxy.java:2264)
> ~[ignite-core-1.9.0.jar!/:1.9.0]
>         at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.remove(
> IgniteCacheProxy.java:1463)
> ~[ignite-core-1.9.0.jar!/:1.9.0]
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/IgniteCacheProxy-connection-
> failure-in-REPLICATAED-mode-tp11769.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Best regards,
Andrey V. Mashenkov

Re: IgniteCacheProxy connection failure in REPLICATAED mode

Posted by Baskar Sikkayan <ba...@gmail.com>.
unsubscribe

On Thu, Apr 6, 2017 at 2:05 AM, rick_tem <rv...@temenos.com> wrote:

> Hi,
>
> I have a cache configuration that looks like this:
>
>                                 <bean class="org.apache.ignite.
> configuration.CacheConfiguration">
>                                         <property name="name"
> value="MCache" />
>                                         <property name="cacheMode"
> value="REPLICATED" />
>                                         <property name="memoryMode"
> value="ONHEAP_TIERED" />
>
>                                         <property name="offHeapMaxMemory"
> value="#{10 * 1024L * 1024L * 1024L}"
> />
>                                         <property name="evictionPolicy">
>
>                                                 <bean
> class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
>
>                                                         <property
> name="maxSize" value="1000000" />
>                                                 </bean>
>                                         </property>
>                                         <property name="atomicityMode"
> value="TRANSACTIONAL" />
>                                         <property name="backups" value="1"
> />
>                                         <property name="writeSynchronizationMode"
> value="FULL_SYNC" />
>                                         <property name="statisticsEnabled"
> value="true" />
>                                 </bean>
>
> I've started two Ignite servers and within JBOSS application server set
> Ignition.setClientMode(true); such that I have 2 servers and 1 client
> (JBOSS) shown in my topology.  When I then terminate one of the Ignite
> servers, I receive the following exception below when my JBOSS app is
> manipulating the cache.  If I am in REPLICATED mode, why doesn't the Ignite
> proxy try the other server automatically on connection failure?  Is there a
> mechanism I should use to protect against connection failure, or how should
> I ensure (in this case a remove operation on the cache) my actions complete
> successfully?
>
> Thanks!
> Rick
>
> Caused by: org.apache.ignite.IgniteClientDisconnectedException: Client
> node
> disconnected: TGrid
>         ... 88 more
> [ERROR] 2017-04-05 14:11:35,675 [tSA 4_1037028145] LOCKING - IgniteCache
> Error
> javax.cache.CacheException: class
> org.apache.ignite.IgniteClientDisconnectedException: Client node
> disconnected: TGrid
>         at
> org.apache.ignite.internal.processors.cache.GridCacheGateway.checkState(
> GridCacheGateway.java:92)
> ~[ignite-core-1.9.0.jar!/:1.9.0]
>         at
> org.apache.ignite.internal.processors.cache.GridCacheGateway.enter(
> GridCacheGateway.java:173)
> ~[ignite-core-1.9.0.jar!/:1.9.0]
>         at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.onEnter(
> IgniteCacheProxy.java:2264)
> ~[ignite-core-1.9.0.jar!/:1.9.0]
>         at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.remove(
> IgniteCacheProxy.java:1463)
> ~[ignite-core-1.9.0.jar!/:1.9.0]
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/IgniteCacheProxy-connection-
> failure-in-REPLICATAED-mode-tp11769.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>