You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@karaf.apache.org by "Jean-Baptiste Onofré (JIRA)" <ji...@apache.org> on 2017/05/10 14:40:04 UTC

[jira] [Updated] (KARAF-4878) Cellar Hazelcast unresponsive when ETH Down

     [ https://issues.apache.org/jira/browse/KARAF-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jean-Baptiste Onofré updated KARAF-4878:
----------------------------------------
    Fix Version/s: cellar-4.1.0
                   cellar-4.0.4

> Cellar Hazelcast unresponsive when ETH Down
> -------------------------------------------
>
>                 Key: KARAF-4878
>                 URL: https://issues.apache.org/jira/browse/KARAF-4878
>             Project: Karaf
>          Issue Type: Bug
>          Components: cellar-hazelcast
>    Affects Versions: 4.0.5
>         Environment: Redhat Linux 7.2, CentOS 7.2
>            Reporter: Suresh Perumal
>            Assignee: Jean-Baptiste Onofré
>            Priority: Blocker
>             Fix For: cellar-4.0.4, cellar-4.1.0
>
>
> Cluster is configured with 2 Nodes. They are up and running.
> As part of fail-over scenario simulation. We are trying to test "ETHERNET down scenario" by running "/etc/sysconfig/network-scripts/ifdown eth0" command on the first node.
> During this scenario we are shutting down the first node where the ETH is  down by using monitoring scripts(in-house scripts). The second node(Among those two nodes) is kept alive.
> Second Node's Hazelcast is not accessible for more than 15 minutes. We are getting bellow exception and no operation related to Hazelcast is working. Applications whichever uses hazelcast kept frozen.
> Invocation                       | 52 - com.hazelcast - 3.5.2 | [10.249.50.80]:5701 [cellar] [3.5.2] While asking 'is-executing': Invocation{ serviceName='hz:impl:mapService', op=PutOperation{unacknowledged-alarm}, partitionId=165, replicaIndex=0, tryCount=250, tryPauseMillis=500, invokeCount=1, callTimeout=60000, target=Address[10.249.50.79]:5701, backupsExpected=0, backupsCompleted=0}
> java.util.concurrent.TimeoutException: Call Invocation{ serviceName='hz:impl:mapService', op=com.hazelcast.spi.impl.operationservice.impl.operations.IsStillExecutingOperation{serviceName='hz:impl:mapService', partitionId=-1, callId=2114, invocationTime=1480511190143, waitTimeout=-1, callTimeout=5000}, partitionId=-1, replicaIndex=0, tryCount=0, tryPauseMillis=0, invokeCount=1, callTimeout=5000, target=Address[10.249.50.79]:5701, backupsExpected=0, backupsCompleted=0} encountered a timeout
>         at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveApplicationResponse(InvocationFuture.java:366)[52:com.hazelcast:3.5.2]
>         at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveApplicationResponseOrThrowException(InvocationFuture.java:334)[52:com.hazelcast:3.5.2]
>         at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.get(InvocationFuture.java:225)[52:com.hazelcast:3.5.2]
>         at com.hazelcast.spi.impl.operationservice.impl.IsStillRunningService.isOperationExecuting(IsStillRunningService.java:85)[52:com.hazelcast:3.5.2]
>         at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.waitForResponse(InvocationFuture.java:275)[52:com.hazelcast:3.5.2]
>         at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.get(InvocationFuture.java:224)[52:com.hazelcast:3.5.2]
>         at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.get(InvocationFuture.java:204)[52:com.hazelcast:3.5.2]
>         at com.hazelcast.map.impl.proxy.MapProxySupport.invokeOperation(MapProxySupport.java:456)[52:com.hazelcast:3.5.2]
>         at com.hazelcast.map.impl.proxy.MapProxySupport.putInternal(MapProxySupport.java:417)[52:com.hazelcast:3.5.2]
>         at com.hazelcast.map.impl.proxy.MapProxyImpl.put(MapProxyImpl.java:97)[52:com.hazelcast:3.5.2]
>         at com.hazelcast.map.impl.proxy.MapProxyImpl.put(MapProxyImpl.java:87)[52:com.hazelcast:3.5.2]
>         at com.fujitsu.fnc.emf.fpmplatform.cachemanager.HazelcastCacheManagerMapServiceImpl.addToMap(HazelcastCacheManagerMapServiceImpl.java:87)[209:FPMHazelcastCache:4.1.0.SNAPSHOT]
>         at Proxy1897a82c_c032_4a5c_9839_e71cb2af452a.addToMap(Unknown Source)[:]
>         at com.fujitsu.fnc.ngemf.fm.server.impl.FpmConsumerTask.prepareJSON(FpmConsumerTask.java:151)[235:com.fujitsu.fnc.ngemf.fm.server.impl:4.1.0.SNAPSHOT]
>         at com.fujitsu.fnc.ngemf.fm.server.impl.FpmConsumerTask.run(FpmConsumerTask.java:244)[235:com.fujitsu.fnc.ngemf.fm.server.impl:4.1.0.SNAPSHOT]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_66]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_66]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_66]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_66]
>         at java.lang.Thread.run(Thread.java:745)[:1.8.0_66]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)