You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Thomas Isaksen <th...@sysco.no> on 2018/01/25 13:02:41 UTC

Failed to activate cluster - table already exists

I simply stopped my cluster and started it again, when I try to activate I get the following:

[12:50:26,308][SEVERE][sys-#38][GridTaskWorker] Failed to obtain remote job result policy for result from ComputeTask.result(..) method (will fail the whole task): GridJobResultImpl [job=C4 [r=o.a.i.i.processors.cluster.GridClusterStateProcessor$ClientChangeGlobalStateComputeRequest@4263b080], sib=GridJobSiblingImpl [sesId=ab5772d2161-6e54ac8c-8405-459f-ba0b-ce7795286781, jobId=cb5772d2161-6e54ac8c-8405-459f-ba0b-ce7795286781, nodeId=0aee4fec-e006-420f-bdc6-4a9ca85cee18, isJobDone=false], jobCtx=GridJobContextImpl [jobId=cb5772d2161-6e54ac8c-8405-459f-ba0b-ce7795286781, timeoutObj=null, attrs={}], node=TcpDiscoveryNode [id=0aee4fec-e006-420f-bdc6-4a9ca85cee18, addrs=[0:0:0:0:0:0:0:1, 10.108.192.88, 127.0.0.1], sockAddrs=[/10.108.192.88:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500], discPort=47500, order=161, intOrder=82, lastExchangeTime=1516881026101, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], ex=class o.a.i.IgniteException: Failed to activate cluster., hasRes=true, isCancelled=false, isOccupied=true]
class org.apache.ignite.IgniteException: Remote job threw user exception (override or implement ComputeTask.result(..) method if you would like to have automatic failover for this exception).
                at org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:101)
                at org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1047)
                at org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1040)
                at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6663)
                at org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:1040)
                at org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:858)
                at org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1066)
                at org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1301)
                at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
                at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
                at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
                at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.IgniteException: Failed to activate cluster.
                at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:966)
                at org.apache.ignite.internal.IgniteKernal.active(IgniteKernal.java:3513)
                at org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor$ClientChangeGlobalStateComputeRequest.run(GridClusterStateProcessor.java:908)
                at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C4.execute(GridClosureProcessor.java:1944)
                at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566)
                at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6631)
                at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560)
                at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489)
                at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
                at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1181)
                at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1913)
                at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
                at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
                at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
                at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
                at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to activate cluster.
                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStateChangeRequest(ClusterCachesInfo.java:1233)
                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onStateChangeRequest(GridCacheProcessor.java:2276)
                at org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.onStateChangeMessage(GridClusterStateProcessor.java:260)
                at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.onDiscovery0(GridDiscoveryManager.java:653)
                at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.onDiscovery(GridDiscoveryManager.java:571)
                at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:5463)
                at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5282)
                at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2656)
                at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2447)
                at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6648)
                at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2533)
                at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
                Suppressed: SchemaOperationException [code=3, msg=Table already exists: USERCACHE]
                               at org.apache.ignite.internal.processors.query.QueryUtils.checkQueryEntityConflicts(QueryUtils.java:1101)
                               at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.processCacheChangeRequests(ClusterCachesInfo.java:507)
                               at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStateChangeRequest(ClusterCachesInfo.java:1230)
                               ... 11 more

class org.apache.ignite.IgniteException: Failed to activate cluster.

                at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:966)
                at org.apache.ignite.internal.IgniteKernal.active(IgniteKernal.java:3513)
                at no.toyota.gatekeeper.test.TokenFactoryTest.getCache(TokenFactoryTest.java:351)
                at no.toyota.gatekeeper.test.TokenFactoryTest.testIndexedTypes(TokenFactoryTest.java:301)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.lang.reflect.Method.invoke(Method.java:498)
                at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
                at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
                at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
                at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
                at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
                at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
                at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
                at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
                at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
                at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
                at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
                at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
                at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
                at org.junit.runners.Suite.runChild(Suite.java:128)
                at org.junit.runners.Suite.runChild(Suite.java:27)
                at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
                at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
                at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
                at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
                at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
                at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
                at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
                at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
                at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
                at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
                at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to activate cluster.
                at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStateChangeRequest(ClusterCachesInfo.java:1233)
                at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onStateChangeRequest(GridCacheProcessor.java:2276)
                at org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.onStateChangeMessage(GridClusterStateProcessor.java:260)
                at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.onDiscovery0(GridDiscoveryManager.java:653)
                at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.onDiscovery(GridDiscoveryManager.java:571)
                at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:5463)
                at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5282)
                at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2656)
                at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2447)
                at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6648)
                at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2533)
                at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
                Suppressed: SchemaOperationException [code=3, msg=Table already exists: USERCACHE]
                               at org.apache.ignite.internal.processors.query.QueryUtils.checkQueryEntityConflicts(QueryUtils.java:1101)
                               at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.processCacheChangeRequests(ClusterCachesInfo.java:507)
                               at org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStateChangeRequest(ClusterCachesInfo.java:1230)
                               ... 11 more


Re: Failed to activate cluster - table already exists

Posted by Thomas Isaksen <th...@sysco.no>.
Hi, I will share next time for sure. I tend to break stuff so I'm guessing I will do it again. As far as I can remember I didn't change anything before starting again.

--
Thomas Isaksen
________________________________
From: Michael Cherkasov <mi...@gmail.com>
Sent: Friday, January 26, 2018 8:48:58 PM
To: user@ignite.apache.org
Subject: Re: Failed to activate cluster - table already exists

Hi Thomas,

Please share with us next time a reproducer with work folder.
It's something that should be checked.

Did you change your QueryEntities between cluster restart?

Thanks,
Mike.

2018-01-26 5:25 GMT-08:00 Thomas Isaksen <th...@sysco.no>>:
Hi Mikhail,

I don't know what happened but I deleted some folders under $IGNITE_HOME/work/db with the same name and the problem cleared.
I think maybe I killed Ignite before it could finish writing or something to that effect, which could have caused the problem.

./t

-----Original Message-----
From: Mikhail [mailto:michael.cherkasov@gmail.com<ma...@gmail.com>]
Sent: fredag 26. januar 2018 01.51
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: Re: Failed to activate cluster - table already exists

Hi Thomas,

Looks like you can reproduce the issue with a unit test.

Could you please share it with us?

Thanks,
Mike.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


Re: Failed to activate cluster - table already exists

Posted by Michael Cherkasov <mi...@gmail.com>.
Hi Thomas,

Please share with us next time a reproducer with work folder.
It's something that should be checked.

Did you change your QueryEntities between cluster restart?

Thanks,
Mike.

2018-01-26 5:25 GMT-08:00 Thomas Isaksen <th...@sysco.no>:

> Hi Mikhail,
>
> I don't know what happened but I deleted some folders under
> $IGNITE_HOME/work/db with the same name and the problem cleared.
> I think maybe I killed Ignite before it could finish writing or something
> to that effect, which could have caused the problem.
>
> ./t
>
> -----Original Message-----
> From: Mikhail [mailto:michael.cherkasov@gmail.com]
> Sent: fredag 26. januar 2018 01.51
> To: user@ignite.apache.org
> Subject: Re: Failed to activate cluster - table already exists
>
> Hi Thomas,
>
> Looks like you can reproduce the issue with a unit test.
>
> Could you please share it with us?
>
> Thanks,
> Mike.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

RE: Failed to activate cluster - table already exists

Posted by Thomas Isaksen <th...@sysco.no>.
Hi Mikhail,

I don't know what happened but I deleted some folders under $IGNITE_HOME/work/db with the same name and the problem cleared.
I think maybe I killed Ignite before it could finish writing or something to that effect, which could have caused the problem.

./t

-----Original Message-----
From: Mikhail [mailto:michael.cherkasov@gmail.com] 
Sent: fredag 26. januar 2018 01.51
To: user@ignite.apache.org
Subject: Re: Failed to activate cluster - table already exists

Hi Thomas,

Looks like you can reproduce the issue with a unit test.

Could you please share it with us?

Thanks,
Mike.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Failed to activate cluster - table already exists

Posted by Mikhail <mi...@gmail.com>.
Hi Thomas,

Looks like you can reproduce the issue with a unit test.

Could you please share it with us?

Thanks,
Mike.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/