You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Denis Mekhanikov <dm...@gmail.com> on 2018/09/14 14:51:23 UTC
Re: Service not found for a deployed service
Did you manage to reproduce the problem?
Denis
пн, 6 авг. 2018 г. в 14:39, Calvin KL Wong, CLSA <ca...@clsa.com>:
> Hi Denis,
>
>
>
> >> Did Service#init() method throw any exceptions?
>
> No, we didn’t see any exception.
>
>
>
> I will get a stack trace if I can reproduce it.
>
>
>
> Thanks,
>
> Calvin
>
>
>
> *From:* Denis Mekhanikov [mailto:dmekhanikov@gmail.com]
> *Sent:* Friday, August 03, 2018 8:08 PM
>
>
> *To:* user@ignite.apache.org
> *Subject:* Re: Service not found for a deployed service
>
>
>
> Calvin,
>
>
>
> Did Service#init() method throw any exceptions?
>
> If so, then you would see the same problem.
>
>
>
> Denis
>
>
>
> пт, 3 авг. 2018 г. в 14:13, Calvin KL Wong, CLSA <calvin.kl.wong@clsa.com
> >:
>
> Actually, we deployed the service at 02:00, and didn’t use the service
> until 16:27 the next day. Then got the error. There were more than 12
> hours in between.
>
> Does this still seem related to IGNITE-1478
> <https://issues.apache.org/jira/browse/IGNITE-1478> ?
>
> If not, can you think of other possible reason?
>
>
>
> Thanks,
>
> Calvin
>
>
>
> *From:* Denis Mekhanikov [mailto:dmekhanikov@gmail.com]
> *Sent:* Thursday, August 02, 2018 11:45 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: Service not found for a deployed service
>
>
>
> Calvin,
>
>
>
> You have this problem due to the following issue: IGNITE-1478
> <https://issues.apache.org/jira/browse/IGNITE-1478>
>
> A workaround here would be to retry method execution with some delay.
>
>
>
> This problem should be fixed under IEP-17
> <https://cwiki.apache.org/confluence/display/IGNITE/IEP-17%3A+Oil+Change+in+Service+Grid>,
> which is in progress right now.
>
>
>
> Denis
>
>
>
> чт, 2 авг. 2018 г. в 14:33, Calvin KL Wong, CLSA <calvin.kl.wong@clsa.com
> >:
>
> Hi,
>
>
>
> I deployed a service from a client node to our grid using the following
> code:
>
>
>
> IgniteCluster cluster = ignite.cluster();
>
> ClusterGroup group = cluster.forAttribute(…);
>
> Ignite.services(workerGroup).deployClusterSingleton(“blaze/hsbc”)
>
>
>
> It is fine most of the time. However we just encountered a case where we
> got an exception when some logic tried to use this service:
>
>
>
> 2018-08-02 16:27:57.713 processors.task.GridTaskWorker [sys-#29%mlog%]
> ERROR - Failed to obtain remote job result policy for result from
> ComputeTask.result(..) method (will fail the whole task): GridJobResultImpl
> [job=C2 [c=ServiceProxyCallable [mtdName=execute, svcName=blaze/hsbc,
> ignite=null]], sib=GridJobSiblingImpl
> [sesId=f66f54be461-65c907a3-8fcf-4ddd-acb1-6553be3d1dc9,
> jobId=076f54be461-65c907a3-8fcf-4ddd-acb1-6553be3d1dc9,
> nodeId=236a47e9-7fdb-464e-be44-b24d0942d75c, isJobDone=false],
> jobCtx=GridJobContextImpl
> [jobId=076f54be461-65c907a3-8fcf-4ddd-acb1-6553be3d1dc9, timeoutObj=null,
> attrs={}], node=TcpDiscoveryNode [id=236a47e9-7fdb-464e-be44-b24d0942d75c,
> addrs=[10.23.8.165], sockAddrs=[zhkdlp1712.int.clsa.com/10.23.8.165:0],
> discPort=0, order=37, intOrder=27, lastExchangeTime=1533148447088,
> loc=false, ver=2.3.0#20180518-sha1:02cf6abf, isClient=true], ex=class
> o.a.i.IgniteException: Service not found: blaze/hsbc, hasRes=true,
> isCancelled=false, isOccupied=true]
>
> org.apache.ignite.IgniteException: Remote job threw user exception
> (override or implement ComputeTask.result(..) method if you would like to
> have automatic failover for this exception).
>
> at
> org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:101)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1047)
> [liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1040)
> [liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6663)
> [liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:1040)
> [liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:858)
> [liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1066)
> [liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1301)
> [liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
> [liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
> [liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
> [liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
> [liquid-logic.jar:2.0.10]
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [?:1.8.0_121]
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [?:1.8.0_121]
>
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
>
> Caused by: org.apache.ignite.IgniteException: Service not found: blaze/hsbc
>
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1858)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6631)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1181)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1913)
> ~[liquid-logic.jar:2.0.10]
>
> ... 7 more
>
> Caused by:
> org.apache.ignite.internal.processors.service.GridServiceNotFoundException:
> Service not found: blaze/hsbc
>
> at
> org.apache.ignite.internal.processors.service.GridServiceProxy$ServiceProxyCallable.call(GridServiceProxy.java:408)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1855)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6631)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1181)
> ~[liquid-logic.jar:2.0.10]
>
> at
> org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1913)
> ~[liquid-logic.jar:2.0.10]
>
> ... 7 more
>
>
>
> And that’s the end of my stack trace (nothing can be found after the … 7
> more.)
>
> The above exception got propagated back to the client which deployed the
> service. Do you know of any reason for this to happen, like why a deployed
> service will disappear from the grid? And is there any event I can use to
> detect this?
>
>
>
> Thanks,
>
> Calvin
>
>
>
> *Calvin KL Wong*
>
> *Sr. Lead Engineer, Execution Services*
>
> D +852 2600 7983 <+852%202600%207983> | M +852 9267 9471
> <+852%209267%209471> | T +852 2600 8888 <+852%202600%208888>
>
> 5/F, One Island East, 18 Westlands Road
> <https://maps.google.com/?q=18+Westlands+Road&entry=gmail&source=g>,
> Island East, Hong Kong
>
>
>
> *Error! Filename not specified.* <https://hk.linkedin.com/company/clsa>*Error!
> Filename not specified.* <https://twitter.com/clsainsights?lang=en>*Error!
> Filename not specified.*
> <https://www.youtube.com/channel/UC0qWp_lLnOcRYmBlCNQgZKA>*Error!
> Filename not specified.* <https://www.facebook.com/clsacommunity/>
>
>
>
> *clsa.com* <https://www.clsa.com/>
>
> *Insights. Liquidity. Capital. *
>
>
>
> *Error! Filename not specified.* <https://www.clsa.com/member>
>
>
>
> *A CITIC Securities Company*
>
>
>
> The content of this communication is intended for the recipient and is
> subject to CLSA Legal and Regulatory Notices.
> These can be viewed at https://www.clsa.com/disclaimer.html or sent to
> you upon request.
> Please consider before printing. CLSA is ISO14001 certified and committed
> to reducing its impact on the environment.
>
> The content of this communication is intended for the recipient and is
> subject to CLSA Legal and Regulatory Notices.
> These can be viewed at https://www.clsa.com/disclaimer.html or sent to
> you upon request.
> Please consider before printing. CLSA is ISO14001 certified and committed
> to reducing its impact on the environment.
>
> The content of this communication is intended for the recipient and is
> subject to CLSA Legal and Regulatory Notices.
> These can be viewed at https://www.clsa.com/disclaimer.html or sent to
> you upon request.
> Please consider before printing. CLSA is ISO14001 certified and committed
> to reducing its impact on the environment.
>
Re: Service not found for a deployed service
Posted by Vyacheslav Daradur <da...@gmail.com>.
Hi!
> I deployed a service from a client node to our grid using the following code:
> IgniteCluster cluster = ignite.cluster();
> ClusterGroup group = cluster.forAttribute(…);
> Ignite.services(workerGroup).deployClusterSingleton(“blaze/hsbc”)
Calvin, please, pay attention to your code and make sure that you
define ClusterGroup in right way.
I think the problem may be in the empty cluster group, that means
Ignite can't define suitable nodes to deploy service.
On Fri, Sep 14, 2018 at 5:51 PM Denis Mekhanikov <dm...@gmail.com> wrote:
>
> Did you manage to reproduce the problem?
>
> Denis
>
> пн, 6 авг. 2018 г. в 14:39, Calvin KL Wong, CLSA <ca...@clsa.com>:
>>
>> Hi Denis,
>>
>>
>>
>> >> Did Service#init() method throw any exceptions?
>>
>> No, we didn’t see any exception.
>>
>>
>>
>> I will get a stack trace if I can reproduce it.
>>
>>
>>
>> Thanks,
>>
>> Calvin
>>
>>
>>
>> From: Denis Mekhanikov [mailto:dmekhanikov@gmail.com]
>> Sent: Friday, August 03, 2018 8:08 PM
>>
>>
>> To: user@ignite.apache.org
>> Subject: Re: Service not found for a deployed service
>>
>>
>>
>> Calvin,
>>
>>
>>
>> Did Service#init() method throw any exceptions?
>>
>> If so, then you would see the same problem.
>>
>>
>>
>> Denis
>>
>>
>>
>> пт, 3 авг. 2018 г. в 14:13, Calvin KL Wong, CLSA <ca...@clsa.com>:
>>
>> Actually, we deployed the service at 02:00, and didn’t use the service until 16:27 the next day. Then got the error. There were more than 12 hours in between.
>>
>> Does this still seem related to IGNITE-1478 ?
>>
>> If not, can you think of other possible reason?
>>
>>
>>
>> Thanks,
>>
>> Calvin
>>
>>
>>
>> From: Denis Mekhanikov [mailto:dmekhanikov@gmail.com]
>> Sent: Thursday, August 02, 2018 11:45 PM
>> To: user@ignite.apache.org
>> Subject: Re: Service not found for a deployed service
>>
>>
>>
>> Calvin,
>>
>>
>>
>> You have this problem due to the following issue: IGNITE-1478
>>
>> A workaround here would be to retry method execution with some delay.
>>
>>
>>
>> This problem should be fixed under IEP-17, which is in progress right now.
>>
>>
>>
>> Denis
>>
>>
>>
>> чт, 2 авг. 2018 г. в 14:33, Calvin KL Wong, CLSA <ca...@clsa.com>:
>>
>> Hi,
>>
>>
>>
>> I deployed a service from a client node to our grid using the following code:
>>
>>
>>
>> IgniteCluster cluster = ignite.cluster();
>>
>> ClusterGroup group = cluster.forAttribute(…);
>>
>> Ignite.services(workerGroup).deployClusterSingleton(“blaze/hsbc”)
>>
>>
>>
>> It is fine most of the time. However we just encountered a case where we got an exception when some logic tried to use this service:
>>
>>
>>
>> 2018-08-02 16:27:57.713 processors.task.GridTaskWorker [sys-#29%mlog%] ERROR - Failed to obtain remote job result policy for result from ComputeTask.result(..) method (will fail the whole task): GridJobResultImpl [job=C2 [c=ServiceProxyCallable [mtdName=execute, svcName=blaze/hsbc, ignite=null]], sib=GridJobSiblingImpl [sesId=f66f54be461-65c907a3-8fcf-4ddd-acb1-6553be3d1dc9, jobId=076f54be461-65c907a3-8fcf-4ddd-acb1-6553be3d1dc9, nodeId=236a47e9-7fdb-464e-be44-b24d0942d75c, isJobDone=false], jobCtx=GridJobContextImpl [jobId=076f54be461-65c907a3-8fcf-4ddd-acb1-6553be3d1dc9, timeoutObj=null, attrs={}], node=TcpDiscoveryNode [id=236a47e9-7fdb-464e-be44-b24d0942d75c, addrs=[10.23.8.165], sockAddrs=[zhkdlp1712.int.clsa.com/10.23.8.165:0], discPort=0, order=37, intOrder=27, lastExchangeTime=1533148447088, loc=false, ver=2.3.0#20180518-sha1:02cf6abf, isClient=true], ex=class o.a.i.IgniteException: Service not found: blaze/hsbc, hasRes=true, isCancelled=false, isOccupied=true]
>>
>> org.apache.ignite.IgniteException: Remote job threw user exception (override or implement ComputeTask.result(..) method if you would like to have automatic failover for this exception).
>>
>> at org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:101) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1047) [liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1040) [liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6663) [liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:1040) [liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:858) [liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1066) [liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1301) [liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555) [liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183) [liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126) [liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090) [liquid-logic.jar:2.0.10]
>>
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
>>
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
>>
>> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
>>
>> Caused by: org.apache.ignite.IgniteException: Service not found: blaze/hsbc
>>
>> at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1858) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6631) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1181) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1913) ~[liquid-logic.jar:2.0.10]
>>
>> ... 7 more
>>
>> Caused by: org.apache.ignite.internal.processors.service.GridServiceNotFoundException: Service not found: blaze/hsbc
>>
>> at org.apache.ignite.internal.processors.service.GridServiceProxy$ServiceProxyCallable.call(GridServiceProxy.java:408) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1855) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6631) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1181) ~[liquid-logic.jar:2.0.10]
>>
>> at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1913) ~[liquid-logic.jar:2.0.10]
>>
>> ... 7 more
>>
>>
>>
>> And that’s the end of my stack trace (nothing can be found after the … 7 more.)
>>
>> The above exception got propagated back to the client which deployed the service. Do you know of any reason for this to happen, like why a deployed service will disappear from the grid? And is there any event I can use to detect this?
>>
>>
>>
>> Thanks,
>>
>> Calvin
>>
>>
>>
>> Calvin KL Wong
>>
>> Sr. Lead Engineer, Execution Services
>>
>> D +852 2600 7983 | M +852 9267 9471 | T +852 2600 8888
>>
>> 5/F, One Island East, 18 Westlands Road, Island East, Hong Kong
>>
>>
>>
>> Error! Filename not specified.Error! Filename not specified.Error! Filename not specified.Error! Filename not specified.
>>
>>
>>
>> clsa.com
>>
>> Insights. Liquidity. Capital.
>>
>>
>>
>> Error! Filename not specified.
>>
>>
>>
>> A CITIC Securities Company
>>
>>
>>
>> The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
>> These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
>> Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
>>
>> The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
>> These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
>> Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
>>
>> The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
>> These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
>> Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
--
Best Regards, Vyacheslav D.