You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airavata.apache.org by DImuthu Upeksha <di...@gmail.com> on 2017/11/16 18:01:58 UTC
Using job specific Participants in Helix
Hi Devs,
I'm working on the technology evaluation to re architecture Apache Airavata
task execution framework and Helix seems like a good candidate for that as
it has an in built distributed generic workflow execution capability. After
going through several tutorials, I tried to implement a simple workflow on
Helix to demonstrate following transition
Data Collecting Job -> Command Executing Job -> Data Uploading Job
I managed to implement this using a Participant node that includes all the
tasks required for above workflow. However my goal is to implement
specialized Participants for each Job type. For example, Participant 1
knows only about the tasks to perform Data Collecting Job and Participant
knows only about the task to perform Command Executing Job. When I tried to
implement such Participants, I got following error from Participant 1. I
can share the code samples that I have tried but before that I need to know
whether my approach is compatible with Helix's design? Does Helix require
all the Participants to be homogeneous?
Executing data collecting
675892 [pool-2-thread-8] ERROR
org.apache.helix.messaging.handling.HelixStateTransitionHandler -
Exception while executing a state transition task
Airavata_Workflow_commandExecuteJob_0
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:344)
at
org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:290)
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:84)
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:41)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: No callback implemented for
task COMMAND
at org.apache.helix.task.TaskStateModel.startTask(TaskStateModel.java:302)
at
org.apache.helix.task.TaskStateModel.onBecomeRunningFromInit(TaskStateModel.java:69)
... 12 more
675900 [pool-2-thread-8] ERROR
org.apache.helix.participant.statemachine.StateModel - Default rollback
method invoked on error. Error Code: ERROR
675908 [pool-2-thread-8] ERROR
org.apache.helix.messaging.handling.HelixTask - Message execution failed.
msgId: 8d9a08c9-0ae7-419a-8940-6b8e6e7c1ecb, errorMsg:
java.lang.reflect.InvocationTargetException
675922 [pool-2-thread-8] ERROR
org.apache.helix.messaging.handling.HelixStateTransitionHandler - Skip
internal error. errCode: ERROR, errMsg: null
Thanks
Dimuthu
Re: Using job specific Participants in Helix
Posted by DImuthu Upeksha <di...@gmail.com>.
Hi Junkai
That did the trick. Thanks a lot
Thanks
Dimuthu
On 17 Nov 2017 10:32, "Xue Junkai" <jx...@apache.org> wrote:
Hi Dlmuthu and Kishore,
Yes. Task framework does support tagged instance at job level. It can be
set in JobConfig for InstanceGroupTag. Also, do not forget to tag the
instances.
Best,
Junkai
On Thu, Nov 16, 2017 at 8:50 PM, kishore g <g....@gmail.com> wrote:
> Hi Dlmuthu,
>
> You can achieve this by using the tag feature in Helix. For example, to
> Participants P1, P2 should only handle DataCollecting tasks, you need to
do
> the following
>
> - Tag Participant P1 and P2 as "DataCollecting"
> - When you create a job resource that represents DataCollecting task,
> you have to tag the resource also as DataCollecting.
>
> Helix will then assign all tasks in this job to nodes that are tagged as
> DataCollecting.
>
> Note this is available only in FULL_AUTO rebalance mode. I am not sure if
> the rebalancer in task framework supports this feature.
>
> Lei/Junkai, do you know if this is supported in Task Framework?
>
> thanks
>
>
> On Thu, Nov 16, 2017 at 8:31 PM, Ajinkya Dhamnaskar <adhamnas@umail.iu.edu
> >
> wrote:
>
> > Dlmuthu,
> >
> > This explains your need. Could you please point me to the code? I was
> > wondering, where and how are we registering callbacks for the
> participants?
> >
> > On Thu, Nov 16, 2017 at 7:16 PM, DImuthu Upeksha <
> > dimuthu.upeksha2@gmail.com
> > > wrote:
> >
> > > Adding Helix Dev
> > >
> > > Hi Ajinkya
> > >
> > > Thank you for the explanation.
> > >
> > > Let me explain my requirement. [1] is the correct task registry
> > > configuration in a working participant code. All the transition call
> > backs
> > > are registered in this code. The problem here is, we have to bundle
> > > binaries of all the tasks into a single Participant. If we want to
> change
> > > one task, we need to rebuild the participant with other tasks as well.
> > What
> > > I thought is, why can't we build Participants that only do specific
set
> > of
> > > tasks. For example, in the cluster there are 3 participants,
> Participant
> > 1
> > > [2], Participant 2 [3] and Participant 3 [4]. If we want to change the
> > > DataCollecting task, we only need to re build Participant 2. I got
> above
> > > error, once I run 3 participants in above configuration. I can
> understand
> > > this is due to the missing transition callback of CommandTask in
> > > Participant 1 as I have purposely commented out it. What I need to
know
> > is
> > > that, is this configuration allowed in Helix architecture or do we
have
> > to
> > > implement Participants that contain all the task implementations as in
> > [1].
> > > Workflow configuration for both scenarios is here [5].
> > >
> > > [1] https://gist.github.com/DImuthuUpe/41c0db579e7d86d101d112f07ed6ea
> 00
> > > [2] https://gist.github.com/DImuthuUpe/ec72df1ec3207ce2dce88ff7f1756d
> a4
> > > [3] https://gist.github.com/DImuthuUpe/4d193a3dff3008315efa2e31f6209c
> ac
> > > [4] https://gist.github.com/DImuthuUpe/872c432935b8d33944dd571b3ac420
> 7b
> > > [5] https://gist.github.com/DImuthuUpe/f61851b68b685b8d6744689dc130ba
> bd
> > >
> > > Thanks
> > > Dimuthu
> > >
> > > On Fri, Nov 17, 2017 at 3:21 AM, Ajinkya Dhamnaskar <
> > adhamnas@umail.iu.edu
> > > > wrote:
> > >
> > >> Hey Dlmuthu,
> > >>
> > >> Not an expert in Helix, but from exceptions it seems, system is
> entering
> > >> in a state not expected by reflection. I feel
> https://github.com/apache
> > >> /helix/blob/master/helix-core/src/main/java/org/apache/helix
> > >> /messaging/handling/HelixStateTransitionHandler.java#L295 is
> triggering
> > >> this exception.
> > >> As mentioned in the later part of the stack trace and from Helix
> Apache
> > >> Docs <http://helix.apache.org/0.7.1-docs/tutorial_agent.html>
> *("Helix
> > >> is built on the following assumption: if your distributed resource is
> > >> modeled by a finite state machine, then Helix can tell participants
> when
> > >> they should transition between states. In the Java API, this means
> > >> implementing transition callbacks. In the Helix agent API, this means
> > >> providing commands than can run for each transition"),* did you
> > >> implement *transition callback* for these tasks?
> > >>
> > >> On Thu, Nov 16, 2017 at 10:01 AM, DImuthu Upeksha <
> > >> dimuthu.upeksha2@gmail.com> wrote:
> > >>
> > >>> Hi Devs,
> > >>>
> > >>> I'm working on the technology evaluation to re architecture Apache
> > >>> Airavata task execution framework and Helix seems like a good
> > candidate for
> > >>> that as it has an in built distributed generic workflow execution
> > >>> capability. After going through several tutorials, I tried to
> > implement a
> > >>> simple workflow on Helix to demonstrate following transition
> > >>>
> > >>> Data Collecting Job -> Command Executing Job -> Data Uploading Job
> > >>>
> > >>> I managed to implement this using a Participant node that includes
> all
> > >>> the tasks required for above workflow. However my goal is to
> implement
> > >>> specialized Participants for each Job type. For example, Participant
> 1
> > >>> knows only about the tasks to perform Data Collecting Job and
> > Participant
> > >>> knows only about the task to perform Command Executing Job. When I
> > tried to
> > >>> implement such Participants, I got following error from Participant
> 1.
> > I
> > >>> can share the code samples that I have tried but before that I need
> to
> > know
> > >>> whether my approach is compatible with Helix's design? Does Helix
> > require
> > >>> all the Participants to be homogeneous?
> > >>>
> > >>> Executing data collecting
> > >>> 675892 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
> > >>> dling.HelixStateTransitionHandler - Exception while executing a
> state
> > >>> transition task Airavata_Workflow_commandExecuteJob_0
> > >>> java.lang.reflect.InvocationTargetException
> > >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> > >>> ssorImpl.java:62)
> > >>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> > >>> thodAccessorImpl.java:43)
> > >>> at java.lang.reflect.Method.invoke(Method.java:498)
> > >>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
> > >>> ler.invoke(HelixStateTransitionHandler.java:344)
> > >>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
> > >>> ler.handleMessage(HelixStateTransitionHandler.java:290)
> > >>> at org.apache.helix.messaging.handling.HelixTask.call(
> > HelixTask.java:84)
> > >>> at org.apache.helix.messaging.handling.HelixTask.call(
> > HelixTask.java:41)
> > >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > >>> Executor.java:1149)
> > >>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > >>> lExecutor.java:624)
> > >>> at java.lang.Thread.run(Thread.java:748)
> > >>> Caused by: java.lang.IllegalStateException: No callback implemented
> > for
> > >>> task COMMAND
> > >>> at org.apache.helix.task.TaskStateModel.startTask(TaskStateMode
> > >>> l.java:302)
> > >>> at org.apache.helix.task.TaskStateModel.onBecomeRunningFromInit
> > >>> (TaskStateModel.java:69)
> > >>> ... 12 more
> > >>> 675900 [pool-2-thread-8] ERROR org.apache.helix.participant.
> > statemachine.StateModel
> > >>> - Default rollback method invoked on error. Error Code: ERROR
> > >>> 675908 [pool-2-thread-8] ERROR org.apache.helix.messaging.
> > handling.HelixTask
> > >>> - Message execution failed. msgId: 8d9a08c9-0ae7-419a-8940-
> > 6b8e6e7c1ecb,
> > >>> errorMsg: java.lang.reflect.InvocationTargetException
> > >>> 675922 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
> > >>> dling.HelixStateTransitionHandler - Skip internal error. errCode:
> > >>> ERROR, errMsg: null
> > >>>
> > >>> Thanks
> > >>> Dimuthu
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Thanks and regards,
> > >>
> > >> Ajinkya Dhamnaskar
> > >> Student ID : 0003469679
> > >> Masters (CS)
> > >> +1 (812) 369- 5416 <(812)%20369-5416>
> > >>
> > >
> > >
> >
> >
> > --
> > Thanks and regards,
> >
> > Ajinkya Dhamnaskar
> > Student ID : 0003469679
> > Masters (CS)
> > +1 (812) 369- 5416
> >
>
Re: Using job specific Participants in Helix
Posted by DImuthu Upeksha <di...@gmail.com>.
Hi Junkai
That did the trick. Thanks a lot
Thanks
Dimuthu
On 17 Nov 2017 10:32, "Xue Junkai" <jx...@apache.org> wrote:
Hi Dlmuthu and Kishore,
Yes. Task framework does support tagged instance at job level. It can be
set in JobConfig for InstanceGroupTag. Also, do not forget to tag the
instances.
Best,
Junkai
On Thu, Nov 16, 2017 at 8:50 PM, kishore g <g....@gmail.com> wrote:
> Hi Dlmuthu,
>
> You can achieve this by using the tag feature in Helix. For example, to
> Participants P1, P2 should only handle DataCollecting tasks, you need to
do
> the following
>
> - Tag Participant P1 and P2 as "DataCollecting"
> - When you create a job resource that represents DataCollecting task,
> you have to tag the resource also as DataCollecting.
>
> Helix will then assign all tasks in this job to nodes that are tagged as
> DataCollecting.
>
> Note this is available only in FULL_AUTO rebalance mode. I am not sure if
> the rebalancer in task framework supports this feature.
>
> Lei/Junkai, do you know if this is supported in Task Framework?
>
> thanks
>
>
> On Thu, Nov 16, 2017 at 8:31 PM, Ajinkya Dhamnaskar <adhamnas@umail.iu.edu
> >
> wrote:
>
> > Dlmuthu,
> >
> > This explains your need. Could you please point me to the code? I was
> > wondering, where and how are we registering callbacks for the
> participants?
> >
> > On Thu, Nov 16, 2017 at 7:16 PM, DImuthu Upeksha <
> > dimuthu.upeksha2@gmail.com
> > > wrote:
> >
> > > Adding Helix Dev
> > >
> > > Hi Ajinkya
> > >
> > > Thank you for the explanation.
> > >
> > > Let me explain my requirement. [1] is the correct task registry
> > > configuration in a working participant code. All the transition call
> > backs
> > > are registered in this code. The problem here is, we have to bundle
> > > binaries of all the tasks into a single Participant. If we want to
> change
> > > one task, we need to rebuild the participant with other tasks as well.
> > What
> > > I thought is, why can't we build Participants that only do specific
set
> > of
> > > tasks. For example, in the cluster there are 3 participants,
> Participant
> > 1
> > > [2], Participant 2 [3] and Participant 3 [4]. If we want to change the
> > > DataCollecting task, we only need to re build Participant 2. I got
> above
> > > error, once I run 3 participants in above configuration. I can
> understand
> > > this is due to the missing transition callback of CommandTask in
> > > Participant 1 as I have purposely commented out it. What I need to
know
> > is
> > > that, is this configuration allowed in Helix architecture or do we
have
> > to
> > > implement Participants that contain all the task implementations as in
> > [1].
> > > Workflow configuration for both scenarios is here [5].
> > >
> > > [1] https://gist.github.com/DImuthuUpe/41c0db579e7d86d101d112f07ed6ea
> 00
> > > [2] https://gist.github.com/DImuthuUpe/ec72df1ec3207ce2dce88ff7f1756d
> a4
> > > [3] https://gist.github.com/DImuthuUpe/4d193a3dff3008315efa2e31f6209c
> ac
> > > [4] https://gist.github.com/DImuthuUpe/872c432935b8d33944dd571b3ac420
> 7b
> > > [5] https://gist.github.com/DImuthuUpe/f61851b68b685b8d6744689dc130ba
> bd
> > >
> > > Thanks
> > > Dimuthu
> > >
> > > On Fri, Nov 17, 2017 at 3:21 AM, Ajinkya Dhamnaskar <
> > adhamnas@umail.iu.edu
> > > > wrote:
> > >
> > >> Hey Dlmuthu,
> > >>
> > >> Not an expert in Helix, but from exceptions it seems, system is
> entering
> > >> in a state not expected by reflection. I feel
> https://github.com/apache
> > >> /helix/blob/master/helix-core/src/main/java/org/apache/helix
> > >> /messaging/handling/HelixStateTransitionHandler.java#L295 is
> triggering
> > >> this exception.
> > >> As mentioned in the later part of the stack trace and from Helix
> Apache
> > >> Docs <http://helix.apache.org/0.7.1-docs/tutorial_agent.html>
> *("Helix
> > >> is built on the following assumption: if your distributed resource is
> > >> modeled by a finite state machine, then Helix can tell participants
> when
> > >> they should transition between states. In the Java API, this means
> > >> implementing transition callbacks. In the Helix agent API, this means
> > >> providing commands than can run for each transition"),* did you
> > >> implement *transition callback* for these tasks?
> > >>
> > >> On Thu, Nov 16, 2017 at 10:01 AM, DImuthu Upeksha <
> > >> dimuthu.upeksha2@gmail.com> wrote:
> > >>
> > >>> Hi Devs,
> > >>>
> > >>> I'm working on the technology evaluation to re architecture Apache
> > >>> Airavata task execution framework and Helix seems like a good
> > candidate for
> > >>> that as it has an in built distributed generic workflow execution
> > >>> capability. After going through several tutorials, I tried to
> > implement a
> > >>> simple workflow on Helix to demonstrate following transition
> > >>>
> > >>> Data Collecting Job -> Command Executing Job -> Data Uploading Job
> > >>>
> > >>> I managed to implement this using a Participant node that includes
> all
> > >>> the tasks required for above workflow. However my goal is to
> implement
> > >>> specialized Participants for each Job type. For example, Participant
> 1
> > >>> knows only about the tasks to perform Data Collecting Job and
> > Participant
> > >>> knows only about the task to perform Command Executing Job. When I
> > tried to
> > >>> implement such Participants, I got following error from Participant
> 1.
> > I
> > >>> can share the code samples that I have tried but before that I need
> to
> > know
> > >>> whether my approach is compatible with Helix's design? Does Helix
> > require
> > >>> all the Participants to be homogeneous?
> > >>>
> > >>> Executing data collecting
> > >>> 675892 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
> > >>> dling.HelixStateTransitionHandler - Exception while executing a
> state
> > >>> transition task Airavata_Workflow_commandExecuteJob_0
> > >>> java.lang.reflect.InvocationTargetException
> > >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> > >>> ssorImpl.java:62)
> > >>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> > >>> thodAccessorImpl.java:43)
> > >>> at java.lang.reflect.Method.invoke(Method.java:498)
> > >>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
> > >>> ler.invoke(HelixStateTransitionHandler.java:344)
> > >>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
> > >>> ler.handleMessage(HelixStateTransitionHandler.java:290)
> > >>> at org.apache.helix.messaging.handling.HelixTask.call(
> > HelixTask.java:84)
> > >>> at org.apache.helix.messaging.handling.HelixTask.call(
> > HelixTask.java:41)
> > >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > >>> Executor.java:1149)
> > >>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > >>> lExecutor.java:624)
> > >>> at java.lang.Thread.run(Thread.java:748)
> > >>> Caused by: java.lang.IllegalStateException: No callback implemented
> > for
> > >>> task COMMAND
> > >>> at org.apache.helix.task.TaskStateModel.startTask(TaskStateMode
> > >>> l.java:302)
> > >>> at org.apache.helix.task.TaskStateModel.onBecomeRunningFromInit
> > >>> (TaskStateModel.java:69)
> > >>> ... 12 more
> > >>> 675900 [pool-2-thread-8] ERROR org.apache.helix.participant.
> > statemachine.StateModel
> > >>> - Default rollback method invoked on error. Error Code: ERROR
> > >>> 675908 [pool-2-thread-8] ERROR org.apache.helix.messaging.
> > handling.HelixTask
> > >>> - Message execution failed. msgId: 8d9a08c9-0ae7-419a-8940-
> > 6b8e6e7c1ecb,
> > >>> errorMsg: java.lang.reflect.InvocationTargetException
> > >>> 675922 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
> > >>> dling.HelixStateTransitionHandler - Skip internal error. errCode:
> > >>> ERROR, errMsg: null
> > >>>
> > >>> Thanks
> > >>> Dimuthu
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Thanks and regards,
> > >>
> > >> Ajinkya Dhamnaskar
> > >> Student ID : 0003469679
> > >> Masters (CS)
> > >> +1 (812) 369- 5416 <(812)%20369-5416>
> > >>
> > >
> > >
> >
> >
> > --
> > Thanks and regards,
> >
> > Ajinkya Dhamnaskar
> > Student ID : 0003469679
> > Masters (CS)
> > +1 (812) 369- 5416
> >
>
Re: Using job specific Participants in Helix
Posted by Xue Junkai <jx...@apache.org>.
Hi Dlmuthu and Kishore,
Yes. Task framework does support tagged instance at job level. It can be
set in JobConfig for InstanceGroupTag. Also, do not forget to tag the
instances.
Best,
Junkai
On Thu, Nov 16, 2017 at 8:50 PM, kishore g <g....@gmail.com> wrote:
> Hi Dlmuthu,
>
> You can achieve this by using the tag feature in Helix. For example, to
> Participants P1, P2 should only handle DataCollecting tasks, you need to do
> the following
>
> - Tag Participant P1 and P2 as "DataCollecting"
> - When you create a job resource that represents DataCollecting task,
> you have to tag the resource also as DataCollecting.
>
> Helix will then assign all tasks in this job to nodes that are tagged as
> DataCollecting.
>
> Note this is available only in FULL_AUTO rebalance mode. I am not sure if
> the rebalancer in task framework supports this feature.
>
> Lei/Junkai, do you know if this is supported in Task Framework?
>
> thanks
>
>
> On Thu, Nov 16, 2017 at 8:31 PM, Ajinkya Dhamnaskar <adhamnas@umail.iu.edu
> >
> wrote:
>
> > Dlmuthu,
> >
> > This explains your need. Could you please point me to the code? I was
> > wondering, where and how are we registering callbacks for the
> participants?
> >
> > On Thu, Nov 16, 2017 at 7:16 PM, DImuthu Upeksha <
> > dimuthu.upeksha2@gmail.com
> > > wrote:
> >
> > > Adding Helix Dev
> > >
> > > Hi Ajinkya
> > >
> > > Thank you for the explanation.
> > >
> > > Let me explain my requirement. [1] is the correct task registry
> > > configuration in a working participant code. All the transition call
> > backs
> > > are registered in this code. The problem here is, we have to bundle
> > > binaries of all the tasks into a single Participant. If we want to
> change
> > > one task, we need to rebuild the participant with other tasks as well.
> > What
> > > I thought is, why can't we build Participants that only do specific set
> > of
> > > tasks. For example, in the cluster there are 3 participants,
> Participant
> > 1
> > > [2], Participant 2 [3] and Participant 3 [4]. If we want to change the
> > > DataCollecting task, we only need to re build Participant 2. I got
> above
> > > error, once I run 3 participants in above configuration. I can
> understand
> > > this is due to the missing transition callback of CommandTask in
> > > Participant 1 as I have purposely commented out it. What I need to know
> > is
> > > that, is this configuration allowed in Helix architecture or do we have
> > to
> > > implement Participants that contain all the task implementations as in
> > [1].
> > > Workflow configuration for both scenarios is here [5].
> > >
> > > [1] https://gist.github.com/DImuthuUpe/41c0db579e7d86d101d112f07ed6ea
> 00
> > > [2] https://gist.github.com/DImuthuUpe/ec72df1ec3207ce2dce88ff7f1756d
> a4
> > > [3] https://gist.github.com/DImuthuUpe/4d193a3dff3008315efa2e31f6209c
> ac
> > > [4] https://gist.github.com/DImuthuUpe/872c432935b8d33944dd571b3ac420
> 7b
> > > [5] https://gist.github.com/DImuthuUpe/f61851b68b685b8d6744689dc130ba
> bd
> > >
> > > Thanks
> > > Dimuthu
> > >
> > > On Fri, Nov 17, 2017 at 3:21 AM, Ajinkya Dhamnaskar <
> > adhamnas@umail.iu.edu
> > > > wrote:
> > >
> > >> Hey Dlmuthu,
> > >>
> > >> Not an expert in Helix, but from exceptions it seems, system is
> entering
> > >> in a state not expected by reflection. I feel
> https://github.com/apache
> > >> /helix/blob/master/helix-core/src/main/java/org/apache/helix
> > >> /messaging/handling/HelixStateTransitionHandler.java#L295 is
> triggering
> > >> this exception.
> > >> As mentioned in the later part of the stack trace and from Helix
> Apache
> > >> Docs <http://helix.apache.org/0.7.1-docs/tutorial_agent.html>
> *("Helix
> > >> is built on the following assumption: if your distributed resource is
> > >> modeled by a finite state machine, then Helix can tell participants
> when
> > >> they should transition between states. In the Java API, this means
> > >> implementing transition callbacks. In the Helix agent API, this means
> > >> providing commands than can run for each transition"),* did you
> > >> implement *transition callback* for these tasks?
> > >>
> > >> On Thu, Nov 16, 2017 at 10:01 AM, DImuthu Upeksha <
> > >> dimuthu.upeksha2@gmail.com> wrote:
> > >>
> > >>> Hi Devs,
> > >>>
> > >>> I'm working on the technology evaluation to re architecture Apache
> > >>> Airavata task execution framework and Helix seems like a good
> > candidate for
> > >>> that as it has an in built distributed generic workflow execution
> > >>> capability. After going through several tutorials, I tried to
> > implement a
> > >>> simple workflow on Helix to demonstrate following transition
> > >>>
> > >>> Data Collecting Job -> Command Executing Job -> Data Uploading Job
> > >>>
> > >>> I managed to implement this using a Participant node that includes
> all
> > >>> the tasks required for above workflow. However my goal is to
> implement
> > >>> specialized Participants for each Job type. For example, Participant
> 1
> > >>> knows only about the tasks to perform Data Collecting Job and
> > Participant
> > >>> knows only about the task to perform Command Executing Job. When I
> > tried to
> > >>> implement such Participants, I got following error from Participant
> 1.
> > I
> > >>> can share the code samples that I have tried but before that I need
> to
> > know
> > >>> whether my approach is compatible with Helix's design? Does Helix
> > require
> > >>> all the Participants to be homogeneous?
> > >>>
> > >>> Executing data collecting
> > >>> 675892 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
> > >>> dling.HelixStateTransitionHandler - Exception while executing a
> state
> > >>> transition task Airavata_Workflow_commandExecuteJob_0
> > >>> java.lang.reflect.InvocationTargetException
> > >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> > >>> ssorImpl.java:62)
> > >>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> > >>> thodAccessorImpl.java:43)
> > >>> at java.lang.reflect.Method.invoke(Method.java:498)
> > >>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
> > >>> ler.invoke(HelixStateTransitionHandler.java:344)
> > >>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
> > >>> ler.handleMessage(HelixStateTransitionHandler.java:290)
> > >>> at org.apache.helix.messaging.handling.HelixTask.call(
> > HelixTask.java:84)
> > >>> at org.apache.helix.messaging.handling.HelixTask.call(
> > HelixTask.java:41)
> > >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > >>> Executor.java:1149)
> > >>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > >>> lExecutor.java:624)
> > >>> at java.lang.Thread.run(Thread.java:748)
> > >>> Caused by: java.lang.IllegalStateException: No callback implemented
> > for
> > >>> task COMMAND
> > >>> at org.apache.helix.task.TaskStateModel.startTask(TaskStateMode
> > >>> l.java:302)
> > >>> at org.apache.helix.task.TaskStateModel.onBecomeRunningFromInit
> > >>> (TaskStateModel.java:69)
> > >>> ... 12 more
> > >>> 675900 [pool-2-thread-8] ERROR org.apache.helix.participant.
> > statemachine.StateModel
> > >>> - Default rollback method invoked on error. Error Code: ERROR
> > >>> 675908 [pool-2-thread-8] ERROR org.apache.helix.messaging.
> > handling.HelixTask
> > >>> - Message execution failed. msgId: 8d9a08c9-0ae7-419a-8940-
> > 6b8e6e7c1ecb,
> > >>> errorMsg: java.lang.reflect.InvocationTargetException
> > >>> 675922 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
> > >>> dling.HelixStateTransitionHandler - Skip internal error. errCode:
> > >>> ERROR, errMsg: null
> > >>>
> > >>> Thanks
> > >>> Dimuthu
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Thanks and regards,
> > >>
> > >> Ajinkya Dhamnaskar
> > >> Student ID : 0003469679
> > >> Masters (CS)
> > >> +1 (812) 369- 5416 <(812)%20369-5416>
> > >>
> > >
> > >
> >
> >
> > --
> > Thanks and regards,
> >
> > Ajinkya Dhamnaskar
> > Student ID : 0003469679
> > Masters (CS)
> > +1 (812) 369- 5416
> >
>
Re: Using job specific Participants in Helix
Posted by kishore g <g....@gmail.com>.
Hi Dlmuthu,
You can achieve this by using the tag feature in Helix. For example, to
Participants P1, P2 should only handle DataCollecting tasks, you need to do
the following
- Tag Participant P1 and P2 as "DataCollecting"
- When you create a job resource that represents DataCollecting task,
you have to tag the resource also as DataCollecting.
Helix will then assign all tasks in this job to nodes that are tagged as
DataCollecting.
Note this is available only in FULL_AUTO rebalance mode. I am not sure if
the rebalancer in task framework supports this feature.
Lei/Junkai, do you know if this is supported in Task Framework?
thanks
On Thu, Nov 16, 2017 at 8:31 PM, Ajinkya Dhamnaskar <ad...@umail.iu.edu>
wrote:
> Dlmuthu,
>
> This explains your need. Could you please point me to the code? I was
> wondering, where and how are we registering callbacks for the participants?
>
> On Thu, Nov 16, 2017 at 7:16 PM, DImuthu Upeksha <
> dimuthu.upeksha2@gmail.com
> > wrote:
>
> > Adding Helix Dev
> >
> > Hi Ajinkya
> >
> > Thank you for the explanation.
> >
> > Let me explain my requirement. [1] is the correct task registry
> > configuration in a working participant code. All the transition call
> backs
> > are registered in this code. The problem here is, we have to bundle
> > binaries of all the tasks into a single Participant. If we want to change
> > one task, we need to rebuild the participant with other tasks as well.
> What
> > I thought is, why can't we build Participants that only do specific set
> of
> > tasks. For example, in the cluster there are 3 participants, Participant
> 1
> > [2], Participant 2 [3] and Participant 3 [4]. If we want to change the
> > DataCollecting task, we only need to re build Participant 2. I got above
> > error, once I run 3 participants in above configuration. I can understand
> > this is due to the missing transition callback of CommandTask in
> > Participant 1 as I have purposely commented out it. What I need to know
> is
> > that, is this configuration allowed in Helix architecture or do we have
> to
> > implement Participants that contain all the task implementations as in
> [1].
> > Workflow configuration for both scenarios is here [5].
> >
> > [1] https://gist.github.com/DImuthuUpe/41c0db579e7d86d101d112f07ed6ea00
> > [2] https://gist.github.com/DImuthuUpe/ec72df1ec3207ce2dce88ff7f1756da4
> > [3] https://gist.github.com/DImuthuUpe/4d193a3dff3008315efa2e31f6209cac
> > [4] https://gist.github.com/DImuthuUpe/872c432935b8d33944dd571b3ac4207b
> > [5] https://gist.github.com/DImuthuUpe/f61851b68b685b8d6744689dc130babd
> >
> > Thanks
> > Dimuthu
> >
> > On Fri, Nov 17, 2017 at 3:21 AM, Ajinkya Dhamnaskar <
> adhamnas@umail.iu.edu
> > > wrote:
> >
> >> Hey Dlmuthu,
> >>
> >> Not an expert in Helix, but from exceptions it seems, system is entering
> >> in a state not expected by reflection. I feel https://github.com/apache
> >> /helix/blob/master/helix-core/src/main/java/org/apache/helix
> >> /messaging/handling/HelixStateTransitionHandler.java#L295 is triggering
> >> this exception.
> >> As mentioned in the later part of the stack trace and from Helix Apache
> >> Docs <http://helix.apache.org/0.7.1-docs/tutorial_agent.html> *("Helix
> >> is built on the following assumption: if your distributed resource is
> >> modeled by a finite state machine, then Helix can tell participants when
> >> they should transition between states. In the Java API, this means
> >> implementing transition callbacks. In the Helix agent API, this means
> >> providing commands than can run for each transition"),* did you
> >> implement *transition callback* for these tasks?
> >>
> >> On Thu, Nov 16, 2017 at 10:01 AM, DImuthu Upeksha <
> >> dimuthu.upeksha2@gmail.com> wrote:
> >>
> >>> Hi Devs,
> >>>
> >>> I'm working on the technology evaluation to re architecture Apache
> >>> Airavata task execution framework and Helix seems like a good
> candidate for
> >>> that as it has an in built distributed generic workflow execution
> >>> capability. After going through several tutorials, I tried to
> implement a
> >>> simple workflow on Helix to demonstrate following transition
> >>>
> >>> Data Collecting Job -> Command Executing Job -> Data Uploading Job
> >>>
> >>> I managed to implement this using a Participant node that includes all
> >>> the tasks required for above workflow. However my goal is to implement
> >>> specialized Participants for each Job type. For example, Participant 1
> >>> knows only about the tasks to perform Data Collecting Job and
> Participant
> >>> knows only about the task to perform Command Executing Job. When I
> tried to
> >>> implement such Participants, I got following error from Participant 1.
> I
> >>> can share the code samples that I have tried but before that I need to
> know
> >>> whether my approach is compatible with Helix's design? Does Helix
> require
> >>> all the Participants to be homogeneous?
> >>>
> >>> Executing data collecting
> >>> 675892 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
> >>> dling.HelixStateTransitionHandler - Exception while executing a state
> >>> transition task Airavata_Workflow_commandExecuteJob_0
> >>> java.lang.reflect.InvocationTargetException
> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> >>> ssorImpl.java:62)
> >>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> >>> thodAccessorImpl.java:43)
> >>> at java.lang.reflect.Method.invoke(Method.java:498)
> >>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
> >>> ler.invoke(HelixStateTransitionHandler.java:344)
> >>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
> >>> ler.handleMessage(HelixStateTransitionHandler.java:290)
> >>> at org.apache.helix.messaging.handling.HelixTask.call(
> HelixTask.java:84)
> >>> at org.apache.helix.messaging.handling.HelixTask.call(
> HelixTask.java:41)
> >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> >>> Executor.java:1149)
> >>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> >>> lExecutor.java:624)
> >>> at java.lang.Thread.run(Thread.java:748)
> >>> Caused by: java.lang.IllegalStateException: No callback implemented
> for
> >>> task COMMAND
> >>> at org.apache.helix.task.TaskStateModel.startTask(TaskStateMode
> >>> l.java:302)
> >>> at org.apache.helix.task.TaskStateModel.onBecomeRunningFromInit
> >>> (TaskStateModel.java:69)
> >>> ... 12 more
> >>> 675900 [pool-2-thread-8] ERROR org.apache.helix.participant.
> statemachine.StateModel
> >>> - Default rollback method invoked on error. Error Code: ERROR
> >>> 675908 [pool-2-thread-8] ERROR org.apache.helix.messaging.
> handling.HelixTask
> >>> - Message execution failed. msgId: 8d9a08c9-0ae7-419a-8940-
> 6b8e6e7c1ecb,
> >>> errorMsg: java.lang.reflect.InvocationTargetException
> >>> 675922 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
> >>> dling.HelixStateTransitionHandler - Skip internal error. errCode:
> >>> ERROR, errMsg: null
> >>>
> >>> Thanks
> >>> Dimuthu
> >>>
> >>
> >>
> >>
> >> --
> >> Thanks and regards,
> >>
> >> Ajinkya Dhamnaskar
> >> Student ID : 0003469679
> >> Masters (CS)
> >> +1 (812) 369- 5416 <(812)%20369-5416>
> >>
> >
> >
>
>
> --
> Thanks and regards,
>
> Ajinkya Dhamnaskar
> Student ID : 0003469679
> Masters (CS)
> +1 (812) 369- 5416
>
Re: Using job specific Participants in Helix
Posted by kishore g <g....@gmail.com>.
Hi Dlmuthu,
You can achieve this by using the tag feature in Helix. For example, to
Participants P1, P2 should only handle DataCollecting tasks, you need to do
the following
- Tag Participant P1 and P2 as "DataCollecting"
- When you create a job resource that represents DataCollecting task,
you have to tag the resource also as DataCollecting.
Helix will then assign all tasks in this job to nodes that are tagged as
DataCollecting.
Note this is available only in FULL_AUTO rebalance mode. I am not sure if
the rebalancer in task framework supports this feature.
Lei/Junkai, do you know if this is supported in Task Framework?
thanks
On Thu, Nov 16, 2017 at 8:31 PM, Ajinkya Dhamnaskar <ad...@umail.iu.edu>
wrote:
> Dlmuthu,
>
> This explains your need. Could you please point me to the code? I was
> wondering, where and how are we registering callbacks for the participants?
>
> On Thu, Nov 16, 2017 at 7:16 PM, DImuthu Upeksha <
> dimuthu.upeksha2@gmail.com
> > wrote:
>
> > Adding Helix Dev
> >
> > Hi Ajinkya
> >
> > Thank you for the explanation.
> >
> > Let me explain my requirement. [1] is the correct task registry
> > configuration in a working participant code. All the transition call
> backs
> > are registered in this code. The problem here is, we have to bundle
> > binaries of all the tasks into a single Participant. If we want to change
> > one task, we need to rebuild the participant with other tasks as well.
> What
> > I thought is, why can't we build Participants that only do specific set
> of
> > tasks. For example, in the cluster there are 3 participants, Participant
> 1
> > [2], Participant 2 [3] and Participant 3 [4]. If we want to change the
> > DataCollecting task, we only need to re build Participant 2. I got above
> > error, once I run 3 participants in above configuration. I can understand
> > this is due to the missing transition callback of CommandTask in
> > Participant 1 as I have purposely commented out it. What I need to know
> is
> > that, is this configuration allowed in Helix architecture or do we have
> to
> > implement Participants that contain all the task implementations as in
> [1].
> > Workflow configuration for both scenarios is here [5].
> >
> > [1] https://gist.github.com/DImuthuUpe/41c0db579e7d86d101d112f07ed6ea00
> > [2] https://gist.github.com/DImuthuUpe/ec72df1ec3207ce2dce88ff7f1756da4
> > [3] https://gist.github.com/DImuthuUpe/4d193a3dff3008315efa2e31f6209cac
> > [4] https://gist.github.com/DImuthuUpe/872c432935b8d33944dd571b3ac4207b
> > [5] https://gist.github.com/DImuthuUpe/f61851b68b685b8d6744689dc130babd
> >
> > Thanks
> > Dimuthu
> >
> > On Fri, Nov 17, 2017 at 3:21 AM, Ajinkya Dhamnaskar <
> adhamnas@umail.iu.edu
> > > wrote:
> >
> >> Hey Dlmuthu,
> >>
> >> Not an expert in Helix, but from exceptions it seems, system is entering
> >> in a state not expected by reflection. I feel https://github.com/apache
> >> /helix/blob/master/helix-core/src/main/java/org/apache/helix
> >> /messaging/handling/HelixStateTransitionHandler.java#L295 is triggering
> >> this exception.
> >> As mentioned in the later part of the stack trace and from Helix Apache
> >> Docs <http://helix.apache.org/0.7.1-docs/tutorial_agent.html> *("Helix
> >> is built on the following assumption: if your distributed resource is
> >> modeled by a finite state machine, then Helix can tell participants when
> >> they should transition between states. In the Java API, this means
> >> implementing transition callbacks. In the Helix agent API, this means
> >> providing commands than can run for each transition"),* did you
> >> implement *transition callback* for these tasks?
> >>
> >> On Thu, Nov 16, 2017 at 10:01 AM, DImuthu Upeksha <
> >> dimuthu.upeksha2@gmail.com> wrote:
> >>
> >>> Hi Devs,
> >>>
> >>> I'm working on the technology evaluation to re architecture Apache
> >>> Airavata task execution framework and Helix seems like a good
> candidate for
> >>> that as it has an in built distributed generic workflow execution
> >>> capability. After going through several tutorials, I tried to
> implement a
> >>> simple workflow on Helix to demonstrate following transition
> >>>
> >>> Data Collecting Job -> Command Executing Job -> Data Uploading Job
> >>>
> >>> I managed to implement this using a Participant node that includes all
> >>> the tasks required for above workflow. However my goal is to implement
> >>> specialized Participants for each Job type. For example, Participant 1
> >>> knows only about the tasks to perform Data Collecting Job and
> Participant
> >>> knows only about the task to perform Command Executing Job. When I
> tried to
> >>> implement such Participants, I got following error from Participant 1.
> I
> >>> can share the code samples that I have tried but before that I need to
> know
> >>> whether my approach is compatible with Helix's design? Does Helix
> require
> >>> all the Participants to be homogeneous?
> >>>
> >>> Executing data collecting
> >>> 675892 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
> >>> dling.HelixStateTransitionHandler - Exception while executing a state
> >>> transition task Airavata_Workflow_commandExecuteJob_0
> >>> java.lang.reflect.InvocationTargetException
> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> >>> ssorImpl.java:62)
> >>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> >>> thodAccessorImpl.java:43)
> >>> at java.lang.reflect.Method.invoke(Method.java:498)
> >>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
> >>> ler.invoke(HelixStateTransitionHandler.java:344)
> >>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
> >>> ler.handleMessage(HelixStateTransitionHandler.java:290)
> >>> at org.apache.helix.messaging.handling.HelixTask.call(
> HelixTask.java:84)
> >>> at org.apache.helix.messaging.handling.HelixTask.call(
> HelixTask.java:41)
> >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> >>> Executor.java:1149)
> >>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> >>> lExecutor.java:624)
> >>> at java.lang.Thread.run(Thread.java:748)
> >>> Caused by: java.lang.IllegalStateException: No callback implemented
> for
> >>> task COMMAND
> >>> at org.apache.helix.task.TaskStateModel.startTask(TaskStateMode
> >>> l.java:302)
> >>> at org.apache.helix.task.TaskStateModel.onBecomeRunningFromInit
> >>> (TaskStateModel.java:69)
> >>> ... 12 more
> >>> 675900 [pool-2-thread-8] ERROR org.apache.helix.participant.
> statemachine.StateModel
> >>> - Default rollback method invoked on error. Error Code: ERROR
> >>> 675908 [pool-2-thread-8] ERROR org.apache.helix.messaging.
> handling.HelixTask
> >>> - Message execution failed. msgId: 8d9a08c9-0ae7-419a-8940-
> 6b8e6e7c1ecb,
> >>> errorMsg: java.lang.reflect.InvocationTargetException
> >>> 675922 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
> >>> dling.HelixStateTransitionHandler - Skip internal error. errCode:
> >>> ERROR, errMsg: null
> >>>
> >>> Thanks
> >>> Dimuthu
> >>>
> >>
> >>
> >>
> >> --
> >> Thanks and regards,
> >>
> >> Ajinkya Dhamnaskar
> >> Student ID : 0003469679
> >> Masters (CS)
> >> +1 (812) 369- 5416 <(812)%20369-5416>
> >>
> >
> >
>
>
> --
> Thanks and regards,
>
> Ajinkya Dhamnaskar
> Student ID : 0003469679
> Masters (CS)
> +1 (812) 369- 5416
>
Re: Using job specific Participants in Helix
Posted by Ajinkya Dhamnaskar <ad...@umail.iu.edu>.
Dlmuthu,
This explains your need. Could you please point me to the code? I was
wondering, where and how are we registering callbacks for the participants?
On Thu, Nov 16, 2017 at 7:16 PM, DImuthu Upeksha <dimuthu.upeksha2@gmail.com
> wrote:
> Adding Helix Dev
>
> Hi Ajinkya
>
> Thank you for the explanation.
>
> Let me explain my requirement. [1] is the correct task registry
> configuration in a working participant code. All the transition call backs
> are registered in this code. The problem here is, we have to bundle
> binaries of all the tasks into a single Participant. If we want to change
> one task, we need to rebuild the participant with other tasks as well. What
> I thought is, why can't we build Participants that only do specific set of
> tasks. For example, in the cluster there are 3 participants, Participant 1
> [2], Participant 2 [3] and Participant 3 [4]. If we want to change the
> DataCollecting task, we only need to re build Participant 2. I got above
> error, once I run 3 participants in above configuration. I can understand
> this is due to the missing transition callback of CommandTask in
> Participant 1 as I have purposely commented out it. What I need to know is
> that, is this configuration allowed in Helix architecture or do we have to
> implement Participants that contain all the task implementations as in [1].
> Workflow configuration for both scenarios is here [5].
>
> [1] https://gist.github.com/DImuthuUpe/41c0db579e7d86d101d112f07ed6ea00
> [2] https://gist.github.com/DImuthuUpe/ec72df1ec3207ce2dce88ff7f1756da4
> [3] https://gist.github.com/DImuthuUpe/4d193a3dff3008315efa2e31f6209cac
> [4] https://gist.github.com/DImuthuUpe/872c432935b8d33944dd571b3ac4207b
> [5] https://gist.github.com/DImuthuUpe/f61851b68b685b8d6744689dc130babd
>
> Thanks
> Dimuthu
>
> On Fri, Nov 17, 2017 at 3:21 AM, Ajinkya Dhamnaskar <adhamnas@umail.iu.edu
> > wrote:
>
>> Hey Dlmuthu,
>>
>> Not an expert in Helix, but from exceptions it seems, system is entering
>> in a state not expected by reflection. I feel https://github.com/apache
>> /helix/blob/master/helix-core/src/main/java/org/apache/helix
>> /messaging/handling/HelixStateTransitionHandler.java#L295 is triggering
>> this exception.
>> As mentioned in the later part of the stack trace and from Helix Apache
>> Docs <http://helix.apache.org/0.7.1-docs/tutorial_agent.html> *("Helix
>> is built on the following assumption: if your distributed resource is
>> modeled by a finite state machine, then Helix can tell participants when
>> they should transition between states. In the Java API, this means
>> implementing transition callbacks. In the Helix agent API, this means
>> providing commands than can run for each transition"),* did you
>> implement *transition callback* for these tasks?
>>
>> On Thu, Nov 16, 2017 at 10:01 AM, DImuthu Upeksha <
>> dimuthu.upeksha2@gmail.com> wrote:
>>
>>> Hi Devs,
>>>
>>> I'm working on the technology evaluation to re architecture Apache
>>> Airavata task execution framework and Helix seems like a good candidate for
>>> that as it has an in built distributed generic workflow execution
>>> capability. After going through several tutorials, I tried to implement a
>>> simple workflow on Helix to demonstrate following transition
>>>
>>> Data Collecting Job -> Command Executing Job -> Data Uploading Job
>>>
>>> I managed to implement this using a Participant node that includes all
>>> the tasks required for above workflow. However my goal is to implement
>>> specialized Participants for each Job type. For example, Participant 1
>>> knows only about the tasks to perform Data Collecting Job and Participant
>>> knows only about the task to perform Command Executing Job. When I tried to
>>> implement such Participants, I got following error from Participant 1. I
>>> can share the code samples that I have tried but before that I need to know
>>> whether my approach is compatible with Helix's design? Does Helix require
>>> all the Participants to be homogeneous?
>>>
>>> Executing data collecting
>>> 675892 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
>>> dling.HelixStateTransitionHandler - Exception while executing a state
>>> transition task Airavata_Workflow_commandExecuteJob_0
>>> java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>>> ssorImpl.java:62)
>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>>> thodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
>>> ler.invoke(HelixStateTransitionHandler.java:344)
>>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
>>> ler.handleMessage(HelixStateTransitionHandler.java:290)
>>> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:84)
>>> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:41)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>>> Executor.java:1149)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>>> lExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Caused by: java.lang.IllegalStateException: No callback implemented for
>>> task COMMAND
>>> at org.apache.helix.task.TaskStateModel.startTask(TaskStateMode
>>> l.java:302)
>>> at org.apache.helix.task.TaskStateModel.onBecomeRunningFromInit
>>> (TaskStateModel.java:69)
>>> ... 12 more
>>> 675900 [pool-2-thread-8] ERROR org.apache.helix.participant.statemachine.StateModel
>>> - Default rollback method invoked on error. Error Code: ERROR
>>> 675908 [pool-2-thread-8] ERROR org.apache.helix.messaging.handling.HelixTask
>>> - Message execution failed. msgId: 8d9a08c9-0ae7-419a-8940-6b8e6e7c1ecb,
>>> errorMsg: java.lang.reflect.InvocationTargetException
>>> 675922 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
>>> dling.HelixStateTransitionHandler - Skip internal error. errCode:
>>> ERROR, errMsg: null
>>>
>>> Thanks
>>> Dimuthu
>>>
>>
>>
>>
>> --
>> Thanks and regards,
>>
>> Ajinkya Dhamnaskar
>> Student ID : 0003469679
>> Masters (CS)
>> +1 (812) 369- 5416 <(812)%20369-5416>
>>
>
>
--
Thanks and regards,
Ajinkya Dhamnaskar
Student ID : 0003469679
Masters (CS)
+1 (812) 369- 5416
Re: Using job specific Participants in Helix
Posted by Ajinkya Dhamnaskar <ad...@umail.iu.edu>.
Dlmuthu,
This explains your need. Could you please point me to the code? I was
wondering, where and how are we registering callbacks for the participants?
On Thu, Nov 16, 2017 at 7:16 PM, DImuthu Upeksha <dimuthu.upeksha2@gmail.com
> wrote:
> Adding Helix Dev
>
> Hi Ajinkya
>
> Thank you for the explanation.
>
> Let me explain my requirement. [1] is the correct task registry
> configuration in a working participant code. All the transition call backs
> are registered in this code. The problem here is, we have to bundle
> binaries of all the tasks into a single Participant. If we want to change
> one task, we need to rebuild the participant with other tasks as well. What
> I thought is, why can't we build Participants that only do specific set of
> tasks. For example, in the cluster there are 3 participants, Participant 1
> [2], Participant 2 [3] and Participant 3 [4]. If we want to change the
> DataCollecting task, we only need to re build Participant 2. I got above
> error, once I run 3 participants in above configuration. I can understand
> this is due to the missing transition callback of CommandTask in
> Participant 1 as I have purposely commented out it. What I need to know is
> that, is this configuration allowed in Helix architecture or do we have to
> implement Participants that contain all the task implementations as in [1].
> Workflow configuration for both scenarios is here [5].
>
> [1] https://gist.github.com/DImuthuUpe/41c0db579e7d86d101d112f07ed6ea00
> [2] https://gist.github.com/DImuthuUpe/ec72df1ec3207ce2dce88ff7f1756da4
> [3] https://gist.github.com/DImuthuUpe/4d193a3dff3008315efa2e31f6209cac
> [4] https://gist.github.com/DImuthuUpe/872c432935b8d33944dd571b3ac4207b
> [5] https://gist.github.com/DImuthuUpe/f61851b68b685b8d6744689dc130babd
>
> Thanks
> Dimuthu
>
> On Fri, Nov 17, 2017 at 3:21 AM, Ajinkya Dhamnaskar <adhamnas@umail.iu.edu
> > wrote:
>
>> Hey Dlmuthu,
>>
>> Not an expert in Helix, but from exceptions it seems, system is entering
>> in a state not expected by reflection. I feel https://github.com/apache
>> /helix/blob/master/helix-core/src/main/java/org/apache/helix
>> /messaging/handling/HelixStateTransitionHandler.java#L295 is triggering
>> this exception.
>> As mentioned in the later part of the stack trace and from Helix Apache
>> Docs <http://helix.apache.org/0.7.1-docs/tutorial_agent.html> *("Helix
>> is built on the following assumption: if your distributed resource is
>> modeled by a finite state machine, then Helix can tell participants when
>> they should transition between states. In the Java API, this means
>> implementing transition callbacks. In the Helix agent API, this means
>> providing commands than can run for each transition"),* did you
>> implement *transition callback* for these tasks?
>>
>> On Thu, Nov 16, 2017 at 10:01 AM, DImuthu Upeksha <
>> dimuthu.upeksha2@gmail.com> wrote:
>>
>>> Hi Devs,
>>>
>>> I'm working on the technology evaluation to re architecture Apache
>>> Airavata task execution framework and Helix seems like a good candidate for
>>> that as it has an in built distributed generic workflow execution
>>> capability. After going through several tutorials, I tried to implement a
>>> simple workflow on Helix to demonstrate following transition
>>>
>>> Data Collecting Job -> Command Executing Job -> Data Uploading Job
>>>
>>> I managed to implement this using a Participant node that includes all
>>> the tasks required for above workflow. However my goal is to implement
>>> specialized Participants for each Job type. For example, Participant 1
>>> knows only about the tasks to perform Data Collecting Job and Participant
>>> knows only about the task to perform Command Executing Job. When I tried to
>>> implement such Participants, I got following error from Participant 1. I
>>> can share the code samples that I have tried but before that I need to know
>>> whether my approach is compatible with Helix's design? Does Helix require
>>> all the Participants to be homogeneous?
>>>
>>> Executing data collecting
>>> 675892 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
>>> dling.HelixStateTransitionHandler - Exception while executing a state
>>> transition task Airavata_Workflow_commandExecuteJob_0
>>> java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>>> ssorImpl.java:62)
>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>>> thodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
>>> ler.invoke(HelixStateTransitionHandler.java:344)
>>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
>>> ler.handleMessage(HelixStateTransitionHandler.java:290)
>>> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:84)
>>> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:41)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>>> Executor.java:1149)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>>> lExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Caused by: java.lang.IllegalStateException: No callback implemented for
>>> task COMMAND
>>> at org.apache.helix.task.TaskStateModel.startTask(TaskStateMode
>>> l.java:302)
>>> at org.apache.helix.task.TaskStateModel.onBecomeRunningFromInit
>>> (TaskStateModel.java:69)
>>> ... 12 more
>>> 675900 [pool-2-thread-8] ERROR org.apache.helix.participant.statemachine.StateModel
>>> - Default rollback method invoked on error. Error Code: ERROR
>>> 675908 [pool-2-thread-8] ERROR org.apache.helix.messaging.handling.HelixTask
>>> - Message execution failed. msgId: 8d9a08c9-0ae7-419a-8940-6b8e6e7c1ecb,
>>> errorMsg: java.lang.reflect.InvocationTargetException
>>> 675922 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
>>> dling.HelixStateTransitionHandler - Skip internal error. errCode:
>>> ERROR, errMsg: null
>>>
>>> Thanks
>>> Dimuthu
>>>
>>
>>
>>
>> --
>> Thanks and regards,
>>
>> Ajinkya Dhamnaskar
>> Student ID : 0003469679
>> Masters (CS)
>> +1 (812) 369- 5416 <(812)%20369-5416>
>>
>
>
--
Thanks and regards,
Ajinkya Dhamnaskar
Student ID : 0003469679
Masters (CS)
+1 (812) 369- 5416
Re: Using job specific Participants in Helix
Posted by DImuthu Upeksha <di...@gmail.com>.
Adding Helix Dev
Hi Ajinkya
Thank you for the explanation.
Let me explain my requirement. [1] is the correct task registry
configuration in a working participant code. All the transition call backs
are registered in this code. The problem here is, we have to bundle
binaries of all the tasks into a single Participant. If we want to change
one task, we need to rebuild the participant with other tasks as well. What
I thought is, why can't we build Participants that only do specific set of
tasks. For example, in the cluster there are 3 participants, Participant 1
[2], Participant 2 [3] and Participant 3 [4]. If we want to change the
DataCollecting task, we only need to re build Participant 2. I got above
error, once I run 3 participants in above configuration. I can understand
this is due to the missing transition callback of CommandTask in
Participant 1 as I have purposely commented out it. What I need to know is
that, is this configuration allowed in Helix architecture or do we have to
implement Participants that contain all the task implementations as in [1].
Workflow configuration for both scenarios is here [5].
[1] https://gist.github.com/DImuthuUpe/41c0db579e7d86d101d112f07ed6ea00
[2] https://gist.github.com/DImuthuUpe/ec72df1ec3207ce2dce88ff7f1756da4
[3] https://gist.github.com/DImuthuUpe/4d193a3dff3008315efa2e31f6209cac
[4] https://gist.github.com/DImuthuUpe/872c432935b8d33944dd571b3ac4207b
[5] https://gist.github.com/DImuthuUpe/f61851b68b685b8d6744689dc130babd
Thanks
Dimuthu
On Fri, Nov 17, 2017 at 3:21 AM, Ajinkya Dhamnaskar <ad...@umail.iu.edu>
wrote:
> Hey Dlmuthu,
>
> Not an expert in Helix, but from exceptions it seems, system is entering
> in a state not expected by reflection. I feel https://github.com/apache
> /helix/blob/master/helix-core/src/main/java/org/apache/
> helix/messaging/handling/HelixStateTransitionHandler.java#L295 is
> triggering this exception.
> As mentioned in the later part of the stack trace and from Helix Apache
> Docs <http://helix.apache.org/0.7.1-docs/tutorial_agent.html> *("Helix is
> built on the following assumption: if your distributed resource is modeled
> by a finite state machine, then Helix can tell participants when they
> should transition between states. In the Java API, this means implementing
> transition callbacks. In the Helix agent API, this means providing commands
> than can run for each transition"),* did you implement *transition
> callback* for these tasks?
>
> On Thu, Nov 16, 2017 at 10:01 AM, DImuthu Upeksha <
> dimuthu.upeksha2@gmail.com> wrote:
>
>> Hi Devs,
>>
>> I'm working on the technology evaluation to re architecture Apache
>> Airavata task execution framework and Helix seems like a good candidate for
>> that as it has an in built distributed generic workflow execution
>> capability. After going through several tutorials, I tried to implement a
>> simple workflow on Helix to demonstrate following transition
>>
>> Data Collecting Job -> Command Executing Job -> Data Uploading Job
>>
>> I managed to implement this using a Participant node that includes all
>> the tasks required for above workflow. However my goal is to implement
>> specialized Participants for each Job type. For example, Participant 1
>> knows only about the tasks to perform Data Collecting Job and Participant
>> knows only about the task to perform Command Executing Job. When I tried to
>> implement such Participants, I got following error from Participant 1. I
>> can share the code samples that I have tried but before that I need to know
>> whether my approach is compatible with Helix's design? Does Helix require
>> all the Participants to be homogeneous?
>>
>> Executing data collecting
>> 675892 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
>> dling.HelixStateTransitionHandler - Exception while executing a state
>> transition task Airavata_Workflow_commandExecuteJob_0
>> java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
>> ler.invoke(HelixStateTransitionHandler.java:344)
>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
>> ler.handleMessage(HelixStateTransitionHandler.java:290)
>> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:84)
>> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:41)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1149)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: java.lang.IllegalStateException: No callback implemented for
>> task COMMAND
>> at org.apache.helix.task.TaskStateModel.startTask(TaskStateMode
>> l.java:302)
>> at org.apache.helix.task.TaskStateModel.onBecomeRunningFromInit
>> (TaskStateModel.java:69)
>> ... 12 more
>> 675900 [pool-2-thread-8] ERROR org.apache.helix.participant.statemachine.StateModel
>> - Default rollback method invoked on error. Error Code: ERROR
>> 675908 [pool-2-thread-8] ERROR org.apache.helix.messaging.handling.HelixTask
>> - Message execution failed. msgId: 8d9a08c9-0ae7-419a-8940-6b8e6e7c1ecb,
>> errorMsg: java.lang.reflect.InvocationTargetException
>> 675922 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
>> dling.HelixStateTransitionHandler - Skip internal error. errCode:
>> ERROR, errMsg: null
>>
>> Thanks
>> Dimuthu
>>
>
>
>
> --
> Thanks and regards,
>
> Ajinkya Dhamnaskar
> Student ID : 0003469679
> Masters (CS)
> +1 (812) 369- 5416 <(812)%20369-5416>
>
Re: Using job specific Participants in Helix
Posted by DImuthu Upeksha <di...@gmail.com>.
Adding Helix Dev
Hi Ajinkya
Thank you for the explanation.
Let me explain my requirement. [1] is the correct task registry
configuration in a working participant code. All the transition call backs
are registered in this code. The problem here is, we have to bundle
binaries of all the tasks into a single Participant. If we want to change
one task, we need to rebuild the participant with other tasks as well. What
I thought is, why can't we build Participants that only do specific set of
tasks. For example, in the cluster there are 3 participants, Participant 1
[2], Participant 2 [3] and Participant 3 [4]. If we want to change the
DataCollecting task, we only need to re build Participant 2. I got above
error, once I run 3 participants in above configuration. I can understand
this is due to the missing transition callback of CommandTask in
Participant 1 as I have purposely commented out it. What I need to know is
that, is this configuration allowed in Helix architecture or do we have to
implement Participants that contain all the task implementations as in [1].
Workflow configuration for both scenarios is here [5].
[1] https://gist.github.com/DImuthuUpe/41c0db579e7d86d101d112f07ed6ea00
[2] https://gist.github.com/DImuthuUpe/ec72df1ec3207ce2dce88ff7f1756da4
[3] https://gist.github.com/DImuthuUpe/4d193a3dff3008315efa2e31f6209cac
[4] https://gist.github.com/DImuthuUpe/872c432935b8d33944dd571b3ac4207b
[5] https://gist.github.com/DImuthuUpe/f61851b68b685b8d6744689dc130babd
Thanks
Dimuthu
On Fri, Nov 17, 2017 at 3:21 AM, Ajinkya Dhamnaskar <ad...@umail.iu.edu>
wrote:
> Hey Dlmuthu,
>
> Not an expert in Helix, but from exceptions it seems, system is entering
> in a state not expected by reflection. I feel https://github.com/apache
> /helix/blob/master/helix-core/src/main/java/org/apache/
> helix/messaging/handling/HelixStateTransitionHandler.java#L295 is
> triggering this exception.
> As mentioned in the later part of the stack trace and from Helix Apache
> Docs <http://helix.apache.org/0.7.1-docs/tutorial_agent.html> *("Helix is
> built on the following assumption: if your distributed resource is modeled
> by a finite state machine, then Helix can tell participants when they
> should transition between states. In the Java API, this means implementing
> transition callbacks. In the Helix agent API, this means providing commands
> than can run for each transition"),* did you implement *transition
> callback* for these tasks?
>
> On Thu, Nov 16, 2017 at 10:01 AM, DImuthu Upeksha <
> dimuthu.upeksha2@gmail.com> wrote:
>
>> Hi Devs,
>>
>> I'm working on the technology evaluation to re architecture Apache
>> Airavata task execution framework and Helix seems like a good candidate for
>> that as it has an in built distributed generic workflow execution
>> capability. After going through several tutorials, I tried to implement a
>> simple workflow on Helix to demonstrate following transition
>>
>> Data Collecting Job -> Command Executing Job -> Data Uploading Job
>>
>> I managed to implement this using a Participant node that includes all
>> the tasks required for above workflow. However my goal is to implement
>> specialized Participants for each Job type. For example, Participant 1
>> knows only about the tasks to perform Data Collecting Job and Participant
>> knows only about the task to perform Command Executing Job. When I tried to
>> implement such Participants, I got following error from Participant 1. I
>> can share the code samples that I have tried but before that I need to know
>> whether my approach is compatible with Helix's design? Does Helix require
>> all the Participants to be homogeneous?
>>
>> Executing data collecting
>> 675892 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
>> dling.HelixStateTransitionHandler - Exception while executing a state
>> transition task Airavata_Workflow_commandExecuteJob_0
>> java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
>> ler.invoke(HelixStateTransitionHandler.java:344)
>> at org.apache.helix.messaging.handling.HelixStateTransitionHand
>> ler.handleMessage(HelixStateTransitionHandler.java:290)
>> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:84)
>> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:41)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1149)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: java.lang.IllegalStateException: No callback implemented for
>> task COMMAND
>> at org.apache.helix.task.TaskStateModel.startTask(TaskStateMode
>> l.java:302)
>> at org.apache.helix.task.TaskStateModel.onBecomeRunningFromInit
>> (TaskStateModel.java:69)
>> ... 12 more
>> 675900 [pool-2-thread-8] ERROR org.apache.helix.participant.statemachine.StateModel
>> - Default rollback method invoked on error. Error Code: ERROR
>> 675908 [pool-2-thread-8] ERROR org.apache.helix.messaging.handling.HelixTask
>> - Message execution failed. msgId: 8d9a08c9-0ae7-419a-8940-6b8e6e7c1ecb,
>> errorMsg: java.lang.reflect.InvocationTargetException
>> 675922 [pool-2-thread-8] ERROR org.apache.helix.messaging.han
>> dling.HelixStateTransitionHandler - Skip internal error. errCode:
>> ERROR, errMsg: null
>>
>> Thanks
>> Dimuthu
>>
>
>
>
> --
> Thanks and regards,
>
> Ajinkya Dhamnaskar
> Student ID : 0003469679
> Masters (CS)
> +1 (812) 369- 5416 <(812)%20369-5416>
>
Re: Using job specific Participants in Helix
Posted by Ajinkya Dhamnaskar <ad...@umail.iu.edu>.
Hey Dlmuthu,
Not an expert in Helix, but from exceptions it seems, system is entering in
a state not expected by reflection. I feel
https://github.com/apache/helix/blob/master/helix-core/src/main/java/org/apache/helix/messaging/handling/HelixStateTransitionHandler.java#L295
is triggering this exception.
As mentioned in the later part of the stack trace and from Helix Apache Docs
<http://helix.apache.org/0.7.1-docs/tutorial_agent.html> *("Helix is built
on the following assumption: if your distributed resource is modeled by a
finite state machine, then Helix can tell participants when they should
transition between states. In the Java API, this means implementing
transition callbacks. In the Helix agent API, this means providing commands
than can run for each transition"),* did you implement *transition
callback* for
these tasks?
On Thu, Nov 16, 2017 at 10:01 AM, DImuthu Upeksha <
dimuthu.upeksha2@gmail.com> wrote:
> Hi Devs,
>
> I'm working on the technology evaluation to re architecture Apache
> Airavata task execution framework and Helix seems like a good candidate for
> that as it has an in built distributed generic workflow execution
> capability. After going through several tutorials, I tried to implement a
> simple workflow on Helix to demonstrate following transition
>
> Data Collecting Job -> Command Executing Job -> Data Uploading Job
>
> I managed to implement this using a Participant node that includes all the
> tasks required for above workflow. However my goal is to implement
> specialized Participants for each Job type. For example, Participant 1
> knows only about the tasks to perform Data Collecting Job and Participant
> knows only about the task to perform Command Executing Job. When I tried to
> implement such Participants, I got following error from Participant 1. I
> can share the code samples that I have tried but before that I need to know
> whether my approach is compatible with Helix's design? Does Helix require
> all the Participants to be homogeneous?
>
> Executing data collecting
> 675892 [pool-2-thread-8] ERROR org.apache.helix.messaging.handling.HelixStateTransitionHandler
> - Exception while executing a state transition task Airavata_Workflow_
> commandExecuteJob_0
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(
> HelixStateTransitionHandler.java:344)
> at org.apache.helix.messaging.handling.HelixStateTransitionHandler.
> handleMessage(HelixStateTransitionHandler.java:290)
> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:84)
> at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:41)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.IllegalStateException: No callback implemented for
> task COMMAND
> at org.apache.helix.task.TaskStateModel.startTask(TaskStateModel.java:302)
> at org.apache.helix.task.TaskStateModel.onBecomeRunningFromInit(
> TaskStateModel.java:69)
> ... 12 more
> 675900 [pool-2-thread-8] ERROR org.apache.helix.participant.statemachine.StateModel
> - Default rollback method invoked on error. Error Code: ERROR
> 675908 [pool-2-thread-8] ERROR org.apache.helix.messaging.handling.HelixTask
> - Message execution failed. msgId: 8d9a08c9-0ae7-419a-8940-6b8e6e7c1ecb,
> errorMsg: java.lang.reflect.InvocationTargetException
> 675922 [pool-2-thread-8] ERROR org.apache.helix.messaging.handling.HelixStateTransitionHandler
> - Skip internal error. errCode: ERROR, errMsg: null
>
> Thanks
> Dimuthu
>
--
Thanks and regards,
Ajinkya Dhamnaskar
Student ID : 0003469679
Masters (CS)
+1 (812) 369- 5416