You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tez.apache.org by Madhusudan Ramanna <m....@ymail.com> on 2016/08/25 18:46:48 UTC
Node unable to start vertex
Hello,
I'm trying to extend TezExamplesBase and get a dag running on yarn (pseudo cluster mode on my host).
For some reason, I'm running into class not found exception on the node
Vertex failed, vertexName=v1, vertexId=vertex_1471907702278_0030_1_00, diagnostics=[Task failed, taskId=task_1471907702278_0030_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1471907702278_0030_1_00_000000_0:org.apache.tez.dag.api.TezReflectionException: Unable to load class: sample.sampletez.OnetoOne$V1Processorat org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:46)at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:87)at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:668)
Should I be adding my application jar somewhere so that it can get distributed ? I tried adding my jar via tezClient.addAppMasterLocalFiles() but it didn't help.
What am I not doing ?
thanks!Madhu
Re: Node unable to start vertex
Posted by Madhusudan Ramanna <m....@ymail.com>.
Have added an entry
https://cwiki.apache.org/confluence/display/TEZ/FAQ
On Thursday, August 25, 2016 1:36 PM, Hitesh Shah <hi...@apache.org> wrote:
Created https://cwiki.apache.org/confluence/display/TEZ/FAQ which might be a better fit for such content and other related questions down the line.
> On Aug 25, 2016, at 1:16 PM, Hitesh Shah <hi...@apache.org> wrote:
>
> +1. Would you like to contribute the content? You should be able to add an article under https://cwiki.apache.org/confluence/display/TEZ/Troubleshooting+articles.
>
> If you hit any permission issues, feel free to reply back with your confluence id.
>
> thanks
> — Hitesh
>
>
>> On Aug 25, 2016, at 12:59 PM, Madhusudan Ramanna <m....@ymail.com> wrote:
>>
>> Thanks, #2 worked !
>>
>> Might be a good idea to add to confluence ?
>>
>> Madhu
>>
>>
>> On Thursday, August 25, 2016 12:00 PM, Hitesh Shah <hi...@apache.org> wrote:
>>
>>
>> Hello Madhu,
>>
>> There are 2 approaches for this:
>>
>> 1) Programmatically, for user code running in tasks, you would need to use either DAG::addTaskLocalFiles() or Vertex::addTaskLocalFiles() - former if the same jars are needed in all tasks of the DAG. TezClient::addAppMasterLocalFiles only impacts the ApplicationMaster.
>>
>> 2) Configure tez.aux.uris. This will ensure that all files specified here will be available in the AM and all tasks.
>>
>> thanks
>> — Hitesh
>>
>>> On Aug 25, 2016, at 11:46 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
>>>
>>> Hello,
>>>
>>> I'm trying to extend TezExamplesBase and get a dag running on yarn (pseudo cluster mode on my host).
>>>
>>> For some reason, I'm running into class not found exception on the node
>>>
>>> Vertex failed, vertexName=v1, vertexId=vertex_1471907702278_0030_1_00, diagnostics=[Task failed, taskId=task_1471907702278_0030_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1471907702278_0030_1_00_000000_0:org.apache.tez.dag.api.TezReflectionException: Unable to load class: sample.sampletez.OnetoOne$V1Processor
>>> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:46)
>>> at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:87)
>>> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:668)
>>>
>>>
>>> Should I be adding my application jar somewhere so that it can get distributed ? I tried adding my jar via tezClient.addAppMasterLocalFiles() but it didn't help.
>>>
>>> What am I not doing ?
>>>
>>> thanks!
>>> Madhu
>>>
>>>
>>>
>>>
>>>
>>
>>
>
Re: Node unable to start vertex
Posted by Hitesh Shah <hi...@apache.org>.
Created https://cwiki.apache.org/confluence/display/TEZ/FAQ which might be a better fit for such content and other related questions down the line.
> On Aug 25, 2016, at 1:16 PM, Hitesh Shah <hi...@apache.org> wrote:
>
> +1. Would you like to contribute the content? You should be able to add an article under https://cwiki.apache.org/confluence/display/TEZ/Troubleshooting+articles.
>
> If you hit any permission issues, feel free to reply back with your confluence id.
>
> thanks
> — Hitesh
>
>
>> On Aug 25, 2016, at 12:59 PM, Madhusudan Ramanna <m....@ymail.com> wrote:
>>
>> Thanks, #2 worked !
>>
>> Might be a good idea to add to confluence ?
>>
>> Madhu
>>
>>
>> On Thursday, August 25, 2016 12:00 PM, Hitesh Shah <hi...@apache.org> wrote:
>>
>>
>> Hello Madhu,
>>
>> There are 2 approaches for this:
>>
>> 1) Programmatically, for user code running in tasks, you would need to use either DAG::addTaskLocalFiles() or Vertex::addTaskLocalFiles() - former if the same jars are needed in all tasks of the DAG. TezClient::addAppMasterLocalFiles only impacts the ApplicationMaster.
>>
>> 2) Configure tez.aux.uris. This will ensure that all files specified here will be available in the AM and all tasks.
>>
>> thanks
>> — Hitesh
>>
>>> On Aug 25, 2016, at 11:46 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
>>>
>>> Hello,
>>>
>>> I'm trying to extend TezExamplesBase and get a dag running on yarn (pseudo cluster mode on my host).
>>>
>>> For some reason, I'm running into class not found exception on the node
>>>
>>> Vertex failed, vertexName=v1, vertexId=vertex_1471907702278_0030_1_00, diagnostics=[Task failed, taskId=task_1471907702278_0030_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1471907702278_0030_1_00_000000_0:org.apache.tez.dag.api.TezReflectionException: Unable to load class: sample.sampletez.OnetoOne$V1Processor
>>> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:46)
>>> at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:87)
>>> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:668)
>>>
>>>
>>> Should I be adding my application jar somewhere so that it can get distributed ? I tried adding my jar via tezClient.addAppMasterLocalFiles() but it didn't help.
>>>
>>> What am I not doing ?
>>>
>>> thanks!
>>> Madhu
>>>
>>>
>>>
>>>
>>>
>>
>>
>
Re: Node unable to start vertex
Posted by Hitesh Shah <hi...@apache.org>.
+1. Would you like to contribute the content? You should be able to add an article under https://cwiki.apache.org/confluence/display/TEZ/Troubleshooting+articles.
If you hit any permission issues, feel free to reply back with your confluence id.
thanks
— Hitesh
> On Aug 25, 2016, at 12:59 PM, Madhusudan Ramanna <m....@ymail.com> wrote:
>
> Thanks, #2 worked !
>
> Might be a good idea to add to confluence ?
>
> Madhu
>
>
> On Thursday, August 25, 2016 12:00 PM, Hitesh Shah <hi...@apache.org> wrote:
>
>
> Hello Madhu,
>
> There are 2 approaches for this:
>
> 1) Programmatically, for user code running in tasks, you would need to use either DAG::addTaskLocalFiles() or Vertex::addTaskLocalFiles() - former if the same jars are needed in all tasks of the DAG. TezClient::addAppMasterLocalFiles only impacts the ApplicationMaster.
>
> 2) Configure tez.aux.uris. This will ensure that all files specified here will be available in the AM and all tasks.
>
> thanks
> — Hitesh
>
> > On Aug 25, 2016, at 11:46 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
> >
> > Hello,
> >
> > I'm trying to extend TezExamplesBase and get a dag running on yarn (pseudo cluster mode on my host).
> >
> > For some reason, I'm running into class not found exception on the node
> >
> > Vertex failed, vertexName=v1, vertexId=vertex_1471907702278_0030_1_00, diagnostics=[Task failed, taskId=task_1471907702278_0030_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1471907702278_0030_1_00_000000_0:org.apache.tez.dag.api.TezReflectionException: Unable to load class: sample.sampletez.OnetoOne$V1Processor
> > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:46)
> > at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:87)
> > at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:668)
> >
> >
> > Should I be adding my application jar somewhere so that it can get distributed ? I tried adding my jar via tezClient.addAppMasterLocalFiles() but it didn't help.
> >
> > What am I not doing ?
> >
> > thanks!
> > Madhu
> >
> >
> >
> >
> >
>
>
Re: Node unable to start vertex
Posted by Madhusudan Ramanna <m....@ymail.com>.
Thanks, #2 worked !
Might be a good idea to add to confluence ?
Madhu
On Thursday, August 25, 2016 12:00 PM, Hitesh Shah <hi...@apache.org> wrote:
Hello Madhu,
There are 2 approaches for this:
1) Programmatically, for user code running in tasks, you would need to use either DAG::addTaskLocalFiles() or Vertex::addTaskLocalFiles() - former if the same jars are needed in all tasks of the DAG. TezClient::addAppMasterLocalFiles only impacts the ApplicationMaster.
2) Configure tez.aux.uris. This will ensure that all files specified here will be available in the AM and all tasks.
thanks
— Hitesh
> On Aug 25, 2016, at 11:46 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
>
> Hello,
>
> I'm trying to extend TezExamplesBase and get a dag running on yarn (pseudo cluster mode on my host).
>
> For some reason, I'm running into class not found exception on the node
>
> Vertex failed, vertexName=v1, vertexId=vertex_1471907702278_0030_1_00, diagnostics=[Task failed, taskId=task_1471907702278_0030_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1471907702278_0030_1_00_000000_0:org.apache.tez.dag.api.TezReflectionException: Unable to load class: sample.sampletez.OnetoOne$V1Processor
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:46)
> at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:87)
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:668)
>
>
> Should I be adding my application jar somewhere so that it can get distributed ? I tried adding my jar via tezClient.addAppMasterLocalFiles() but it didn't help.
>
> What am I not doing ?
>
> thanks!
> Madhu
>
>
>
>
>
Re: Node unable to start vertex
Posted by Hitesh Shah <hi...@apache.org>.
Hello Madhu,
There are 2 approaches for this:
1) Programmatically, for user code running in tasks, you would need to use either DAG::addTaskLocalFiles() or Vertex::addTaskLocalFiles() - former if the same jars are needed in all tasks of the DAG. TezClient::addAppMasterLocalFiles only impacts the ApplicationMaster.
2) Configure tez.aux.uris. This will ensure that all files specified here will be available in the AM and all tasks.
thanks
— Hitesh
> On Aug 25, 2016, at 11:46 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
>
> Hello,
>
> I'm trying to extend TezExamplesBase and get a dag running on yarn (pseudo cluster mode on my host).
>
> For some reason, I'm running into class not found exception on the node
>
> Vertex failed, vertexName=v1, vertexId=vertex_1471907702278_0030_1_00, diagnostics=[Task failed, taskId=task_1471907702278_0030_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1471907702278_0030_1_00_000000_0:org.apache.tez.dag.api.TezReflectionException: Unable to load class: sample.sampletez.OnetoOne$V1Processor
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:46)
> at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:87)
> at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:668)
>
>
> Should I be adding my application jar somewhere so that it can get distributed ? I tried adding my jar via tezClient.addAppMasterLocalFiles() but it didn't help.
>
> What am I not doing ?
>
> thanks!
> Madhu
>
>
>
>
>