You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by "York, Brennon" <Br...@capitalone.com> on 2015/08/24 23:09:06 UTC
Issues submitting app to YARN
All, seeing some odd behavior when submitting a job through `dtcli` that, I believe has something to do with Malhar(??). We’re using both `malhar-contrib` and `malhar-library` in the application and, when both are provided as `scope: provided`, the app will fail with the below (full stack at bottom):
java.lang.NoClassDefFoundError: com/datatorrent/lib/io/fs/AbstractFileInputOperator
Now, when we package both into the resultant jar, the application then fails at the YARN level with a statement like:
Application application_1440084097966_0009 failed 2 times due to AM Container for appattempt_1440084097966_0009_000002 exited with exitCode: 1
>From the YARN Resource Manager. Digging deeper into the YARN container logs only provides the following exception:
Exception in thread "main" java.lang.AbstractMethodError: org.apache.hadoop.yarn.api.records.ContainerId.setContainerId(J)V
at org.apache.hadoop.yarn.api.records.ContainerId.newInstance(ContainerId.java:60)
at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:178)
at com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:87)
Any thoughts on this and why, when we jar the malhar packages into our app jar, it fails like this? Could this be a version mismatch?
For reference we’re using the DT Community Edition and building against Malhar v3.0.0 as well as DT v3.0.0.
Full stack trace when `scope: provided` for the Malhar packages:
java.lang.NoClassDefFoundError: com/datatorrent/lib/io/fs/AbstractFileInputOperator
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at com.capitalone.vault8.citadel.H20Application.populateDAG(H20Application.java:40)
at com.datatorrent.stram.plan.logical.LogicalPlanConfiguration.prepareDAG(LogicalPlanConfiguration.java:1171)
at com.datatorrent.stram.client.StramAppLauncher$1.createApp(StramAppLauncher.java:404)
at com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:479)
at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2048)
at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3452)
at com.datatorrent.stram.cli.DTCli.access$7000(DTCli.java:104)
at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1893)
at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1447)
Caused by: java.lang.ClassNotFoundException: com.datatorrent.lib.io.fs.AbstractFileInputOperator
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 21 more
________________________________________________________
The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
Re: Issues submitting app to YARN
Posted by "York, Brennon" <Br...@capitalone.com>.
That did it! Thanks!
On 8/24/15, 2:22 PM, "Gaurav Gupta" <ga...@datatorrent.com> wrote:
>Looks like you are bundling hadoop jars in the app package and the
>version is not same as available in hadoop class path.
>You should not package hadoop jars in your app package.
>
>Thanks
>- Gaurav
>
>> On Aug 24, 2015, at 2:18 PM, York, Brennon
>><Br...@capitalone.com> wrote:
>>
>> Awesome! That¹s what we figured, but we¹re still not understanding why
>>the
>> containers are failing for YARN. Any thoughts there?
>>
>> On 8/24/15, 2:11 PM, "Gaurav Gupta" <ga...@datatorrent.com> wrote:
>>
>>> The scope should not be provided.
>>>
>>> Thanks
>>> - Gaurav
>>>
>>>> On Aug 24, 2015, at 2:09 PM, York, Brennon
>>>> <Br...@capitalone.com> wrote:
>>>>
>>>> All, seeing some odd behavior when submitting a job through `dtcli`
>>>> that, I believe has something to do with Malhar(??). We¹re using both
>>>> `malhar-contrib` and `malhar-library` in the application and, when
>>>>both
>>>> are provided as `scope: provided`, the app will fail with the below
>>>> (full stack at bottom):
>>>>
>>>> java.lang.NoClassDefFoundError:
>>>> com/datatorrent/lib/io/fs/AbstractFileInputOperator
>>>>
>>>> Now, when we package both into the resultant jar, the application then
>>>> fails at the YARN level with a statement like:
>>>>
>>>> Application application_1440084097966_0009 failed 2 times due to AM
>>>> Container for appattempt_1440084097966_0009_000002 exited with
>>>>exitCode:
>>>> 1
>>>>
>>>> From the YARN Resource Manager. Digging deeper into the YARN container
>>>> logs only provides the following exception:
>>>>
>>>> Exception in thread "main" java.lang.AbstractMethodError:
>>>> org.apache.hadoop.yarn.api.records.ContainerId.setContainerId(J)V
>>>> at
>>>>
>>>>org.apache.hadoop.yarn.api.records.ContainerId.newInstance(ContainerId.
>>>>ja
>>>> va:60)
>>>> at
>>>>
>>>>org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils
>>>>.j
>>>> ava:178)
>>>> at
>>>>
>>>>com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:8
>>>>7)
>>>>
>>>> Any thoughts on this and why, when we jar the malhar packages into our
>>>> app jar, it fails like this? Could this be a version mismatch?
>>>>
>>>> For reference we¹re using the DT Community Edition and building
>>>>against
>>>> Malhar v3.0.0 as well as DT v3.0.0.
>>>>
>>>> Full stack trace when `scope: provided` for the Malhar packages:
>>>> java.lang.NoClassDefFoundError:
>>>> com/datatorrent/lib/io/fs/AbstractFileInputOperator
>>>> at java.lang.ClassLoader.defineClass1(Native Method)
>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>>>> at
>>>>
>>>>java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>>> at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>>> at
>>>>
>>>>com.capitalone.vault8.citadel.H20Application.populateDAG(H20Application
>>>>.j
>>>> ava:40)
>>>> at
>>>>
>>>>com.datatorrent.stram.plan.logical.LogicalPlanConfiguration.prepareDAG(
>>>>Lo
>>>> gicalPlanConfiguration.java:1171)
>>>> at
>>>>
>>>>com.datatorrent.stram.client.StramAppLauncher$1.createApp(StramAppLaunc
>>>>he
>>>> r.java:404)
>>>> at
>>>>
>>>>com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLaunche
>>>>r.
>>>> java:479)
>>>> at
>>>> com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2048)
>>>> at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3452)
>>>> at com.datatorrent.stram.cli.DTCli.access$7000(DTCli.java:104)
>>>> at
>>>> com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1893)
>>>> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1447)
>>>> Caused by: java.lang.ClassNotFoundException:
>>>> com.datatorrent.lib.io.fs.AbstractFileInputOperator
>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>>> at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>>> ... 21 more
>>>> ________________________________________________________
>>>>
>>>> The information contained in this e-mail is confidential and/or
>>>> proprietary to Capital One and/or its affiliates and may only be used
>>>> solely in performance of work or services for Capital One. The
>>>> information transmitted herewith is intended only for use by the
>>>> individual or entity to which it is addressed. If the reader of this
>>>> message is not the intended recipient, you are hereby notified that
>>>>any
>>>> review, retransmission, dissemination, distribution, copying or other
>>>> use of, or taking of any action in reliance upon this information is
>>>> strictly prohibited. If you have received this communication in error,
>>>> please contact the sender and delete the material from your computer.
>>>
>>
>> ________________________________________________________
>>
>> The information contained in this e-mail is confidential and/or
>>proprietary to Capital One and/or its affiliates and may only be used
>>solely in performance of work or services for Capital One. The
>>information transmitted herewith is intended only for use by the
>>individual or entity to which it is addressed. If the reader of this
>>message is not the intended recipient, you are hereby notified that any
>>review, retransmission, dissemination, distribution, copying or other
>>use of, or taking of any action in reliance upon this information is
>>strictly prohibited. If you have received this communication in error,
>>please contact the sender and delete the material from your computer.
>>
>
________________________________________________________
The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
Re: Issues submitting app to YARN
Posted by Gaurav Gupta <ga...@datatorrent.com>.
Looks like you are bundling hadoop jars in the app package and the version is not same as available in hadoop class path.
You should not package hadoop jars in your app package.
Thanks
- Gaurav
> On Aug 24, 2015, at 2:18 PM, York, Brennon <Br...@capitalone.com> wrote:
>
> Awesome! That¹s what we figured, but we¹re still not understanding why the
> containers are failing for YARN. Any thoughts there?
>
> On 8/24/15, 2:11 PM, "Gaurav Gupta" <ga...@datatorrent.com> wrote:
>
>> The scope should not be provided.
>>
>> Thanks
>> - Gaurav
>>
>>> On Aug 24, 2015, at 2:09 PM, York, Brennon
>>> <Br...@capitalone.com> wrote:
>>>
>>> All, seeing some odd behavior when submitting a job through `dtcli`
>>> that, I believe has something to do with Malhar(??). We¹re using both
>>> `malhar-contrib` and `malhar-library` in the application and, when both
>>> are provided as `scope: provided`, the app will fail with the below
>>> (full stack at bottom):
>>>
>>> java.lang.NoClassDefFoundError:
>>> com/datatorrent/lib/io/fs/AbstractFileInputOperator
>>>
>>> Now, when we package both into the resultant jar, the application then
>>> fails at the YARN level with a statement like:
>>>
>>> Application application_1440084097966_0009 failed 2 times due to AM
>>> Container for appattempt_1440084097966_0009_000002 exited with exitCode:
>>> 1
>>>
>>> From the YARN Resource Manager. Digging deeper into the YARN container
>>> logs only provides the following exception:
>>>
>>> Exception in thread "main" java.lang.AbstractMethodError:
>>> org.apache.hadoop.yarn.api.records.ContainerId.setContainerId(J)V
>>> at
>>> org.apache.hadoop.yarn.api.records.ContainerId.newInstance(ContainerId.ja
>>> va:60)
>>> at
>>> org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.j
>>> ava:178)
>>> at
>>> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:87)
>>>
>>> Any thoughts on this and why, when we jar the malhar packages into our
>>> app jar, it fails like this? Could this be a version mismatch?
>>>
>>> For reference we¹re using the DT Community Edition and building against
>>> Malhar v3.0.0 as well as DT v3.0.0.
>>>
>>> Full stack trace when `scope: provided` for the Malhar packages:
>>> java.lang.NoClassDefFoundError:
>>> com/datatorrent/lib/io/fs/AbstractFileInputOperator
>>> at java.lang.ClassLoader.defineClass1(Native Method)
>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>>> at
>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> at
>>> com.capitalone.vault8.citadel.H20Application.populateDAG(H20Application.j
>>> ava:40)
>>> at
>>> com.datatorrent.stram.plan.logical.LogicalPlanConfiguration.prepareDAG(Lo
>>> gicalPlanConfiguration.java:1171)
>>> at
>>> com.datatorrent.stram.client.StramAppLauncher$1.createApp(StramAppLaunche
>>> r.java:404)
>>> at
>>> com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.
>>> java:479)
>>> at
>>> com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2048)
>>> at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3452)
>>> at com.datatorrent.stram.cli.DTCli.access$7000(DTCli.java:104)
>>> at
>>> com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1893)
>>> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1447)
>>> Caused by: java.lang.ClassNotFoundException:
>>> com.datatorrent.lib.io.fs.AbstractFileInputOperator
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> ... 21 more
>>> ________________________________________________________
>>>
>>> The information contained in this e-mail is confidential and/or
>>> proprietary to Capital One and/or its affiliates and may only be used
>>> solely in performance of work or services for Capital One. The
>>> information transmitted herewith is intended only for use by the
>>> individual or entity to which it is addressed. If the reader of this
>>> message is not the intended recipient, you are hereby notified that any
>>> review, retransmission, dissemination, distribution, copying or other
>>> use of, or taking of any action in reliance upon this information is
>>> strictly prohibited. If you have received this communication in error,
>>> please contact the sender and delete the material from your computer.
>>
>
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
>
Re: Issues submitting app to YARN
Posted by "York, Brennon" <Br...@capitalone.com>.
Awesome! That¹s what we figured, but we¹re still not understanding why the
containers are failing for YARN. Any thoughts there?
On 8/24/15, 2:11 PM, "Gaurav Gupta" <ga...@datatorrent.com> wrote:
>The scope should not be provided.
>
>Thanks
>- Gaurav
>
>> On Aug 24, 2015, at 2:09 PM, York, Brennon
>><Br...@capitalone.com> wrote:
>>
>> All, seeing some odd behavior when submitting a job through `dtcli`
>>that, I believe has something to do with Malhar(??). We¹re using both
>>`malhar-contrib` and `malhar-library` in the application and, when both
>>are provided as `scope: provided`, the app will fail with the below
>>(full stack at bottom):
>>
>> java.lang.NoClassDefFoundError:
>>com/datatorrent/lib/io/fs/AbstractFileInputOperator
>>
>> Now, when we package both into the resultant jar, the application then
>>fails at the YARN level with a statement like:
>>
>> Application application_1440084097966_0009 failed 2 times due to AM
>>Container for appattempt_1440084097966_0009_000002 exited with exitCode:
>>1
>>
>> From the YARN Resource Manager. Digging deeper into the YARN container
>>logs only provides the following exception:
>>
>> Exception in thread "main" java.lang.AbstractMethodError:
>>org.apache.hadoop.yarn.api.records.ContainerId.setContainerId(J)V
>> at
>>org.apache.hadoop.yarn.api.records.ContainerId.newInstance(ContainerId.ja
>>va:60)
>> at
>>org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.j
>>ava:178)
>> at
>>com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:87)
>>
>> Any thoughts on this and why, when we jar the malhar packages into our
>>app jar, it fails like this? Could this be a version mismatch?
>>
>> For reference we¹re using the DT Community Edition and building against
>>Malhar v3.0.0 as well as DT v3.0.0.
>>
>> Full stack trace when `scope: provided` for the Malhar packages:
>> java.lang.NoClassDefFoundError:
>>com/datatorrent/lib/io/fs/AbstractFileInputOperator
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>> at
>>java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> at
>>com.capitalone.vault8.citadel.H20Application.populateDAG(H20Application.j
>>ava:40)
>> at
>>com.datatorrent.stram.plan.logical.LogicalPlanConfiguration.prepareDAG(Lo
>>gicalPlanConfiguration.java:1171)
>> at
>>com.datatorrent.stram.client.StramAppLauncher$1.createApp(StramAppLaunche
>>r.java:404)
>> at
>>com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.
>>java:479)
>> at
>>com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2048)
>> at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3452)
>> at com.datatorrent.stram.cli.DTCli.access$7000(DTCli.java:104)
>> at
>>com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1893)
>> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1447)
>> Caused by: java.lang.ClassNotFoundException:
>>com.datatorrent.lib.io.fs.AbstractFileInputOperator
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> ... 21 more
>> ________________________________________________________
>>
>> The information contained in this e-mail is confidential and/or
>>proprietary to Capital One and/or its affiliates and may only be used
>>solely in performance of work or services for Capital One. The
>>information transmitted herewith is intended only for use by the
>>individual or entity to which it is addressed. If the reader of this
>>message is not the intended recipient, you are hereby notified that any
>>review, retransmission, dissemination, distribution, copying or other
>>use of, or taking of any action in reliance upon this information is
>>strictly prohibited. If you have received this communication in error,
>>please contact the sender and delete the material from your computer.
>
________________________________________________________
The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
Re: Issues submitting app to YARN
Posted by Gaurav Gupta <ga...@datatorrent.com>.
The scope should not be provided.
Thanks
- Gaurav
> On Aug 24, 2015, at 2:09 PM, York, Brennon <Br...@capitalone.com> wrote:
>
> All, seeing some odd behavior when submitting a job through `dtcli` that, I believe has something to do with Malhar(??). We’re using both `malhar-contrib` and `malhar-library` in the application and, when both are provided as `scope: provided`, the app will fail with the below (full stack at bottom):
>
> java.lang.NoClassDefFoundError: com/datatorrent/lib/io/fs/AbstractFileInputOperator
>
> Now, when we package both into the resultant jar, the application then fails at the YARN level with a statement like:
>
> Application application_1440084097966_0009 failed 2 times due to AM Container for appattempt_1440084097966_0009_000002 exited with exitCode: 1
>
> From the YARN Resource Manager. Digging deeper into the YARN container logs only provides the following exception:
>
> Exception in thread "main" java.lang.AbstractMethodError: org.apache.hadoop.yarn.api.records.ContainerId.setContainerId(J)V
> at org.apache.hadoop.yarn.api.records.ContainerId.newInstance(ContainerId.java:60)
> at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:178)
> at com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:87)
>
> Any thoughts on this and why, when we jar the malhar packages into our app jar, it fails like this? Could this be a version mismatch?
>
> For reference we’re using the DT Community Edition and building against Malhar v3.0.0 as well as DT v3.0.0.
>
> Full stack trace when `scope: provided` for the Malhar packages:
> java.lang.NoClassDefFoundError: com/datatorrent/lib/io/fs/AbstractFileInputOperator
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at com.capitalone.vault8.citadel.H20Application.populateDAG(H20Application.java:40)
> at com.datatorrent.stram.plan.logical.LogicalPlanConfiguration.prepareDAG(LogicalPlanConfiguration.java:1171)
> at com.datatorrent.stram.client.StramAppLauncher$1.createApp(StramAppLauncher.java:404)
> at com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:479)
> at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2048)
> at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3452)
> at com.datatorrent.stram.cli.DTCli.access$7000(DTCli.java:104)
> at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1893)
> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1447)
> Caused by: java.lang.ClassNotFoundException: com.datatorrent.lib.io.fs.AbstractFileInputOperator
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 21 more
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.