You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Snehal Nagmote <na...@gmail.com> on 2015/01/15 02:05:22 UTC

Issue while running samza on multi node Yarn

Hello ,

I am trying to deploy samza on multi node yarn using HTTPFileSystem.
I am running into classpath issues .
Earlier few times it complained about no file system for Scheme HTTP.

I am using samza 0.8 and yarn 2.5.2

I have samza_yarn_2.10 jar in yarn_home/share/hadoop/hdfs/lib

I also copied it to yarn_home/share/hadoop/yarn/lib

Now I am getting Application application_1421283273623_0002 failed 2 times
due to AM Container for appattempt_1421283273623_0002_000002 exited with
exitCode: -1000 due to: java.lang.ClassNotFoundException: Class
org.apache.samza.util.hadoop.HttpFileSystem not found
.Failing this attempt.. Failing the application.


Any Ideas ?

Thanks,
Snehal

Re: Issue while running samza on multi node Yarn

Posted by Chris Riccomini <cr...@linkedin.com.INVALID>.
Bonus points if anyone wants to post the patch. :)

On 1/14/15 6:02 PM, "Tommy Becker" <to...@Tivo.com> wrote:

>In Samza 0.8.0 you need to copy samza-core to hadoop lib dir also.  See
>my comment here:
>
>https://issues.apache.org/jira/browse/SAMZA-456
>
>________________________________________
>From: Snehal Nagmote [nagmote.snehal@gmail.com]
>Sent: Wednesday, January 14, 2015 8:05 PM
>To: dev@samza.incubator.apache.org
>Subject: Issue while running samza on multi node Yarn
>
>Hello ,
>
>I am trying to deploy samza on multi node yarn using HTTPFileSystem.
>I am running into classpath issues .
>Earlier few times it complained about no file system for Scheme HTTP.
>
>I am using samza 0.8 and yarn 2.5.2
>
>I have samza_yarn_2.10 jar in yarn_home/share/hadoop/hdfs/lib
>
>I also copied it to yarn_home/share/hadoop/yarn/lib
>
>Now I am getting Application application_1421283273623_0002 failed 2 times
>due to AM Container for appattempt_1421283273623_0002_000002 exited with
>exitCode: -1000 due to: java.lang.ClassNotFoundException: Class
>org.apache.samza.util.hadoop.HttpFileSystem not found
>.Failing this attempt.. Failing the application.
>
>
>Any Ideas ?
>
>Thanks,
>Snehal
>
>________________________________
>
>This email and any attachments may contain confidential and privileged
>material for the sole use of the intended recipient. Any review, copying,
>or distribution of this email (or any attachments) by others is
>prohibited. If you are not the intended recipient, please contact the
>sender immediately and permanently delete this email and any attachments.
>No employee or agent of TiVo Inc. is authorized to conclude any binding
>agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo
>Inc. may only be made by a signed written agreement.

RE: Issue while running samza on multi node Yarn

Posted by Tommy Becker <to...@Tivo.com>.
In Samza 0.8.0 you need to copy samza-core to hadoop lib dir also.  See my comment here:

https://issues.apache.org/jira/browse/SAMZA-456

________________________________________
From: Snehal Nagmote [nagmote.snehal@gmail.com]
Sent: Wednesday, January 14, 2015 8:05 PM
To: dev@samza.incubator.apache.org
Subject: Issue while running samza on multi node Yarn

Hello ,

I am trying to deploy samza on multi node yarn using HTTPFileSystem.
I am running into classpath issues .
Earlier few times it complained about no file system for Scheme HTTP.

I am using samza 0.8 and yarn 2.5.2

I have samza_yarn_2.10 jar in yarn_home/share/hadoop/hdfs/lib

I also copied it to yarn_home/share/hadoop/yarn/lib

Now I am getting Application application_1421283273623_0002 failed 2 times
due to AM Container for appattempt_1421283273623_0002_000002 exited with
exitCode: -1000 due to: java.lang.ClassNotFoundException: Class
org.apache.samza.util.hadoop.HttpFileSystem not found
.Failing this attempt.. Failing the application.


Any Ideas ?

Thanks,
Snehal

________________________________

This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo Inc. is authorized to conclude any binding agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo Inc. may only be made by a signed written agreement.

Re: Issue while running samza on multi node Yarn

Posted by Chris Riccomini <cr...@linkedin.com.INVALID>.
Hey Snehal,

Hmm, I have no immediate ideas as to what the problem is.

1. Have you followed
http://samza.incubator.apache.org/learn/tutorials/latest/run-in-multi-node-
yarn.html
2. Have you verified that the JARs are not only on the RM machine, but
also all NM machines as well?
3. Have you verified that the JARs end up on the classpath of the NMs?

Cheers,
Chris

On 1/14/15 5:05 PM, "Snehal Nagmote" <na...@gmail.com> wrote:

>Hello ,
>
>I am trying to deploy samza on multi node yarn using HTTPFileSystem.
>I am running into classpath issues .
>Earlier few times it complained about no file system for Scheme HTTP.
>
>I am using samza 0.8 and yarn 2.5.2
>
>I have samza_yarn_2.10 jar in yarn_home/share/hadoop/hdfs/lib
>
>I also copied it to yarn_home/share/hadoop/yarn/lib
>
>Now I am getting Application application_1421283273623_0002 failed 2 times
>due to AM Container for appattempt_1421283273623_0002_000002 exited with
>exitCode: -1000 due to: java.lang.ClassNotFoundException: Class
>org.apache.samza.util.hadoop.HttpFileSystem not found
>.Failing this attempt.. Failing the application.
>
>
>Any Ideas ?
>
>Thanks,
>Snehal