Running Giraph application on Yarn

["Patel, Nishant - Time Inc India <Nishant.Patel@timeinc.com>" <Ni...@timeinc.com>]

Hi,

I was running giraph application on MR1 and it was working fine. I moved my application from MR1 Yarn and its not working. What changes I need to make for running Giraph jon on Yarn?

I am running my job on EMR. One solution I found is that I need to build giraph from source code with hadoop_yarn profile.

Currently I add Giraph as maven dependency and it work without any issue. Is there any easy way to handle this situation?

I am getting below error.


Exception in thread "main" java.lang.IllegalArgumentException: checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only one worker since only 1 task at a time


Thanks,
Nishant

Re: Running Giraph application on Yarn

["Patel, Nishant - Time Inc India <Nishant.Patel@timeinc.com>" <Ni...@timeinc.com>]

Hi Jason,

Thank you for your answer.

As of now I proceed by building Giraph with yarn profile. Next time when I launch EMR will try with your suggestion.

Thanks,
Nishant

From: Jason Plurad <pl...@gmail.com>>
Reply-To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Date: Friday, February 26, 2016 at 10:25 PM
To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Subject: Re: Running Giraph application on Yarn

Hi Nishant,

Try adding this either to your mapred-site.xml or your job properties:


# Let the YARN Resource Manager handle the MR2 job (any value other than local will do)
mapreduce.jobtracker.address=yarn

-- Jason


On Fri, Feb 19, 2016 at 5:39 AM, Patel, Nishant - Time Inc India <Ni...@timeinc.com>> <Ni...@timeinc.com>> wrote:
Hi,

I was running giraph application on MR1 and it was working fine. I moved my application from MR1 Yarn and its not working. What changes I need to make for running Giraph jon on Yarn?

I am running my job on EMR. One solution I found is that I need to build giraph from source code with hadoop_yarn profile.

Currently I add Giraph as maven dependency and it work without any issue. Is there any easy way to handle this situation?

I am getting below error.


Exception in thread "main" java.lang.IllegalArgumentException: checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only one worker since only 1 task at a time


Thanks,
Nishant



Please be alert for any emails that may ask you for login information or directs you to login via a link. If you believe this message is a phish or aren't sure whether this message is trustworthy, please send the original message as an attachment to 'phishing@timeinc.com<ma...@timeinc.com>'.

Re: Running Giraph application on Yarn

[Jason Plurad <pl...@gmail.com>]

Hi Nishant,

Try adding this either to your mapred-site.xml or your job properties:

# Let the YARN Resource Manager handle the MR2 job (any value other
than local will do)
mapreduce.jobtracker.address=yarn


-- Jason


On Fri, Feb 19, 2016 at 5:39 AM, Patel, Nishant - Time Inc India <
Nishant.Patel@timeinc.com> <Ni...@timeinc.com> wrote:

> Hi,
>
> I was running giraph application on MR1 and it was working fine. I moved
> my application from MR1 Yarn and its not working. What changes I need to
> make for running Giraph jon on Yarn?
>
> I am running my job on EMR. One solution I found is that I need to build
> giraph from source code with hadoop_yarn profile.
>
> Currently I add Giraph as maven dependency and it work without any issue.
> Is there any easy way to handle this situation?
>
> I am getting below error.
>
> Exception in thread "main" java.lang.IllegalArgumentException:
> checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only
> one worker since only 1 task at a time
>
>
> Thanks,
> Nishant
>