You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Henning Blohm <he...@zfabrik.de> on 2012/09/07 11:01:41 UTC
Can I use YARN with embedded job history server
Hi,
I am using CDH4.01 from a tarball and so far everything works kind of ok
(albeit quite different from CDH3). It seems though that I fail to get
an embedded job history server running when using YARN. On the other
hand, there is no $HADOOP_HOME/sbin/start-historyserver.sh or the like.
I can start the history server using mr-jobhistory-deamon.sh with
appropriate params.
In other words: Is there a way to have an embedded job history server
(small setup) when using YARN?
Thanks,
Henning
Re: Can I use YARN with embedded job history server
Posted by Henning Blohm <he...@zfabrik.de>.
Thanks for that fast reply!
That explains it.
Thanks,
Henning
On 09/07/2012 11:39 AM, Harsh J wrote:
> Hi Henning,
>
> On Fri, Sep 7, 2012 at 2:31 PM, Henning Blohm <he...@zfabrik.de> wrote:
>> Hi,
>>
>> I am using CDH4.01 from a tarball and so far everything works kind of ok
>> (albeit quite different from CDH3). It seems though that I fail to get an
>> embedded job history server running when using YARN. On the other hand,
>> there is no $HADOOP_HOME/sbin/start-historyserver.sh or the like. I can
>> start the history server using mr-jobhistory-deamon.sh with appropriate
>> params.
> Please use HADOOP_PREFIX instead of HADOOP_HOME. The latter's been deprecated.
>
> As you've found out, using the following is the correct way for
> starting the MR2 Job History Server when starting via a tarball:
>
> $ sbin/start-historyserver.sh start historyserver
>
>> In other words: Is there a way to have an embedded job history server (small
>> setup) when using YARN?
> By "embedded" do you mean a non-separate daemon for MR jobs? Or do you
> mean start-yarn.sh itself starting the
>
> YARN and MR2 (which uses YARN) are two separate set of components, and
> hence start-yarn.sh doesn't also start the MR-specific job history
> server. The JHS doesn't apply/work for any other apps of YARN other
> than MR2 right now, so its not unified.
>
>> Thanks,
>> Henning
>>
>>
>
>
Re: Can I use YARN with embedded job history server
Posted by Henning Blohm <he...@zfabrik.de>.
Thanks for that fast reply!
That explains it.
Thanks,
Henning
On 09/07/2012 11:39 AM, Harsh J wrote:
> Hi Henning,
>
> On Fri, Sep 7, 2012 at 2:31 PM, Henning Blohm <he...@zfabrik.de> wrote:
>> Hi,
>>
>> I am using CDH4.01 from a tarball and so far everything works kind of ok
>> (albeit quite different from CDH3). It seems though that I fail to get an
>> embedded job history server running when using YARN. On the other hand,
>> there is no $HADOOP_HOME/sbin/start-historyserver.sh or the like. I can
>> start the history server using mr-jobhistory-deamon.sh with appropriate
>> params.
> Please use HADOOP_PREFIX instead of HADOOP_HOME. The latter's been deprecated.
>
> As you've found out, using the following is the correct way for
> starting the MR2 Job History Server when starting via a tarball:
>
> $ sbin/start-historyserver.sh start historyserver
>
>> In other words: Is there a way to have an embedded job history server (small
>> setup) when using YARN?
> By "embedded" do you mean a non-separate daemon for MR jobs? Or do you
> mean start-yarn.sh itself starting the
>
> YARN and MR2 (which uses YARN) are two separate set of components, and
> hence start-yarn.sh doesn't also start the MR-specific job history
> server. The JHS doesn't apply/work for any other apps of YARN other
> than MR2 right now, so its not unified.
>
>> Thanks,
>> Henning
>>
>>
>
>
Re: Can I use YARN with embedded job history server
Posted by Henning Blohm <he...@zfabrik.de>.
Thanks for that fast reply!
That explains it.
Thanks,
Henning
On 09/07/2012 11:39 AM, Harsh J wrote:
> Hi Henning,
>
> On Fri, Sep 7, 2012 at 2:31 PM, Henning Blohm <he...@zfabrik.de> wrote:
>> Hi,
>>
>> I am using CDH4.01 from a tarball and so far everything works kind of ok
>> (albeit quite different from CDH3). It seems though that I fail to get an
>> embedded job history server running when using YARN. On the other hand,
>> there is no $HADOOP_HOME/sbin/start-historyserver.sh or the like. I can
>> start the history server using mr-jobhistory-deamon.sh with appropriate
>> params.
> Please use HADOOP_PREFIX instead of HADOOP_HOME. The latter's been deprecated.
>
> As you've found out, using the following is the correct way for
> starting the MR2 Job History Server when starting via a tarball:
>
> $ sbin/start-historyserver.sh start historyserver
>
>> In other words: Is there a way to have an embedded job history server (small
>> setup) when using YARN?
> By "embedded" do you mean a non-separate daemon for MR jobs? Or do you
> mean start-yarn.sh itself starting the
>
> YARN and MR2 (which uses YARN) are two separate set of components, and
> hence start-yarn.sh doesn't also start the MR-specific job history
> server. The JHS doesn't apply/work for any other apps of YARN other
> than MR2 right now, so its not unified.
>
>> Thanks,
>> Henning
>>
>>
>
>
Re: Can I use YARN with embedded job history server
Posted by Henning Blohm <he...@zfabrik.de>.
Thanks for that fast reply!
That explains it.
Thanks,
Henning
On 09/07/2012 11:39 AM, Harsh J wrote:
> Hi Henning,
>
> On Fri, Sep 7, 2012 at 2:31 PM, Henning Blohm <he...@zfabrik.de> wrote:
>> Hi,
>>
>> I am using CDH4.01 from a tarball and so far everything works kind of ok
>> (albeit quite different from CDH3). It seems though that I fail to get an
>> embedded job history server running when using YARN. On the other hand,
>> there is no $HADOOP_HOME/sbin/start-historyserver.sh or the like. I can
>> start the history server using mr-jobhistory-deamon.sh with appropriate
>> params.
> Please use HADOOP_PREFIX instead of HADOOP_HOME. The latter's been deprecated.
>
> As you've found out, using the following is the correct way for
> starting the MR2 Job History Server when starting via a tarball:
>
> $ sbin/start-historyserver.sh start historyserver
>
>> In other words: Is there a way to have an embedded job history server (small
>> setup) when using YARN?
> By "embedded" do you mean a non-separate daemon for MR jobs? Or do you
> mean start-yarn.sh itself starting the
>
> YARN and MR2 (which uses YARN) are two separate set of components, and
> hence start-yarn.sh doesn't also start the MR-specific job history
> server. The JHS doesn't apply/work for any other apps of YARN other
> than MR2 right now, so its not unified.
>
>> Thanks,
>> Henning
>>
>>
>
>
Re: Can I use YARN with embedded job history server
Posted by Harsh J <ha...@cloudera.com>.
Hi Henning,
On Fri, Sep 7, 2012 at 2:31 PM, Henning Blohm <he...@zfabrik.de> wrote:
> Hi,
>
> I am using CDH4.01 from a tarball and so far everything works kind of ok
> (albeit quite different from CDH3). It seems though that I fail to get an
> embedded job history server running when using YARN. On the other hand,
> there is no $HADOOP_HOME/sbin/start-historyserver.sh or the like. I can
> start the history server using mr-jobhistory-deamon.sh with appropriate
> params.
Please use HADOOP_PREFIX instead of HADOOP_HOME. The latter's been deprecated.
As you've found out, using the following is the correct way for
starting the MR2 Job History Server when starting via a tarball:
$ sbin/start-historyserver.sh start historyserver
> In other words: Is there a way to have an embedded job history server (small
> setup) when using YARN?
By "embedded" do you mean a non-separate daemon for MR jobs? Or do you
mean start-yarn.sh itself starting the
YARN and MR2 (which uses YARN) are two separate set of components, and
hence start-yarn.sh doesn't also start the MR-specific job history
server. The JHS doesn't apply/work for any other apps of YARN other
than MR2 right now, so its not unified.
> Thanks,
> Henning
>
>
--
Harsh J
Re: Can I use YARN with embedded job history server
Posted by Harsh J <ha...@cloudera.com>.
Hi Henning,
On Fri, Sep 7, 2012 at 2:31 PM, Henning Blohm <he...@zfabrik.de> wrote:
> Hi,
>
> I am using CDH4.01 from a tarball and so far everything works kind of ok
> (albeit quite different from CDH3). It seems though that I fail to get an
> embedded job history server running when using YARN. On the other hand,
> there is no $HADOOP_HOME/sbin/start-historyserver.sh or the like. I can
> start the history server using mr-jobhistory-deamon.sh with appropriate
> params.
Please use HADOOP_PREFIX instead of HADOOP_HOME. The latter's been deprecated.
As you've found out, using the following is the correct way for
starting the MR2 Job History Server when starting via a tarball:
$ sbin/start-historyserver.sh start historyserver
> In other words: Is there a way to have an embedded job history server (small
> setup) when using YARN?
By "embedded" do you mean a non-separate daemon for MR jobs? Or do you
mean start-yarn.sh itself starting the
YARN and MR2 (which uses YARN) are two separate set of components, and
hence start-yarn.sh doesn't also start the MR-specific job history
server. The JHS doesn't apply/work for any other apps of YARN other
than MR2 right now, so its not unified.
> Thanks,
> Henning
>
>
--
Harsh J
Re: Can I use YARN with embedded job history server
Posted by Harsh J <ha...@cloudera.com>.
Hi Henning,
On Fri, Sep 7, 2012 at 2:31 PM, Henning Blohm <he...@zfabrik.de> wrote:
> Hi,
>
> I am using CDH4.01 from a tarball and so far everything works kind of ok
> (albeit quite different from CDH3). It seems though that I fail to get an
> embedded job history server running when using YARN. On the other hand,
> there is no $HADOOP_HOME/sbin/start-historyserver.sh or the like. I can
> start the history server using mr-jobhistory-deamon.sh with appropriate
> params.
Please use HADOOP_PREFIX instead of HADOOP_HOME. The latter's been deprecated.
As you've found out, using the following is the correct way for
starting the MR2 Job History Server when starting via a tarball:
$ sbin/start-historyserver.sh start historyserver
> In other words: Is there a way to have an embedded job history server (small
> setup) when using YARN?
By "embedded" do you mean a non-separate daemon for MR jobs? Or do you
mean start-yarn.sh itself starting the
YARN and MR2 (which uses YARN) are two separate set of components, and
hence start-yarn.sh doesn't also start the MR-specific job history
server. The JHS doesn't apply/work for any other apps of YARN other
than MR2 right now, so its not unified.
> Thanks,
> Henning
>
>
--
Harsh J
Re: Can I use YARN with embedded job history server
Posted by Harsh J <ha...@cloudera.com>.
Hi Henning,
On Fri, Sep 7, 2012 at 2:31 PM, Henning Blohm <he...@zfabrik.de> wrote:
> Hi,
>
> I am using CDH4.01 from a tarball and so far everything works kind of ok
> (albeit quite different from CDH3). It seems though that I fail to get an
> embedded job history server running when using YARN. On the other hand,
> there is no $HADOOP_HOME/sbin/start-historyserver.sh or the like. I can
> start the history server using mr-jobhistory-deamon.sh with appropriate
> params.
Please use HADOOP_PREFIX instead of HADOOP_HOME. The latter's been deprecated.
As you've found out, using the following is the correct way for
starting the MR2 Job History Server when starting via a tarball:
$ sbin/start-historyserver.sh start historyserver
> In other words: Is there a way to have an embedded job history server (small
> setup) when using YARN?
By "embedded" do you mean a non-separate daemon for MR jobs? Or do you
mean start-yarn.sh itself starting the
YARN and MR2 (which uses YARN) are two separate set of components, and
hence start-yarn.sh doesn't also start the MR-specific job history
server. The JHS doesn't apply/work for any other apps of YARN other
than MR2 right now, so its not unified.
> Thanks,
> Henning
>
>
--
Harsh J