You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by hadoop hive <ha...@gmail.com> on 2015/10/11 20:36:50 UTC
Hadoop Development
Hi Experts,
Did anyone have idea how to execute MR on remote cluster. Actually i have
been using windows for hadoop development and i have pseudo distributed
cluster on linux.
Did anyone have any doc or wrote any blog on this. Please provide some
pointers.
Thanks
Vikas Srivastava
Re: Hadoop Development
Posted by hadoop hive <ha...@gmail.com>.
Hey Ted,
I am using hadoop 2.6 on ubuntu, its good but i was looking for some java
code executions. Do you have anything or every tried that way .
Thanks
On Mon, Oct 12, 2015 at 12:39 AM, Ted Yu <yu...@gmail.com> wrote:
> See ResourceManager REST API to submit new applications :
>
>
> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_APISubmit_Application
>
> What's the version of hadoop on Linux ?
>
> Cheers
>
> On Sun, Oct 11, 2015 at 11:36 AM, hadoop hive <ha...@gmail.com>
> wrote:
>
>> Hi Experts,
>>
>> Did anyone have idea how to execute MR on remote cluster. Actually i have
>> been using windows for hadoop development and i have pseudo distributed
>> cluster on linux.
>>
>> Did anyone have any doc or wrote any blog on this. Please provide some
>> pointers.
>>
>> Thanks
>> Vikas Srivastava
>>
>>
>>
>
Re: Hadoop Development
Posted by hadoop hive <ha...@gmail.com>.
Hey Ted,
I am using hadoop 2.6 on ubuntu, its good but i was looking for some java
code executions. Do you have anything or every tried that way .
Thanks
On Mon, Oct 12, 2015 at 12:39 AM, Ted Yu <yu...@gmail.com> wrote:
> See ResourceManager REST API to submit new applications :
>
>
> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_APISubmit_Application
>
> What's the version of hadoop on Linux ?
>
> Cheers
>
> On Sun, Oct 11, 2015 at 11:36 AM, hadoop hive <ha...@gmail.com>
> wrote:
>
>> Hi Experts,
>>
>> Did anyone have idea how to execute MR on remote cluster. Actually i have
>> been using windows for hadoop development and i have pseudo distributed
>> cluster on linux.
>>
>> Did anyone have any doc or wrote any blog on this. Please provide some
>> pointers.
>>
>> Thanks
>> Vikas Srivastava
>>
>>
>>
>
Re: Hadoop Development
Posted by hadoop hive <ha...@gmail.com>.
Hey Ted,
I am using hadoop 2.6 on ubuntu, its good but i was looking for some java
code executions. Do you have anything or every tried that way .
Thanks
On Mon, Oct 12, 2015 at 12:39 AM, Ted Yu <yu...@gmail.com> wrote:
> See ResourceManager REST API to submit new applications :
>
>
> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_APISubmit_Application
>
> What's the version of hadoop on Linux ?
>
> Cheers
>
> On Sun, Oct 11, 2015 at 11:36 AM, hadoop hive <ha...@gmail.com>
> wrote:
>
>> Hi Experts,
>>
>> Did anyone have idea how to execute MR on remote cluster. Actually i have
>> been using windows for hadoop development and i have pseudo distributed
>> cluster on linux.
>>
>> Did anyone have any doc or wrote any blog on this. Please provide some
>> pointers.
>>
>> Thanks
>> Vikas Srivastava
>>
>>
>>
>
Re: Hadoop Development
Posted by hadoop hive <ha...@gmail.com>.
Hey Ted,
I am using hadoop 2.6 on ubuntu, its good but i was looking for some java
code executions. Do you have anything or every tried that way .
Thanks
On Mon, Oct 12, 2015 at 12:39 AM, Ted Yu <yu...@gmail.com> wrote:
> See ResourceManager REST API to submit new applications :
>
>
> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_APISubmit_Application
>
> What's the version of hadoop on Linux ?
>
> Cheers
>
> On Sun, Oct 11, 2015 at 11:36 AM, hadoop hive <ha...@gmail.com>
> wrote:
>
>> Hi Experts,
>>
>> Did anyone have idea how to execute MR on remote cluster. Actually i have
>> been using windows for hadoop development and i have pseudo distributed
>> cluster on linux.
>>
>> Did anyone have any doc or wrote any blog on this. Please provide some
>> pointers.
>>
>> Thanks
>> Vikas Srivastava
>>
>>
>>
>
Re: Hadoop Development
Posted by Ted Yu <yu...@gmail.com>.
See ResourceManager REST API to submit new applications :
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_APISubmit_Application
What's the version of hadoop on Linux ?
Cheers
On Sun, Oct 11, 2015 at 11:36 AM, hadoop hive <ha...@gmail.com> wrote:
> Hi Experts,
>
> Did anyone have idea how to execute MR on remote cluster. Actually i have
> been using windows for hadoop development and i have pseudo distributed
> cluster on linux.
>
> Did anyone have any doc or wrote any blog on this. Please provide some
> pointers.
>
> Thanks
> Vikas Srivastava
>
>
>
Re: Hadoop Development
Posted by Ted Yu <yu...@gmail.com>.
See ResourceManager REST API to submit new applications :
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_APISubmit_Application
What's the version of hadoop on Linux ?
Cheers
On Sun, Oct 11, 2015 at 11:36 AM, hadoop hive <ha...@gmail.com> wrote:
> Hi Experts,
>
> Did anyone have idea how to execute MR on remote cluster. Actually i have
> been using windows for hadoop development and i have pseudo distributed
> cluster on linux.
>
> Did anyone have any doc or wrote any blog on this. Please provide some
> pointers.
>
> Thanks
> Vikas Srivastava
>
>
>
Re: Hadoop Development
Posted by hadoop hive <ha...@gmail.com>.
Thanks Daniel.
On Mon, Oct 12, 2015 at 12:52 AM, Daniel Schulz <
danielschulz2005@hotmail.com> wrote:
> Hi Vikas,
>
> Thanks for reaching out to us.
>
> For local development to me MRUnit is a game changer: you may speed up
> your development significantly due to better debugging features.
> mrunit.apache.org
>
> To upload your final jar and start it from a remote host, like Windows,
> just use web services REST API of YARN. You may use putty or cmd for that.
>
> hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html
>
> Please let us know iff you have further questions about Hadoop.
>
> Kind regards, Daniel.
>
> ------------------------------
> Date: Mon, 12 Oct 2015 00:06:50 +0530
> Subject: Hadoop Development
> From: hadoophive@gmail.com
> To: user@hadoop.apache.org
>
>
> Hi Experts,
>
> Did anyone have idea how to execute MR on remote cluster. Actually i have
> been using windows for hadoop development and i have pseudo distributed
> cluster on linux.
>
> Did anyone have any doc or wrote any blog on this. Please provide some
> pointers.
>
> Thanks
> Vikas Srivastava
>
>
>
Re: Hadoop Development
Posted by hadoop hive <ha...@gmail.com>.
Thanks Daniel.
On Mon, Oct 12, 2015 at 12:52 AM, Daniel Schulz <
danielschulz2005@hotmail.com> wrote:
> Hi Vikas,
>
> Thanks for reaching out to us.
>
> For local development to me MRUnit is a game changer: you may speed up
> your development significantly due to better debugging features.
> mrunit.apache.org
>
> To upload your final jar and start it from a remote host, like Windows,
> just use web services REST API of YARN. You may use putty or cmd for that.
>
> hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html
>
> Please let us know iff you have further questions about Hadoop.
>
> Kind regards, Daniel.
>
> ------------------------------
> Date: Mon, 12 Oct 2015 00:06:50 +0530
> Subject: Hadoop Development
> From: hadoophive@gmail.com
> To: user@hadoop.apache.org
>
>
> Hi Experts,
>
> Did anyone have idea how to execute MR on remote cluster. Actually i have
> been using windows for hadoop development and i have pseudo distributed
> cluster on linux.
>
> Did anyone have any doc or wrote any blog on this. Please provide some
> pointers.
>
> Thanks
> Vikas Srivastava
>
>
>
Re: Hadoop Development
Posted by hadoop hive <ha...@gmail.com>.
Thanks Daniel.
On Mon, Oct 12, 2015 at 12:52 AM, Daniel Schulz <
danielschulz2005@hotmail.com> wrote:
> Hi Vikas,
>
> Thanks for reaching out to us.
>
> For local development to me MRUnit is a game changer: you may speed up
> your development significantly due to better debugging features.
> mrunit.apache.org
>
> To upload your final jar and start it from a remote host, like Windows,
> just use web services REST API of YARN. You may use putty or cmd for that.
>
> hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html
>
> Please let us know iff you have further questions about Hadoop.
>
> Kind regards, Daniel.
>
> ------------------------------
> Date: Mon, 12 Oct 2015 00:06:50 +0530
> Subject: Hadoop Development
> From: hadoophive@gmail.com
> To: user@hadoop.apache.org
>
>
> Hi Experts,
>
> Did anyone have idea how to execute MR on remote cluster. Actually i have
> been using windows for hadoop development and i have pseudo distributed
> cluster on linux.
>
> Did anyone have any doc or wrote any blog on this. Please provide some
> pointers.
>
> Thanks
> Vikas Srivastava
>
>
>
Re: Hadoop Development
Posted by hadoop hive <ha...@gmail.com>.
Thanks Daniel.
On Mon, Oct 12, 2015 at 12:52 AM, Daniel Schulz <
danielschulz2005@hotmail.com> wrote:
> Hi Vikas,
>
> Thanks for reaching out to us.
>
> For local development to me MRUnit is a game changer: you may speed up
> your development significantly due to better debugging features.
> mrunit.apache.org
>
> To upload your final jar and start it from a remote host, like Windows,
> just use web services REST API of YARN. You may use putty or cmd for that.
>
> hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html
>
> Please let us know iff you have further questions about Hadoop.
>
> Kind regards, Daniel.
>
> ------------------------------
> Date: Mon, 12 Oct 2015 00:06:50 +0530
> Subject: Hadoop Development
> From: hadoophive@gmail.com
> To: user@hadoop.apache.org
>
>
> Hi Experts,
>
> Did anyone have idea how to execute MR on remote cluster. Actually i have
> been using windows for hadoop development and i have pseudo distributed
> cluster on linux.
>
> Did anyone have any doc or wrote any blog on this. Please provide some
> pointers.
>
> Thanks
> Vikas Srivastava
>
>
>
RE: Hadoop Development
Posted by Daniel Schulz <da...@hotmail.com>.
Hi Vikas,
Thanks for reaching out to us.
For local development to me MRUnit is a game changer: you may speed up your development significantly due to better debugging features.mrunit.apache.org
To upload your final jar and start it from a remote host, like Windows, just use web services REST API of YARN. You may use putty or cmd for that.hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html
Please let us know iff you have further questions about Hadoop.
Kind regards, Daniel.
Date: Mon, 12 Oct 2015 00:06:50 +0530
Subject: Hadoop Development
From: hadoophive@gmail.com
To: user@hadoop.apache.org
Hi Experts,
Did anyone have idea how to execute MR on remote cluster. Actually i have been using windows for hadoop development and i have pseudo distributed cluster on linux.
Did anyone have any doc or wrote any blog on this. Please provide some pointers.
Thanks
Vikas Srivastava
Re: Hadoop Development
Posted by Ted Yu <yu...@gmail.com>.
See ResourceManager REST API to submit new applications :
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_APISubmit_Application
What's the version of hadoop on Linux ?
Cheers
On Sun, Oct 11, 2015 at 11:36 AM, hadoop hive <ha...@gmail.com> wrote:
> Hi Experts,
>
> Did anyone have idea how to execute MR on remote cluster. Actually i have
> been using windows for hadoop development and i have pseudo distributed
> cluster on linux.
>
> Did anyone have any doc or wrote any blog on this. Please provide some
> pointers.
>
> Thanks
> Vikas Srivastava
>
>
>
RE: Hadoop Development
Posted by Daniel Schulz <da...@hotmail.com>.
Hi Vikas,
Thanks for reaching out to us.
For local development to me MRUnit is a game changer: you may speed up your development significantly due to better debugging features.mrunit.apache.org
To upload your final jar and start it from a remote host, like Windows, just use web services REST API of YARN. You may use putty or cmd for that.hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html
Please let us know iff you have further questions about Hadoop.
Kind regards, Daniel.
Date: Mon, 12 Oct 2015 00:06:50 +0530
Subject: Hadoop Development
From: hadoophive@gmail.com
To: user@hadoop.apache.org
Hi Experts,
Did anyone have idea how to execute MR on remote cluster. Actually i have been using windows for hadoop development and i have pseudo distributed cluster on linux.
Did anyone have any doc or wrote any blog on this. Please provide some pointers.
Thanks
Vikas Srivastava
Re: Hadoop Development
Posted by Ted Yu <yu...@gmail.com>.
See ResourceManager REST API to submit new applications :
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_APISubmit_Application
What's the version of hadoop on Linux ?
Cheers
On Sun, Oct 11, 2015 at 11:36 AM, hadoop hive <ha...@gmail.com> wrote:
> Hi Experts,
>
> Did anyone have idea how to execute MR on remote cluster. Actually i have
> been using windows for hadoop development and i have pseudo distributed
> cluster on linux.
>
> Did anyone have any doc or wrote any blog on this. Please provide some
> pointers.
>
> Thanks
> Vikas Srivastava
>
>
>
RE: Hadoop Development
Posted by Daniel Schulz <da...@hotmail.com>.
Hi Vikas,
Thanks for reaching out to us.
For local development to me MRUnit is a game changer: you may speed up your development significantly due to better debugging features.mrunit.apache.org
To upload your final jar and start it from a remote host, like Windows, just use web services REST API of YARN. You may use putty or cmd for that.hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html
Please let us know iff you have further questions about Hadoop.
Kind regards, Daniel.
Date: Mon, 12 Oct 2015 00:06:50 +0530
Subject: Hadoop Development
From: hadoophive@gmail.com
To: user@hadoop.apache.org
Hi Experts,
Did anyone have idea how to execute MR on remote cluster. Actually i have been using windows for hadoop development and i have pseudo distributed cluster on linux.
Did anyone have any doc or wrote any blog on this. Please provide some pointers.
Thanks
Vikas Srivastava
RE: Hadoop Development
Posted by Daniel Schulz <da...@hotmail.com>.
Hi Vikas,
Thanks for reaching out to us.
For local development to me MRUnit is a game changer: you may speed up your development significantly due to better debugging features.mrunit.apache.org
To upload your final jar and start it from a remote host, like Windows, just use web services REST API of YARN. You may use putty or cmd for that.hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html
Please let us know iff you have further questions about Hadoop.
Kind regards, Daniel.
Date: Mon, 12 Oct 2015 00:06:50 +0530
Subject: Hadoop Development
From: hadoophive@gmail.com
To: user@hadoop.apache.org
Hi Experts,
Did anyone have idea how to execute MR on remote cluster. Actually i have been using windows for hadoop development and i have pseudo distributed cluster on linux.
Did anyone have any doc or wrote any blog on this. Please provide some pointers.
Thanks
Vikas Srivastava