You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Amit Sela <am...@infolinks.com> on 2013/04/15 12:06:40 UTC
Submitting mapreduce and nothing happens
Hi all,
I'm trying to submit a mapreduce job remotely using job.submit()
I get the following:
[WARN ] org.apache.hadoop.mapred.JobClient » Use GenericOptionsParser
for parsing the arguments. Applications should implement Tool for the same.
[INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
area hdfs://{namenode
address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
and nothing happens...
I set the the mapred.job.tracker and changed permissions
for hadoop.tmp.dir. I also set "hadoop.job.ugi" as "hadoop,supergroup" but
some how I think that it's not making any difference.
The system submitting the job is running with another user, call it:
myusername and not hadoop.
I believe it is related to the user permissions but I can't seem to get it
right.
Thanks for the help,
Amit.
Re: Submitting mapreduce and nothing happens
Posted by Azuryy Yu <az...@gmail.com>.
do you have data exists on your input path?
On Wed, Apr 17, 2013 at 1:18 AM, Amit Sela <am...@infolinks.com> wrote:
> Nothing on JT log, but as I mentioned I see this in the client log:
>
> [WARN ] org.apache.hadoop.mapred.JobClient » Use
> GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
> area
> hdfs://hadoop-name-node.address:8000{hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
>
> And this on the NN log:
>
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
> transactions: 1 Total time for transactions(ms): 0Number of transactions
> batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 0
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
> clientusername ip=/xx.xx.xx.xxx cmd=mkdirs src=
> {hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034 dst=null
> perm=clientusername:supergroup:rwxr-xr-x
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
> clientusername ip=/xx.xx.xx.xxx cmd=setPermission src={hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
> dst=null perm=clientusername:supergroup:rwx------
>
> Thanks.
>
>
> On Tue, Apr 16, 2013 at 10:30 AM, Bejoy Ks <be...@gmail.com> wrote:
>
>> Hi Amit
>>
>> Are you seeing any errors or warnings on JT logs?
>>
>> Regards
>> Bejoy KS
>>
>
>
Re: Submitting mapreduce and nothing happens
Posted by Azuryy Yu <az...@gmail.com>.
do you have data exists on your input path?
On Wed, Apr 17, 2013 at 1:18 AM, Amit Sela <am...@infolinks.com> wrote:
> Nothing on JT log, but as I mentioned I see this in the client log:
>
> [WARN ] org.apache.hadoop.mapred.JobClient » Use
> GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
> area
> hdfs://hadoop-name-node.address:8000{hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
>
> And this on the NN log:
>
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
> transactions: 1 Total time for transactions(ms): 0Number of transactions
> batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 0
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
> clientusername ip=/xx.xx.xx.xxx cmd=mkdirs src=
> {hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034 dst=null
> perm=clientusername:supergroup:rwxr-xr-x
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
> clientusername ip=/xx.xx.xx.xxx cmd=setPermission src={hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
> dst=null perm=clientusername:supergroup:rwx------
>
> Thanks.
>
>
> On Tue, Apr 16, 2013 at 10:30 AM, Bejoy Ks <be...@gmail.com> wrote:
>
>> Hi Amit
>>
>> Are you seeing any errors or warnings on JT logs?
>>
>> Regards
>> Bejoy KS
>>
>
>
Re: Submitting mapreduce and nothing happens
Posted by Azuryy Yu <az...@gmail.com>.
do you have data exists on your input path?
On Wed, Apr 17, 2013 at 1:18 AM, Amit Sela <am...@infolinks.com> wrote:
> Nothing on JT log, but as I mentioned I see this in the client log:
>
> [WARN ] org.apache.hadoop.mapred.JobClient » Use
> GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
> area
> hdfs://hadoop-name-node.address:8000{hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
>
> And this on the NN log:
>
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
> transactions: 1 Total time for transactions(ms): 0Number of transactions
> batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 0
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
> clientusername ip=/xx.xx.xx.xxx cmd=mkdirs src=
> {hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034 dst=null
> perm=clientusername:supergroup:rwxr-xr-x
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
> clientusername ip=/xx.xx.xx.xxx cmd=setPermission src={hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
> dst=null perm=clientusername:supergroup:rwx------
>
> Thanks.
>
>
> On Tue, Apr 16, 2013 at 10:30 AM, Bejoy Ks <be...@gmail.com> wrote:
>
>> Hi Amit
>>
>> Are you seeing any errors or warnings on JT logs?
>>
>> Regards
>> Bejoy KS
>>
>
>
Re: Submitting mapreduce and nothing happens
Posted by Azuryy Yu <az...@gmail.com>.
do you have data exists on your input path?
On Wed, Apr 17, 2013 at 1:18 AM, Amit Sela <am...@infolinks.com> wrote:
> Nothing on JT log, but as I mentioned I see this in the client log:
>
> [WARN ] org.apache.hadoop.mapred.JobClient » Use
> GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
> area
> hdfs://hadoop-name-node.address:8000{hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
>
> And this on the NN log:
>
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
> transactions: 1 Total time for transactions(ms): 0Number of transactions
> batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 0
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
> clientusername ip=/xx.xx.xx.xxx cmd=mkdirs src=
> {hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034 dst=null
> perm=clientusername:supergroup:rwxr-xr-x
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
> clientusername ip=/xx.xx.xx.xxx cmd=setPermission src={hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
> dst=null perm=clientusername:supergroup:rwx------
>
> Thanks.
>
>
> On Tue, Apr 16, 2013 at 10:30 AM, Bejoy Ks <be...@gmail.com> wrote:
>
>> Hi Amit
>>
>> Are you seeing any errors or warnings on JT logs?
>>
>> Regards
>> Bejoy KS
>>
>
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
Nothing on JT log, but as I mentioned I see this in the client log:
[WARN ] org.apache.hadoop.mapred.JobClient » Use GenericOptionsParser
for parsing the arguments. Applications should implement Tool for the same.
[INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
area
hdfs://hadoop-name-node.address:8000{hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
And this on the NN log:
INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
transactions: 1 Total time for transactions(ms): 0Number of transactions
batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 0
INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
clientusername ip=/xx.xx.xx.xxx cmd=mkdirs src=
{hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
dst=null
perm=clientusername:supergroup:rwxr-xr-x
INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
clientusername ip=/xx.xx.xx.xxx cmd=setPermission
src={hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
dst=null perm=clientusername:supergroup:rwx------
Thanks.
On Tue, Apr 16, 2013 at 10:30 AM, Bejoy Ks <be...@gmail.com> wrote:
> Hi Amit
>
> Are you seeing any errors or warnings on JT logs?
>
> Regards
> Bejoy KS
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
Nothing on JT log, but as I mentioned I see this in the client log:
[WARN ] org.apache.hadoop.mapred.JobClient » Use GenericOptionsParser
for parsing the arguments. Applications should implement Tool for the same.
[INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
area
hdfs://hadoop-name-node.address:8000{hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
And this on the NN log:
INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
transactions: 1 Total time for transactions(ms): 0Number of transactions
batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 0
INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
clientusername ip=/xx.xx.xx.xxx cmd=mkdirs src=
{hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
dst=null
perm=clientusername:supergroup:rwxr-xr-x
INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
clientusername ip=/xx.xx.xx.xxx cmd=setPermission
src={hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
dst=null perm=clientusername:supergroup:rwx------
Thanks.
On Tue, Apr 16, 2013 at 10:30 AM, Bejoy Ks <be...@gmail.com> wrote:
> Hi Amit
>
> Are you seeing any errors or warnings on JT logs?
>
> Regards
> Bejoy KS
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
Nothing on JT log, but as I mentioned I see this in the client log:
[WARN ] org.apache.hadoop.mapred.JobClient » Use GenericOptionsParser
for parsing the arguments. Applications should implement Tool for the same.
[INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
area
hdfs://hadoop-name-node.address:8000{hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
And this on the NN log:
INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
transactions: 1 Total time for transactions(ms): 0Number of transactions
batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 0
INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
clientusername ip=/xx.xx.xx.xxx cmd=mkdirs src=
{hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
dst=null
perm=clientusername:supergroup:rwxr-xr-x
INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
clientusername ip=/xx.xx.xx.xxx cmd=setPermission
src={hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
dst=null perm=clientusername:supergroup:rwx------
Thanks.
On Tue, Apr 16, 2013 at 10:30 AM, Bejoy Ks <be...@gmail.com> wrote:
> Hi Amit
>
> Are you seeing any errors or warnings on JT logs?
>
> Regards
> Bejoy KS
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
Nothing on JT log, but as I mentioned I see this in the client log:
[WARN ] org.apache.hadoop.mapred.JobClient » Use GenericOptionsParser
for parsing the arguments. Applications should implement Tool for the same.
[INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
area
hdfs://hadoop-name-node.address:8000{hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
And this on the NN log:
INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
transactions: 1 Total time for transactions(ms): 0Number of transactions
batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 0
INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
clientusername ip=/xx.xx.xx.xxx cmd=mkdirs src=
{hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
dst=null
perm=clientusername:supergroup:rwxr-xr-x
INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=
clientusername ip=/xx.xx.xx.xxx cmd=setPermission
src={hadoop.tmp.dir}/mapred/staging/{clientusername}/.staging/job_201304150711_0034
dst=null perm=clientusername:supergroup:rwx------
Thanks.
On Tue, Apr 16, 2013 at 10:30 AM, Bejoy Ks <be...@gmail.com> wrote:
> Hi Amit
>
> Are you seeing any errors or warnings on JT logs?
>
> Regards
> Bejoy KS
>
Re: Submitting mapreduce and nothing happens
Posted by Bejoy Ks <be...@gmail.com>.
Hi Amit
Are you seeing any errors or warnings on JT logs?
Regards
Bejoy KS
Re: Submitting mapreduce and nothing happens
Posted by Bejoy Ks <be...@gmail.com>.
Hi Amit
Are you seeing any errors or warnings on JT logs?
Regards
Bejoy KS
Re: Submitting mapreduce and nothing happens
Posted by Bejoy Ks <be...@gmail.com>.
Hi Amit
Are you seeing any errors or warnings on JT logs?
Regards
Bejoy KS
Re: Submitting mapreduce and nothing happens
Posted by Bejoy Ks <be...@gmail.com>.
Hi Amit
Are you seeing any errors or warnings on JT logs?
Regards
Bejoy KS
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
Reading my own message I understand that maybe it's not clear so just to
clarify - the previously mentioned JT ID is indeed the correct ID.
Thanks.
On Apr 15, 2013 4:35 PM, "Amit Sela" <am...@infolinks.com> wrote:
> This is the JT ID and there is no problem running jobs from command line,
> just remote.
> On Apr 15, 2013 4:24 PM, "Harsh J" <ha...@cloudera.com> wrote:
>
>> Thats interesting; is the JT you're running on the cluster started
>> with the ID 201304150711 or something else?
>>
>> On Mon, Apr 15, 2013 at 6:47 PM, Amit Sela <am...@infolinks.com> wrote:
>> > The client prints the two lines I posted and the cluster shows nothing.
>> Not
>> > even incrementing the number of submitted jobs.
>> >
>> > On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
>> >>
>> >> When you say "nothing happens"; where exactly do you mean? The client
>> >> doesn't print anything, or the cluster doesn't run anything?
>> >>
>> >> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com>
>> wrote:
>> >> > Hi all,
>> >> >
>> >> > I'm trying to submit a mapreduce job remotely using job.submit()
>> >> >
>> >> > I get the following:
>> >> >
>> >> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
>> >> > GenericOptionsParser
>> >> > for parsing the arguments. Applications should implement Tool for the
>> >> > same.
>> >> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
>> >> > staging
>> >> > area hdfs://{namenode
>> >> >
>> >> >
>> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>> >> >
>> >> > and nothing happens...
>> >> >
>> >> > I set the the mapred.job.tracker and changed permissions for
>> >> > hadoop.tmp.dir.
>> >> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I
>> think
>> >> > that
>> >> > it's not making any difference.
>> >> > The system submitting the job is running with another user, call it:
>> >> > myusername and not hadoop.
>> >> >
>> >> > I believe it is related to the user permissions but I can't seem to
>> get
>> >> > it
>> >> > right.
>> >> >
>> >> > Thanks for the help,
>> >> >
>> >> > Amit.
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Harsh J
>>
>>
>>
>> --
>> Harsh J
>>
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
Reading my own message I understand that maybe it's not clear so just to
clarify - the previously mentioned JT ID is indeed the correct ID.
Thanks.
On Apr 15, 2013 4:35 PM, "Amit Sela" <am...@infolinks.com> wrote:
> This is the JT ID and there is no problem running jobs from command line,
> just remote.
> On Apr 15, 2013 4:24 PM, "Harsh J" <ha...@cloudera.com> wrote:
>
>> Thats interesting; is the JT you're running on the cluster started
>> with the ID 201304150711 or something else?
>>
>> On Mon, Apr 15, 2013 at 6:47 PM, Amit Sela <am...@infolinks.com> wrote:
>> > The client prints the two lines I posted and the cluster shows nothing.
>> Not
>> > even incrementing the number of submitted jobs.
>> >
>> > On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
>> >>
>> >> When you say "nothing happens"; where exactly do you mean? The client
>> >> doesn't print anything, or the cluster doesn't run anything?
>> >>
>> >> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com>
>> wrote:
>> >> > Hi all,
>> >> >
>> >> > I'm trying to submit a mapreduce job remotely using job.submit()
>> >> >
>> >> > I get the following:
>> >> >
>> >> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
>> >> > GenericOptionsParser
>> >> > for parsing the arguments. Applications should implement Tool for the
>> >> > same.
>> >> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
>> >> > staging
>> >> > area hdfs://{namenode
>> >> >
>> >> >
>> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>> >> >
>> >> > and nothing happens...
>> >> >
>> >> > I set the the mapred.job.tracker and changed permissions for
>> >> > hadoop.tmp.dir.
>> >> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I
>> think
>> >> > that
>> >> > it's not making any difference.
>> >> > The system submitting the job is running with another user, call it:
>> >> > myusername and not hadoop.
>> >> >
>> >> > I believe it is related to the user permissions but I can't seem to
>> get
>> >> > it
>> >> > right.
>> >> >
>> >> > Thanks for the help,
>> >> >
>> >> > Amit.
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Harsh J
>>
>>
>>
>> --
>> Harsh J
>>
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
Reading my own message I understand that maybe it's not clear so just to
clarify - the previously mentioned JT ID is indeed the correct ID.
Thanks.
On Apr 15, 2013 4:35 PM, "Amit Sela" <am...@infolinks.com> wrote:
> This is the JT ID and there is no problem running jobs from command line,
> just remote.
> On Apr 15, 2013 4:24 PM, "Harsh J" <ha...@cloudera.com> wrote:
>
>> Thats interesting; is the JT you're running on the cluster started
>> with the ID 201304150711 or something else?
>>
>> On Mon, Apr 15, 2013 at 6:47 PM, Amit Sela <am...@infolinks.com> wrote:
>> > The client prints the two lines I posted and the cluster shows nothing.
>> Not
>> > even incrementing the number of submitted jobs.
>> >
>> > On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
>> >>
>> >> When you say "nothing happens"; where exactly do you mean? The client
>> >> doesn't print anything, or the cluster doesn't run anything?
>> >>
>> >> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com>
>> wrote:
>> >> > Hi all,
>> >> >
>> >> > I'm trying to submit a mapreduce job remotely using job.submit()
>> >> >
>> >> > I get the following:
>> >> >
>> >> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
>> >> > GenericOptionsParser
>> >> > for parsing the arguments. Applications should implement Tool for the
>> >> > same.
>> >> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
>> >> > staging
>> >> > area hdfs://{namenode
>> >> >
>> >> >
>> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>> >> >
>> >> > and nothing happens...
>> >> >
>> >> > I set the the mapred.job.tracker and changed permissions for
>> >> > hadoop.tmp.dir.
>> >> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I
>> think
>> >> > that
>> >> > it's not making any difference.
>> >> > The system submitting the job is running with another user, call it:
>> >> > myusername and not hadoop.
>> >> >
>> >> > I believe it is related to the user permissions but I can't seem to
>> get
>> >> > it
>> >> > right.
>> >> >
>> >> > Thanks for the help,
>> >> >
>> >> > Amit.
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Harsh J
>>
>>
>>
>> --
>> Harsh J
>>
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
Reading my own message I understand that maybe it's not clear so just to
clarify - the previously mentioned JT ID is indeed the correct ID.
Thanks.
On Apr 15, 2013 4:35 PM, "Amit Sela" <am...@infolinks.com> wrote:
> This is the JT ID and there is no problem running jobs from command line,
> just remote.
> On Apr 15, 2013 4:24 PM, "Harsh J" <ha...@cloudera.com> wrote:
>
>> Thats interesting; is the JT you're running on the cluster started
>> with the ID 201304150711 or something else?
>>
>> On Mon, Apr 15, 2013 at 6:47 PM, Amit Sela <am...@infolinks.com> wrote:
>> > The client prints the two lines I posted and the cluster shows nothing.
>> Not
>> > even incrementing the number of submitted jobs.
>> >
>> > On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
>> >>
>> >> When you say "nothing happens"; where exactly do you mean? The client
>> >> doesn't print anything, or the cluster doesn't run anything?
>> >>
>> >> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com>
>> wrote:
>> >> > Hi all,
>> >> >
>> >> > I'm trying to submit a mapreduce job remotely using job.submit()
>> >> >
>> >> > I get the following:
>> >> >
>> >> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
>> >> > GenericOptionsParser
>> >> > for parsing the arguments. Applications should implement Tool for the
>> >> > same.
>> >> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
>> >> > staging
>> >> > area hdfs://{namenode
>> >> >
>> >> >
>> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>> >> >
>> >> > and nothing happens...
>> >> >
>> >> > I set the the mapred.job.tracker and changed permissions for
>> >> > hadoop.tmp.dir.
>> >> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I
>> think
>> >> > that
>> >> > it's not making any difference.
>> >> > The system submitting the job is running with another user, call it:
>> >> > myusername and not hadoop.
>> >> >
>> >> > I believe it is related to the user permissions but I can't seem to
>> get
>> >> > it
>> >> > right.
>> >> >
>> >> > Thanks for the help,
>> >> >
>> >> > Amit.
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Harsh J
>>
>>
>>
>> --
>> Harsh J
>>
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
This is the JT ID and there is no problem running jobs from command line,
just remote.
On Apr 15, 2013 4:24 PM, "Harsh J" <ha...@cloudera.com> wrote:
> Thats interesting; is the JT you're running on the cluster started
> with the ID 201304150711 or something else?
>
> On Mon, Apr 15, 2013 at 6:47 PM, Amit Sela <am...@infolinks.com> wrote:
> > The client prints the two lines I posted and the cluster shows nothing.
> Not
> > even incrementing the number of submitted jobs.
> >
> > On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
> >>
> >> When you say "nothing happens"; where exactly do you mean? The client
> >> doesn't print anything, or the cluster doesn't run anything?
> >>
> >> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
> >> > Hi all,
> >> >
> >> > I'm trying to submit a mapreduce job remotely using job.submit()
> >> >
> >> > I get the following:
> >> >
> >> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
> >> > GenericOptionsParser
> >> > for parsing the arguments. Applications should implement Tool for the
> >> > same.
> >> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
> >> > staging
> >> > area hdfs://{namenode
> >> >
> >> >
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
> >> >
> >> > and nothing happens...
> >> >
> >> > I set the the mapred.job.tracker and changed permissions for
> >> > hadoop.tmp.dir.
> >> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I
> think
> >> > that
> >> > it's not making any difference.
> >> > The system submitting the job is running with another user, call it:
> >> > myusername and not hadoop.
> >> >
> >> > I believe it is related to the user permissions but I can't seem to
> get
> >> > it
> >> > right.
> >> >
> >> > Thanks for the help,
> >> >
> >> > Amit.
> >> >
> >>
> >>
> >>
> >> --
> >> Harsh J
>
>
>
> --
> Harsh J
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
This is the JT ID and there is no problem running jobs from command line,
just remote.
On Apr 15, 2013 4:24 PM, "Harsh J" <ha...@cloudera.com> wrote:
> Thats interesting; is the JT you're running on the cluster started
> with the ID 201304150711 or something else?
>
> On Mon, Apr 15, 2013 at 6:47 PM, Amit Sela <am...@infolinks.com> wrote:
> > The client prints the two lines I posted and the cluster shows nothing.
> Not
> > even incrementing the number of submitted jobs.
> >
> > On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
> >>
> >> When you say "nothing happens"; where exactly do you mean? The client
> >> doesn't print anything, or the cluster doesn't run anything?
> >>
> >> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
> >> > Hi all,
> >> >
> >> > I'm trying to submit a mapreduce job remotely using job.submit()
> >> >
> >> > I get the following:
> >> >
> >> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
> >> > GenericOptionsParser
> >> > for parsing the arguments. Applications should implement Tool for the
> >> > same.
> >> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
> >> > staging
> >> > area hdfs://{namenode
> >> >
> >> >
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
> >> >
> >> > and nothing happens...
> >> >
> >> > I set the the mapred.job.tracker and changed permissions for
> >> > hadoop.tmp.dir.
> >> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I
> think
> >> > that
> >> > it's not making any difference.
> >> > The system submitting the job is running with another user, call it:
> >> > myusername and not hadoop.
> >> >
> >> > I believe it is related to the user permissions but I can't seem to
> get
> >> > it
> >> > right.
> >> >
> >> > Thanks for the help,
> >> >
> >> > Amit.
> >> >
> >>
> >>
> >>
> >> --
> >> Harsh J
>
>
>
> --
> Harsh J
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
This is the JT ID and there is no problem running jobs from command line,
just remote.
On Apr 15, 2013 4:24 PM, "Harsh J" <ha...@cloudera.com> wrote:
> Thats interesting; is the JT you're running on the cluster started
> with the ID 201304150711 or something else?
>
> On Mon, Apr 15, 2013 at 6:47 PM, Amit Sela <am...@infolinks.com> wrote:
> > The client prints the two lines I posted and the cluster shows nothing.
> Not
> > even incrementing the number of submitted jobs.
> >
> > On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
> >>
> >> When you say "nothing happens"; where exactly do you mean? The client
> >> doesn't print anything, or the cluster doesn't run anything?
> >>
> >> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
> >> > Hi all,
> >> >
> >> > I'm trying to submit a mapreduce job remotely using job.submit()
> >> >
> >> > I get the following:
> >> >
> >> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
> >> > GenericOptionsParser
> >> > for parsing the arguments. Applications should implement Tool for the
> >> > same.
> >> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
> >> > staging
> >> > area hdfs://{namenode
> >> >
> >> >
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
> >> >
> >> > and nothing happens...
> >> >
> >> > I set the the mapred.job.tracker and changed permissions for
> >> > hadoop.tmp.dir.
> >> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I
> think
> >> > that
> >> > it's not making any difference.
> >> > The system submitting the job is running with another user, call it:
> >> > myusername and not hadoop.
> >> >
> >> > I believe it is related to the user permissions but I can't seem to
> get
> >> > it
> >> > right.
> >> >
> >> > Thanks for the help,
> >> >
> >> > Amit.
> >> >
> >>
> >>
> >>
> >> --
> >> Harsh J
>
>
>
> --
> Harsh J
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
This is the JT ID and there is no problem running jobs from command line,
just remote.
On Apr 15, 2013 4:24 PM, "Harsh J" <ha...@cloudera.com> wrote:
> Thats interesting; is the JT you're running on the cluster started
> with the ID 201304150711 or something else?
>
> On Mon, Apr 15, 2013 at 6:47 PM, Amit Sela <am...@infolinks.com> wrote:
> > The client prints the two lines I posted and the cluster shows nothing.
> Not
> > even incrementing the number of submitted jobs.
> >
> > On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
> >>
> >> When you say "nothing happens"; where exactly do you mean? The client
> >> doesn't print anything, or the cluster doesn't run anything?
> >>
> >> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
> >> > Hi all,
> >> >
> >> > I'm trying to submit a mapreduce job remotely using job.submit()
> >> >
> >> > I get the following:
> >> >
> >> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
> >> > GenericOptionsParser
> >> > for parsing the arguments. Applications should implement Tool for the
> >> > same.
> >> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
> >> > staging
> >> > area hdfs://{namenode
> >> >
> >> >
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
> >> >
> >> > and nothing happens...
> >> >
> >> > I set the the mapred.job.tracker and changed permissions for
> >> > hadoop.tmp.dir.
> >> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I
> think
> >> > that
> >> > it's not making any difference.
> >> > The system submitting the job is running with another user, call it:
> >> > myusername and not hadoop.
> >> >
> >> > I believe it is related to the user permissions but I can't seem to
> get
> >> > it
> >> > right.
> >> >
> >> > Thanks for the help,
> >> >
> >> > Amit.
> >> >
> >>
> >>
> >>
> >> --
> >> Harsh J
>
>
>
> --
> Harsh J
>
Re: Submitting mapreduce and nothing happens
Posted by Harsh J <ha...@cloudera.com>.
Thats interesting; is the JT you're running on the cluster started
with the ID 201304150711 or something else?
On Mon, Apr 15, 2013 at 6:47 PM, Amit Sela <am...@infolinks.com> wrote:
> The client prints the two lines I posted and the cluster shows nothing. Not
> even incrementing the number of submitted jobs.
>
> On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
>>
>> When you say "nothing happens"; where exactly do you mean? The client
>> doesn't print anything, or the cluster doesn't run anything?
>>
>> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
>> > Hi all,
>> >
>> > I'm trying to submit a mapreduce job remotely using job.submit()
>> >
>> > I get the following:
>> >
>> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
>> > GenericOptionsParser
>> > for parsing the arguments. Applications should implement Tool for the
>> > same.
>> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
>> > staging
>> > area hdfs://{namenode
>> >
>> > address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>> >
>> > and nothing happens...
>> >
>> > I set the the mapred.job.tracker and changed permissions for
>> > hadoop.tmp.dir.
>> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I think
>> > that
>> > it's not making any difference.
>> > The system submitting the job is running with another user, call it:
>> > myusername and not hadoop.
>> >
>> > I believe it is related to the user permissions but I can't seem to get
>> > it
>> > right.
>> >
>> > Thanks for the help,
>> >
>> > Amit.
>> >
>>
>>
>>
>> --
>> Harsh J
--
Harsh J
Re: Submitting mapreduce and nothing happens
Posted by Harsh J <ha...@cloudera.com>.
Thats interesting; is the JT you're running on the cluster started
with the ID 201304150711 or something else?
On Mon, Apr 15, 2013 at 6:47 PM, Amit Sela <am...@infolinks.com> wrote:
> The client prints the two lines I posted and the cluster shows nothing. Not
> even incrementing the number of submitted jobs.
>
> On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
>>
>> When you say "nothing happens"; where exactly do you mean? The client
>> doesn't print anything, or the cluster doesn't run anything?
>>
>> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
>> > Hi all,
>> >
>> > I'm trying to submit a mapreduce job remotely using job.submit()
>> >
>> > I get the following:
>> >
>> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
>> > GenericOptionsParser
>> > for parsing the arguments. Applications should implement Tool for the
>> > same.
>> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
>> > staging
>> > area hdfs://{namenode
>> >
>> > address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>> >
>> > and nothing happens...
>> >
>> > I set the the mapred.job.tracker and changed permissions for
>> > hadoop.tmp.dir.
>> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I think
>> > that
>> > it's not making any difference.
>> > The system submitting the job is running with another user, call it:
>> > myusername and not hadoop.
>> >
>> > I believe it is related to the user permissions but I can't seem to get
>> > it
>> > right.
>> >
>> > Thanks for the help,
>> >
>> > Amit.
>> >
>>
>>
>>
>> --
>> Harsh J
--
Harsh J
Re: Submitting mapreduce and nothing happens
Posted by Harsh J <ha...@cloudera.com>.
Thats interesting; is the JT you're running on the cluster started
with the ID 201304150711 or something else?
On Mon, Apr 15, 2013 at 6:47 PM, Amit Sela <am...@infolinks.com> wrote:
> The client prints the two lines I posted and the cluster shows nothing. Not
> even incrementing the number of submitted jobs.
>
> On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
>>
>> When you say "nothing happens"; where exactly do you mean? The client
>> doesn't print anything, or the cluster doesn't run anything?
>>
>> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
>> > Hi all,
>> >
>> > I'm trying to submit a mapreduce job remotely using job.submit()
>> >
>> > I get the following:
>> >
>> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
>> > GenericOptionsParser
>> > for parsing the arguments. Applications should implement Tool for the
>> > same.
>> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
>> > staging
>> > area hdfs://{namenode
>> >
>> > address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>> >
>> > and nothing happens...
>> >
>> > I set the the mapred.job.tracker and changed permissions for
>> > hadoop.tmp.dir.
>> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I think
>> > that
>> > it's not making any difference.
>> > The system submitting the job is running with another user, call it:
>> > myusername and not hadoop.
>> >
>> > I believe it is related to the user permissions but I can't seem to get
>> > it
>> > right.
>> >
>> > Thanks for the help,
>> >
>> > Amit.
>> >
>>
>>
>>
>> --
>> Harsh J
--
Harsh J
Re: Submitting mapreduce and nothing happens
Posted by Harsh J <ha...@cloudera.com>.
Thats interesting; is the JT you're running on the cluster started
with the ID 201304150711 or something else?
On Mon, Apr 15, 2013 at 6:47 PM, Amit Sela <am...@infolinks.com> wrote:
> The client prints the two lines I posted and the cluster shows nothing. Not
> even incrementing the number of submitted jobs.
>
> On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
>>
>> When you say "nothing happens"; where exactly do you mean? The client
>> doesn't print anything, or the cluster doesn't run anything?
>>
>> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
>> > Hi all,
>> >
>> > I'm trying to submit a mapreduce job remotely using job.submit()
>> >
>> > I get the following:
>> >
>> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
>> > GenericOptionsParser
>> > for parsing the arguments. Applications should implement Tool for the
>> > same.
>> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
>> > staging
>> > area hdfs://{namenode
>> >
>> > address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>> >
>> > and nothing happens...
>> >
>> > I set the the mapred.job.tracker and changed permissions for
>> > hadoop.tmp.dir.
>> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I think
>> > that
>> > it's not making any difference.
>> > The system submitting the job is running with another user, call it:
>> > myusername and not hadoop.
>> >
>> > I believe it is related to the user permissions but I can't seem to get
>> > it
>> > right.
>> >
>> > Thanks for the help,
>> >
>> > Amit.
>> >
>>
>>
>>
>> --
>> Harsh J
--
Harsh J
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
The client prints the two lines I posted and the cluster shows nothing. Not
even incrementing the number of submitted jobs.
On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
> When you say "nothing happens"; where exactly do you mean? The client
> doesn't print anything, or the cluster doesn't run anything?
>
> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
> > Hi all,
> >
> > I'm trying to submit a mapreduce job remotely using job.submit()
> >
> > I get the following:
> >
> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
> GenericOptionsParser
> > for parsing the arguments. Applications should implement Tool for the
> same.
> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
> staging
> > area hdfs://{namenode
> >
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
> >
> > and nothing happens...
> >
> > I set the the mapred.job.tracker and changed permissions for
> hadoop.tmp.dir.
> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I think
> that
> > it's not making any difference.
> > The system submitting the job is running with another user, call it:
> > myusername and not hadoop.
> >
> > I believe it is related to the user permissions but I can't seem to get
> it
> > right.
> >
> > Thanks for the help,
> >
> > Amit.
> >
>
>
>
> --
> Harsh J
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
The client prints the two lines I posted and the cluster shows nothing. Not
even incrementing the number of submitted jobs.
On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
> When you say "nothing happens"; where exactly do you mean? The client
> doesn't print anything, or the cluster doesn't run anything?
>
> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
> > Hi all,
> >
> > I'm trying to submit a mapreduce job remotely using job.submit()
> >
> > I get the following:
> >
> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
> GenericOptionsParser
> > for parsing the arguments. Applications should implement Tool for the
> same.
> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
> staging
> > area hdfs://{namenode
> >
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
> >
> > and nothing happens...
> >
> > I set the the mapred.job.tracker and changed permissions for
> hadoop.tmp.dir.
> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I think
> that
> > it's not making any difference.
> > The system submitting the job is running with another user, call it:
> > myusername and not hadoop.
> >
> > I believe it is related to the user permissions but I can't seem to get
> it
> > right.
> >
> > Thanks for the help,
> >
> > Amit.
> >
>
>
>
> --
> Harsh J
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
The client prints the two lines I posted and the cluster shows nothing. Not
even incrementing the number of submitted jobs.
On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
> When you say "nothing happens"; where exactly do you mean? The client
> doesn't print anything, or the cluster doesn't run anything?
>
> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
> > Hi all,
> >
> > I'm trying to submit a mapreduce job remotely using job.submit()
> >
> > I get the following:
> >
> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
> GenericOptionsParser
> > for parsing the arguments. Applications should implement Tool for the
> same.
> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
> staging
> > area hdfs://{namenode
> >
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
> >
> > and nothing happens...
> >
> > I set the the mapred.job.tracker and changed permissions for
> hadoop.tmp.dir.
> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I think
> that
> > it's not making any difference.
> > The system submitting the job is running with another user, call it:
> > myusername and not hadoop.
> >
> > I believe it is related to the user permissions but I can't seem to get
> it
> > right.
> >
> > Thanks for the help,
> >
> > Amit.
> >
>
>
>
> --
> Harsh J
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
The client prints the two lines I posted and the cluster shows nothing. Not
even incrementing the number of submitted jobs.
On Apr 15, 2013 4:10 PM, "Harsh J" <ha...@cloudera.com> wrote:
> When you say "nothing happens"; where exactly do you mean? The client
> doesn't print anything, or the cluster doesn't run anything?
>
> On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
> > Hi all,
> >
> > I'm trying to submit a mapreduce job remotely using job.submit()
> >
> > I get the following:
> >
> > [WARN ] org.apache.hadoop.mapred.JobClient » Use
> GenericOptionsParser
> > for parsing the arguments. Applications should implement Tool for the
> same.
> > [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
> staging
> > area hdfs://{namenode
> >
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
> >
> > and nothing happens...
> >
> > I set the the mapred.job.tracker and changed permissions for
> hadoop.tmp.dir.
> > I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I think
> that
> > it's not making any difference.
> > The system submitting the job is running with another user, call it:
> > myusername and not hadoop.
> >
> > I believe it is related to the user permissions but I can't seem to get
> it
> > right.
> >
> > Thanks for the help,
> >
> > Amit.
> >
>
>
>
> --
> Harsh J
>
Re: Submitting mapreduce and nothing happens
Posted by Harsh J <ha...@cloudera.com>.
When you say "nothing happens"; where exactly do you mean? The client
doesn't print anything, or the cluster doesn't run anything?
On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
> Hi all,
>
> I'm trying to submit a mapreduce job remotely using job.submit()
>
> I get the following:
>
> [WARN ] org.apache.hadoop.mapred.JobClient » Use GenericOptionsParser
> for parsing the arguments. Applications should implement Tool for the same.
> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
> area hdfs://{namenode
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>
> and nothing happens...
>
> I set the the mapred.job.tracker and changed permissions for hadoop.tmp.dir.
> I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I think that
> it's not making any difference.
> The system submitting the job is running with another user, call it:
> myusername and not hadoop.
>
> I believe it is related to the user permissions but I can't seem to get it
> right.
>
> Thanks for the help,
>
> Amit.
>
--
Harsh J
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
Input path includes data in it. I also tried job.waitFromComplete(true) but
it acts exactly the same.
For what it's worth, in the staging dir in hdfs I do see empty (or cleaned
up) folders created with the correct JT ID and with an incremented count:
job_201304150711_01, job_201304150711_02....job_201304150711_37 while the
webapp shows 11 submissions that were actually executed (not remotely...)
On Wed, Apr 17, 2013 at 6:40 AM, Zizon Qiu <zz...@gmail.com> wrote:
> try use job.waitFromComplete(true) instead of job.submit().
> it should show more details.
>
>
> On Mon, Apr 15, 2013 at 6:06 PM, Amit Sela <am...@infolinks.com> wrote:
>
>> Hi all,
>>
>> I'm trying to submit a mapreduce job remotely using job.submit()
>>
>> I get the following:
>>
>> [WARN ] org.apache.hadoop.mapred.JobClient » Use
>> GenericOptionsParser for parsing the arguments. Applications should
>> implement Tool for the same.
>> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
>> staging area hdfs://{namenode
>> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>>
>> and nothing happens...
>>
>> I set the the mapred.job.tracker and changed permissions
>> for hadoop.tmp.dir. I also set "hadoop.job.ugi" as "hadoop,supergroup" but
>> some how I think that it's not making any difference.
>> The system submitting the job is running with another user, call it:
>> myusername and not hadoop.
>>
>> I believe it is related to the user permissions but I can't seem to get
>> it right.
>>
>> Thanks for the help,
>>
>> Amit.
>>
>>
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
Input path includes data in it. I also tried job.waitFromComplete(true) but
it acts exactly the same.
For what it's worth, in the staging dir in hdfs I do see empty (or cleaned
up) folders created with the correct JT ID and with an incremented count:
job_201304150711_01, job_201304150711_02....job_201304150711_37 while the
webapp shows 11 submissions that were actually executed (not remotely...)
On Wed, Apr 17, 2013 at 6:40 AM, Zizon Qiu <zz...@gmail.com> wrote:
> try use job.waitFromComplete(true) instead of job.submit().
> it should show more details.
>
>
> On Mon, Apr 15, 2013 at 6:06 PM, Amit Sela <am...@infolinks.com> wrote:
>
>> Hi all,
>>
>> I'm trying to submit a mapreduce job remotely using job.submit()
>>
>> I get the following:
>>
>> [WARN ] org.apache.hadoop.mapred.JobClient » Use
>> GenericOptionsParser for parsing the arguments. Applications should
>> implement Tool for the same.
>> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
>> staging area hdfs://{namenode
>> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>>
>> and nothing happens...
>>
>> I set the the mapred.job.tracker and changed permissions
>> for hadoop.tmp.dir. I also set "hadoop.job.ugi" as "hadoop,supergroup" but
>> some how I think that it's not making any difference.
>> The system submitting the job is running with another user, call it:
>> myusername and not hadoop.
>>
>> I believe it is related to the user permissions but I can't seem to get
>> it right.
>>
>> Thanks for the help,
>>
>> Amit.
>>
>>
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
Input path includes data in it. I also tried job.waitFromComplete(true) but
it acts exactly the same.
For what it's worth, in the staging dir in hdfs I do see empty (or cleaned
up) folders created with the correct JT ID and with an incremented count:
job_201304150711_01, job_201304150711_02....job_201304150711_37 while the
webapp shows 11 submissions that were actually executed (not remotely...)
On Wed, Apr 17, 2013 at 6:40 AM, Zizon Qiu <zz...@gmail.com> wrote:
> try use job.waitFromComplete(true) instead of job.submit().
> it should show more details.
>
>
> On Mon, Apr 15, 2013 at 6:06 PM, Amit Sela <am...@infolinks.com> wrote:
>
>> Hi all,
>>
>> I'm trying to submit a mapreduce job remotely using job.submit()
>>
>> I get the following:
>>
>> [WARN ] org.apache.hadoop.mapred.JobClient » Use
>> GenericOptionsParser for parsing the arguments. Applications should
>> implement Tool for the same.
>> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
>> staging area hdfs://{namenode
>> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>>
>> and nothing happens...
>>
>> I set the the mapred.job.tracker and changed permissions
>> for hadoop.tmp.dir. I also set "hadoop.job.ugi" as "hadoop,supergroup" but
>> some how I think that it's not making any difference.
>> The system submitting the job is running with another user, call it:
>> myusername and not hadoop.
>>
>> I believe it is related to the user permissions but I can't seem to get
>> it right.
>>
>> Thanks for the help,
>>
>> Amit.
>>
>>
>
Re: Submitting mapreduce and nothing happens
Posted by Amit Sela <am...@infolinks.com>.
Input path includes data in it. I also tried job.waitFromComplete(true) but
it acts exactly the same.
For what it's worth, in the staging dir in hdfs I do see empty (or cleaned
up) folders created with the correct JT ID and with an incremented count:
job_201304150711_01, job_201304150711_02....job_201304150711_37 while the
webapp shows 11 submissions that were actually executed (not remotely...)
On Wed, Apr 17, 2013 at 6:40 AM, Zizon Qiu <zz...@gmail.com> wrote:
> try use job.waitFromComplete(true) instead of job.submit().
> it should show more details.
>
>
> On Mon, Apr 15, 2013 at 6:06 PM, Amit Sela <am...@infolinks.com> wrote:
>
>> Hi all,
>>
>> I'm trying to submit a mapreduce job remotely using job.submit()
>>
>> I get the following:
>>
>> [WARN ] org.apache.hadoop.mapred.JobClient » Use
>> GenericOptionsParser for parsing the arguments. Applications should
>> implement Tool for the same.
>> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the
>> staging area hdfs://{namenode
>> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>>
>> and nothing happens...
>>
>> I set the the mapred.job.tracker and changed permissions
>> for hadoop.tmp.dir. I also set "hadoop.job.ugi" as "hadoop,supergroup" but
>> some how I think that it's not making any difference.
>> The system submitting the job is running with another user, call it:
>> myusername and not hadoop.
>>
>> I believe it is related to the user permissions but I can't seem to get
>> it right.
>>
>> Thanks for the help,
>>
>> Amit.
>>
>>
>
Re: Submitting mapreduce and nothing happens
Posted by Zizon Qiu <zz...@gmail.com>.
try use job.waitFromComplete(true) instead of job.submit().
it should show more details.
On Mon, Apr 15, 2013 at 6:06 PM, Amit Sela <am...@infolinks.com> wrote:
> Hi all,
>
> I'm trying to submit a mapreduce job remotely using job.submit()
>
> I get the following:
>
> [WARN ] org.apache.hadoop.mapred.JobClient » Use
> GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
> area hdfs://{namenode
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>
> and nothing happens...
>
> I set the the mapred.job.tracker and changed permissions
> for hadoop.tmp.dir. I also set "hadoop.job.ugi" as "hadoop,supergroup" but
> some how I think that it's not making any difference.
> The system submitting the job is running with another user, call it:
> myusername and not hadoop.
>
> I believe it is related to the user permissions but I can't seem to get it
> right.
>
> Thanks for the help,
>
> Amit.
>
>
Re: Submitting mapreduce and nothing happens
Posted by Zizon Qiu <zz...@gmail.com>.
try use job.waitFromComplete(true) instead of job.submit().
it should show more details.
On Mon, Apr 15, 2013 at 6:06 PM, Amit Sela <am...@infolinks.com> wrote:
> Hi all,
>
> I'm trying to submit a mapreduce job remotely using job.submit()
>
> I get the following:
>
> [WARN ] org.apache.hadoop.mapred.JobClient » Use
> GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
> area hdfs://{namenode
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>
> and nothing happens...
>
> I set the the mapred.job.tracker and changed permissions
> for hadoop.tmp.dir. I also set "hadoop.job.ugi" as "hadoop,supergroup" but
> some how I think that it's not making any difference.
> The system submitting the job is running with another user, call it:
> myusername and not hadoop.
>
> I believe it is related to the user permissions but I can't seem to get it
> right.
>
> Thanks for the help,
>
> Amit.
>
>
Re: Submitting mapreduce and nothing happens
Posted by Zizon Qiu <zz...@gmail.com>.
try use job.waitFromComplete(true) instead of job.submit().
it should show more details.
On Mon, Apr 15, 2013 at 6:06 PM, Amit Sela <am...@infolinks.com> wrote:
> Hi all,
>
> I'm trying to submit a mapreduce job remotely using job.submit()
>
> I get the following:
>
> [WARN ] org.apache.hadoop.mapred.JobClient » Use
> GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
> area hdfs://{namenode
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>
> and nothing happens...
>
> I set the the mapred.job.tracker and changed permissions
> for hadoop.tmp.dir. I also set "hadoop.job.ugi" as "hadoop,supergroup" but
> some how I think that it's not making any difference.
> The system submitting the job is running with another user, call it:
> myusername and not hadoop.
>
> I believe it is related to the user permissions but I can't seem to get it
> right.
>
> Thanks for the help,
>
> Amit.
>
>
Re: Submitting mapreduce and nothing happens
Posted by Zizon Qiu <zz...@gmail.com>.
try use job.waitFromComplete(true) instead of job.submit().
it should show more details.
On Mon, Apr 15, 2013 at 6:06 PM, Amit Sela <am...@infolinks.com> wrote:
> Hi all,
>
> I'm trying to submit a mapreduce job remotely using job.submit()
>
> I get the following:
>
> [WARN ] org.apache.hadoop.mapred.JobClient » Use
> GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
> area hdfs://{namenode
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>
> and nothing happens...
>
> I set the the mapred.job.tracker and changed permissions
> for hadoop.tmp.dir. I also set "hadoop.job.ugi" as "hadoop,supergroup" but
> some how I think that it's not making any difference.
> The system submitting the job is running with another user, call it:
> myusername and not hadoop.
>
> I believe it is related to the user permissions but I can't seem to get it
> right.
>
> Thanks for the help,
>
> Amit.
>
>
Re: Submitting mapreduce and nothing happens
Posted by Harsh J <ha...@cloudera.com>.
When you say "nothing happens"; where exactly do you mean? The client
doesn't print anything, or the cluster doesn't run anything?
On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
> Hi all,
>
> I'm trying to submit a mapreduce job remotely using job.submit()
>
> I get the following:
>
> [WARN ] org.apache.hadoop.mapred.JobClient » Use GenericOptionsParser
> for parsing the arguments. Applications should implement Tool for the same.
> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
> area hdfs://{namenode
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>
> and nothing happens...
>
> I set the the mapred.job.tracker and changed permissions for hadoop.tmp.dir.
> I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I think that
> it's not making any difference.
> The system submitting the job is running with another user, call it:
> myusername and not hadoop.
>
> I believe it is related to the user permissions but I can't seem to get it
> right.
>
> Thanks for the help,
>
> Amit.
>
--
Harsh J
Re: Submitting mapreduce and nothing happens
Posted by Harsh J <ha...@cloudera.com>.
When you say "nothing happens"; where exactly do you mean? The client
doesn't print anything, or the cluster doesn't run anything?
On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
> Hi all,
>
> I'm trying to submit a mapreduce job remotely using job.submit()
>
> I get the following:
>
> [WARN ] org.apache.hadoop.mapred.JobClient » Use GenericOptionsParser
> for parsing the arguments. Applications should implement Tool for the same.
> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
> area hdfs://{namenode
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>
> and nothing happens...
>
> I set the the mapred.job.tracker and changed permissions for hadoop.tmp.dir.
> I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I think that
> it's not making any difference.
> The system submitting the job is running with another user, call it:
> myusername and not hadoop.
>
> I believe it is related to the user permissions but I can't seem to get it
> right.
>
> Thanks for the help,
>
> Amit.
>
--
Harsh J
Re: Submitting mapreduce and nothing happens
Posted by Harsh J <ha...@cloudera.com>.
When you say "nothing happens"; where exactly do you mean? The client
doesn't print anything, or the cluster doesn't run anything?
On Mon, Apr 15, 2013 at 3:36 PM, Amit Sela <am...@infolinks.com> wrote:
> Hi all,
>
> I'm trying to submit a mapreduce job remotely using job.submit()
>
> I get the following:
>
> [WARN ] org.apache.hadoop.mapred.JobClient » Use GenericOptionsParser
> for parsing the arguments. Applications should implement Tool for the same.
> [INFO ] org.apache.hadoop.mapred.JobClient » Cleaning up the staging
> area hdfs://{namenode
> address}:{port}{hadoop.tmp.dir}/mapred/staging/myusername/.staging/job_201304150711_0022
>
> and nothing happens...
>
> I set the the mapred.job.tracker and changed permissions for hadoop.tmp.dir.
> I also set "hadoop.job.ugi" as "hadoop,supergroup" but some how I think that
> it's not making any difference.
> The system submitting the job is running with another user, call it:
> myusername and not hadoop.
>
> I believe it is related to the user permissions but I can't seem to get it
> right.
>
> Thanks for the help,
>
> Amit.
>
--
Harsh J