You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Ashwin Shankar <as...@gmail.com> on 2014/03/21 09:32:05 UTC
Job fails if I change HADOOP_USER_NAME
Hi,
I'm writing a new feature in Fair scheduler and wanted to test it out
by running jobs submitted by different users from my laptop.
My sleep job runs fine as long as the user name is my mac user name.
If I change my hadoop user name by setting HADOOP_USER_NAME,
my jobs fail with the exception
*org.apache.hadoop.util.Shell$ExitCodeException.*
I also tried creating a new user account on my laptop and running a job as
that user but I get the same exception.
Please let me know if any of you have come across this.
I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
Here is the stack trace :
Job job_1395389889916_0001 failed with state FAILED due to: Application
application_1395389889916_0001 failed 3 times due to AM Container for
appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to:
Exception from container-launch:
org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
--
Thanks,
Ashwin
RE: Job fails if I change HADOOP_USER_NAME
Posted by Rohith Sharma K S <ro...@huawei.com>.
Hi Ashwin,
How I enable debug for AM container logs ?
>> Set below configurations changing log level for AM, map and reducer task. Default values are INFO.
yarn.app.mapreduce.am.log.level
mapreduce.map.log.level
mapreduce.reduce.log.level
and to which location are they written to ?
>> These are written into {yarn.nodemanager.log-dirs}/<application-id> while executing job.
Once application is finished,
1. If log aggregation is enabled, then all container logs aggregated to HDFS. The log path in hdfs is {yarn.nodemanager.remote-app-log-dir}/${user}
2. If log aggregation is disabled,then all container logs remain in local machine where containers has run i.e {yarn.nodemanager.log-dirs}/<application-id.
Thanks & Regards
Rohith Sharma K S
From: Ashwin Shankar [mailto:ashwinshankar77@gmail.com]
Sent: 22 March 2014 03:38
To: user@hadoop.apache.org
Subject: Re: Job fails if I change HADOOP_USER_NAME
Hi Rohit,
How I enable debug for AM container logs ? and to which location are they written to ?
I tried changing log4j.prop and can see DEBUGs for RM,NM etc but I don't see AM related debug logs.
Thanks,
Ashwin
On Fri, Mar 21, 2014 at 3:05 AM, Rohith Sharma K S <ro...@huawei.com>> wrote:
Hi
The below stack trace is generic for any am launcher failed to launch. Can debug on AM container logs, so get proper stacktrace.?
Thanks & Regards
Rohith Sharma K S
From: Ashwin Shankar [mailto:ashwinshankar77@gmail.com<ma...@gmail.com>]
Sent: 21 March 2014 14:02
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Job fails if I change HADOOP_USER_NAME
Hi,
I'm writing a new feature in Fair scheduler and wanted to test it out
by running jobs submitted by different users from my laptop.
My sleep job runs fine as long as the user name is my mac user name.
If I change my hadoop user name by setting HADOOP_USER_NAME,
my jobs fail with the exception org.apache.hadoop.util.Shell$ExitCodeException.
I also tried creating a new user account on my laptop and running a job as that user but I get the same exception.
Please let me know if any of you have come across this.
I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
Here is the stack trace :
Job job_1395389889916_0001 failed with state FAILED due to: Application application_1395389889916_0001 failed 3 times due to AM Container for appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
--
Thanks,
Ashwin
--
Thanks,
Ashwin
RE: Job fails if I change HADOOP_USER_NAME
Posted by Rohith Sharma K S <ro...@huawei.com>.
Hi Ashwin,
How I enable debug for AM container logs ?
>> Set below configurations changing log level for AM, map and reducer task. Default values are INFO.
yarn.app.mapreduce.am.log.level
mapreduce.map.log.level
mapreduce.reduce.log.level
and to which location are they written to ?
>> These are written into {yarn.nodemanager.log-dirs}/<application-id> while executing job.
Once application is finished,
1. If log aggregation is enabled, then all container logs aggregated to HDFS. The log path in hdfs is {yarn.nodemanager.remote-app-log-dir}/${user}
2. If log aggregation is disabled,then all container logs remain in local machine where containers has run i.e {yarn.nodemanager.log-dirs}/<application-id.
Thanks & Regards
Rohith Sharma K S
From: Ashwin Shankar [mailto:ashwinshankar77@gmail.com]
Sent: 22 March 2014 03:38
To: user@hadoop.apache.org
Subject: Re: Job fails if I change HADOOP_USER_NAME
Hi Rohit,
How I enable debug for AM container logs ? and to which location are they written to ?
I tried changing log4j.prop and can see DEBUGs for RM,NM etc but I don't see AM related debug logs.
Thanks,
Ashwin
On Fri, Mar 21, 2014 at 3:05 AM, Rohith Sharma K S <ro...@huawei.com>> wrote:
Hi
The below stack trace is generic for any am launcher failed to launch. Can debug on AM container logs, so get proper stacktrace.?
Thanks & Regards
Rohith Sharma K S
From: Ashwin Shankar [mailto:ashwinshankar77@gmail.com<ma...@gmail.com>]
Sent: 21 March 2014 14:02
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Job fails if I change HADOOP_USER_NAME
Hi,
I'm writing a new feature in Fair scheduler and wanted to test it out
by running jobs submitted by different users from my laptop.
My sleep job runs fine as long as the user name is my mac user name.
If I change my hadoop user name by setting HADOOP_USER_NAME,
my jobs fail with the exception org.apache.hadoop.util.Shell$ExitCodeException.
I also tried creating a new user account on my laptop and running a job as that user but I get the same exception.
Please let me know if any of you have come across this.
I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
Here is the stack trace :
Job job_1395389889916_0001 failed with state FAILED due to: Application application_1395389889916_0001 failed 3 times due to AM Container for appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
--
Thanks,
Ashwin
--
Thanks,
Ashwin
RE: Job fails if I change HADOOP_USER_NAME
Posted by Rohith Sharma K S <ro...@huawei.com>.
Hi Ashwin,
How I enable debug for AM container logs ?
>> Set below configurations changing log level for AM, map and reducer task. Default values are INFO.
yarn.app.mapreduce.am.log.level
mapreduce.map.log.level
mapreduce.reduce.log.level
and to which location are they written to ?
>> These are written into {yarn.nodemanager.log-dirs}/<application-id> while executing job.
Once application is finished,
1. If log aggregation is enabled, then all container logs aggregated to HDFS. The log path in hdfs is {yarn.nodemanager.remote-app-log-dir}/${user}
2. If log aggregation is disabled,then all container logs remain in local machine where containers has run i.e {yarn.nodemanager.log-dirs}/<application-id.
Thanks & Regards
Rohith Sharma K S
From: Ashwin Shankar [mailto:ashwinshankar77@gmail.com]
Sent: 22 March 2014 03:38
To: user@hadoop.apache.org
Subject: Re: Job fails if I change HADOOP_USER_NAME
Hi Rohit,
How I enable debug for AM container logs ? and to which location are they written to ?
I tried changing log4j.prop and can see DEBUGs for RM,NM etc but I don't see AM related debug logs.
Thanks,
Ashwin
On Fri, Mar 21, 2014 at 3:05 AM, Rohith Sharma K S <ro...@huawei.com>> wrote:
Hi
The below stack trace is generic for any am launcher failed to launch. Can debug on AM container logs, so get proper stacktrace.?
Thanks & Regards
Rohith Sharma K S
From: Ashwin Shankar [mailto:ashwinshankar77@gmail.com<ma...@gmail.com>]
Sent: 21 March 2014 14:02
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Job fails if I change HADOOP_USER_NAME
Hi,
I'm writing a new feature in Fair scheduler and wanted to test it out
by running jobs submitted by different users from my laptop.
My sleep job runs fine as long as the user name is my mac user name.
If I change my hadoop user name by setting HADOOP_USER_NAME,
my jobs fail with the exception org.apache.hadoop.util.Shell$ExitCodeException.
I also tried creating a new user account on my laptop and running a job as that user but I get the same exception.
Please let me know if any of you have come across this.
I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
Here is the stack trace :
Job job_1395389889916_0001 failed with state FAILED due to: Application application_1395389889916_0001 failed 3 times due to AM Container for appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
--
Thanks,
Ashwin
--
Thanks,
Ashwin
RE: Job fails if I change HADOOP_USER_NAME
Posted by Rohith Sharma K S <ro...@huawei.com>.
Hi Ashwin,
How I enable debug for AM container logs ?
>> Set below configurations changing log level for AM, map and reducer task. Default values are INFO.
yarn.app.mapreduce.am.log.level
mapreduce.map.log.level
mapreduce.reduce.log.level
and to which location are they written to ?
>> These are written into {yarn.nodemanager.log-dirs}/<application-id> while executing job.
Once application is finished,
1. If log aggregation is enabled, then all container logs aggregated to HDFS. The log path in hdfs is {yarn.nodemanager.remote-app-log-dir}/${user}
2. If log aggregation is disabled,then all container logs remain in local machine where containers has run i.e {yarn.nodemanager.log-dirs}/<application-id.
Thanks & Regards
Rohith Sharma K S
From: Ashwin Shankar [mailto:ashwinshankar77@gmail.com]
Sent: 22 March 2014 03:38
To: user@hadoop.apache.org
Subject: Re: Job fails if I change HADOOP_USER_NAME
Hi Rohit,
How I enable debug for AM container logs ? and to which location are they written to ?
I tried changing log4j.prop and can see DEBUGs for RM,NM etc but I don't see AM related debug logs.
Thanks,
Ashwin
On Fri, Mar 21, 2014 at 3:05 AM, Rohith Sharma K S <ro...@huawei.com>> wrote:
Hi
The below stack trace is generic for any am launcher failed to launch. Can debug on AM container logs, so get proper stacktrace.?
Thanks & Regards
Rohith Sharma K S
From: Ashwin Shankar [mailto:ashwinshankar77@gmail.com<ma...@gmail.com>]
Sent: 21 March 2014 14:02
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Job fails if I change HADOOP_USER_NAME
Hi,
I'm writing a new feature in Fair scheduler and wanted to test it out
by running jobs submitted by different users from my laptop.
My sleep job runs fine as long as the user name is my mac user name.
If I change my hadoop user name by setting HADOOP_USER_NAME,
my jobs fail with the exception org.apache.hadoop.util.Shell$ExitCodeException.
I also tried creating a new user account on my laptop and running a job as that user but I get the same exception.
Please let me know if any of you have come across this.
I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
Here is the stack trace :
Job job_1395389889916_0001 failed with state FAILED due to: Application application_1395389889916_0001 failed 3 times due to AM Container for appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
--
Thanks,
Ashwin
--
Thanks,
Ashwin
Re: Job fails if I change HADOOP_USER_NAME
Posted by Ashwin Shankar <as...@gmail.com>.
Hi Rohit,
How I enable debug for AM container logs ? and to which location are they
written to ?
I tried changing log4j.prop and can see DEBUGs for RM,NM etc but I don't
see AM related debug logs.
Thanks,
Ashwin
On Fri, Mar 21, 2014 at 3:05 AM, Rohith Sharma K S <
rohithsharmaks@huawei.com> wrote:
> Hi
>
>
>
> The below stack trace is generic for any am launcher failed to launch. Can
> debug on AM container logs, so get proper stacktrace.?
>
>
>
>
>
> Thanks & Regards
>
> Rohith Sharma K S
>
>
>
> *From:* Ashwin Shankar [mailto:ashwinshankar77@gmail.com]
> *Sent:* 21 March 2014 14:02
> *To:* user@hadoop.apache.org
> *Subject:* Job fails if I change HADOOP_USER_NAME
>
>
>
> Hi,
>
> I'm writing a new feature in Fair scheduler and wanted to test it out
>
> by running jobs submitted by different users from my laptop.
>
>
>
> My sleep job runs fine as long as the user name is my mac user name.
>
> If I change my hadoop user name by setting HADOOP_USER_NAME,
>
> my jobs fail with the exception
> *org.apache.hadoop.util.Shell$ExitCodeException.*
>
> I also tried creating a new user account on my laptop and running a job as
> that user but I get the same exception.
>
>
>
> Please let me know if any of you have come across this.
>
> I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
>
>
>
> Here is the stack trace :
>
>
>
> Job job_1395389889916_0001 failed with state FAILED due to: Application
> application_1395389889916_0001 failed 3 times due to AM Container for
> appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to:
> Exception from container-launch:
> org.apache.hadoop.util.Shell$ExitCodeException:
>
> org.apache.hadoop.util.Shell$ExitCodeException:
>
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
>
> at org.apache.hadoop.util.Shell.run(Shell.java:418)
>
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
>
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>
>
>
> --
>
> Thanks,
> Ashwin
>
>
--
Thanks,
Ashwin
Re: Job fails if I change HADOOP_USER_NAME
Posted by Ashwin Shankar <as...@gmail.com>.
Hi Rohit,
How I enable debug for AM container logs ? and to which location are they
written to ?
I tried changing log4j.prop and can see DEBUGs for RM,NM etc but I don't
see AM related debug logs.
Thanks,
Ashwin
On Fri, Mar 21, 2014 at 3:05 AM, Rohith Sharma K S <
rohithsharmaks@huawei.com> wrote:
> Hi
>
>
>
> The below stack trace is generic for any am launcher failed to launch. Can
> debug on AM container logs, so get proper stacktrace.?
>
>
>
>
>
> Thanks & Regards
>
> Rohith Sharma K S
>
>
>
> *From:* Ashwin Shankar [mailto:ashwinshankar77@gmail.com]
> *Sent:* 21 March 2014 14:02
> *To:* user@hadoop.apache.org
> *Subject:* Job fails if I change HADOOP_USER_NAME
>
>
>
> Hi,
>
> I'm writing a new feature in Fair scheduler and wanted to test it out
>
> by running jobs submitted by different users from my laptop.
>
>
>
> My sleep job runs fine as long as the user name is my mac user name.
>
> If I change my hadoop user name by setting HADOOP_USER_NAME,
>
> my jobs fail with the exception
> *org.apache.hadoop.util.Shell$ExitCodeException.*
>
> I also tried creating a new user account on my laptop and running a job as
> that user but I get the same exception.
>
>
>
> Please let me know if any of you have come across this.
>
> I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
>
>
>
> Here is the stack trace :
>
>
>
> Job job_1395389889916_0001 failed with state FAILED due to: Application
> application_1395389889916_0001 failed 3 times due to AM Container for
> appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to:
> Exception from container-launch:
> org.apache.hadoop.util.Shell$ExitCodeException:
>
> org.apache.hadoop.util.Shell$ExitCodeException:
>
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
>
> at org.apache.hadoop.util.Shell.run(Shell.java:418)
>
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
>
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>
>
>
> --
>
> Thanks,
> Ashwin
>
>
--
Thanks,
Ashwin
Re: Job fails if I change HADOOP_USER_NAME
Posted by Ashwin Shankar <as...@gmail.com>.
Hi Rohit,
How I enable debug for AM container logs ? and to which location are they
written to ?
I tried changing log4j.prop and can see DEBUGs for RM,NM etc but I don't
see AM related debug logs.
Thanks,
Ashwin
On Fri, Mar 21, 2014 at 3:05 AM, Rohith Sharma K S <
rohithsharmaks@huawei.com> wrote:
> Hi
>
>
>
> The below stack trace is generic for any am launcher failed to launch. Can
> debug on AM container logs, so get proper stacktrace.?
>
>
>
>
>
> Thanks & Regards
>
> Rohith Sharma K S
>
>
>
> *From:* Ashwin Shankar [mailto:ashwinshankar77@gmail.com]
> *Sent:* 21 March 2014 14:02
> *To:* user@hadoop.apache.org
> *Subject:* Job fails if I change HADOOP_USER_NAME
>
>
>
> Hi,
>
> I'm writing a new feature in Fair scheduler and wanted to test it out
>
> by running jobs submitted by different users from my laptop.
>
>
>
> My sleep job runs fine as long as the user name is my mac user name.
>
> If I change my hadoop user name by setting HADOOP_USER_NAME,
>
> my jobs fail with the exception
> *org.apache.hadoop.util.Shell$ExitCodeException.*
>
> I also tried creating a new user account on my laptop and running a job as
> that user but I get the same exception.
>
>
>
> Please let me know if any of you have come across this.
>
> I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
>
>
>
> Here is the stack trace :
>
>
>
> Job job_1395389889916_0001 failed with state FAILED due to: Application
> application_1395389889916_0001 failed 3 times due to AM Container for
> appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to:
> Exception from container-launch:
> org.apache.hadoop.util.Shell$ExitCodeException:
>
> org.apache.hadoop.util.Shell$ExitCodeException:
>
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
>
> at org.apache.hadoop.util.Shell.run(Shell.java:418)
>
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
>
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>
>
>
> --
>
> Thanks,
> Ashwin
>
>
--
Thanks,
Ashwin
Re: Job fails if I change HADOOP_USER_NAME
Posted by Ashwin Shankar <as...@gmail.com>.
Hi Rohit,
How I enable debug for AM container logs ? and to which location are they
written to ?
I tried changing log4j.prop and can see DEBUGs for RM,NM etc but I don't
see AM related debug logs.
Thanks,
Ashwin
On Fri, Mar 21, 2014 at 3:05 AM, Rohith Sharma K S <
rohithsharmaks@huawei.com> wrote:
> Hi
>
>
>
> The below stack trace is generic for any am launcher failed to launch. Can
> debug on AM container logs, so get proper stacktrace.?
>
>
>
>
>
> Thanks & Regards
>
> Rohith Sharma K S
>
>
>
> *From:* Ashwin Shankar [mailto:ashwinshankar77@gmail.com]
> *Sent:* 21 March 2014 14:02
> *To:* user@hadoop.apache.org
> *Subject:* Job fails if I change HADOOP_USER_NAME
>
>
>
> Hi,
>
> I'm writing a new feature in Fair scheduler and wanted to test it out
>
> by running jobs submitted by different users from my laptop.
>
>
>
> My sleep job runs fine as long as the user name is my mac user name.
>
> If I change my hadoop user name by setting HADOOP_USER_NAME,
>
> my jobs fail with the exception
> *org.apache.hadoop.util.Shell$ExitCodeException.*
>
> I also tried creating a new user account on my laptop and running a job as
> that user but I get the same exception.
>
>
>
> Please let me know if any of you have come across this.
>
> I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
>
>
>
> Here is the stack trace :
>
>
>
> Job job_1395389889916_0001 failed with state FAILED due to: Application
> application_1395389889916_0001 failed 3 times due to AM Container for
> appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to:
> Exception from container-launch:
> org.apache.hadoop.util.Shell$ExitCodeException:
>
> org.apache.hadoop.util.Shell$ExitCodeException:
>
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
>
> at org.apache.hadoop.util.Shell.run(Shell.java:418)
>
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
>
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>
>
>
> --
>
> Thanks,
> Ashwin
>
>
--
Thanks,
Ashwin
RE: Job fails if I change HADOOP_USER_NAME
Posted by Rohith Sharma K S <ro...@huawei.com>.
Hi
The below stack trace is generic for any am launcher failed to launch. Can debug on AM container logs, so get proper stacktrace.?
Thanks & Regards
Rohith Sharma K S
From: Ashwin Shankar [mailto:ashwinshankar77@gmail.com]
Sent: 21 March 2014 14:02
To: user@hadoop.apache.org
Subject: Job fails if I change HADOOP_USER_NAME
Hi,
I'm writing a new feature in Fair scheduler and wanted to test it out
by running jobs submitted by different users from my laptop.
My sleep job runs fine as long as the user name is my mac user name.
If I change my hadoop user name by setting HADOOP_USER_NAME,
my jobs fail with the exception org.apache.hadoop.util.Shell$ExitCodeException.
I also tried creating a new user account on my laptop and running a job as that user but I get the same exception.
Please let me know if any of you have come across this.
I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
Here is the stack trace :
Job job_1395389889916_0001 failed with state FAILED due to: Application application_1395389889916_0001 failed 3 times due to AM Container for appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
--
Thanks,
Ashwin
RE: Job fails if I change HADOOP_USER_NAME
Posted by Rohith Sharma K S <ro...@huawei.com>.
Hi
The below stack trace is generic for any am launcher failed to launch. Can debug on AM container logs, so get proper stacktrace.?
Thanks & Regards
Rohith Sharma K S
From: Ashwin Shankar [mailto:ashwinshankar77@gmail.com]
Sent: 21 March 2014 14:02
To: user@hadoop.apache.org
Subject: Job fails if I change HADOOP_USER_NAME
Hi,
I'm writing a new feature in Fair scheduler and wanted to test it out
by running jobs submitted by different users from my laptop.
My sleep job runs fine as long as the user name is my mac user name.
If I change my hadoop user name by setting HADOOP_USER_NAME,
my jobs fail with the exception org.apache.hadoop.util.Shell$ExitCodeException.
I also tried creating a new user account on my laptop and running a job as that user but I get the same exception.
Please let me know if any of you have come across this.
I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
Here is the stack trace :
Job job_1395389889916_0001 failed with state FAILED due to: Application application_1395389889916_0001 failed 3 times due to AM Container for appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
--
Thanks,
Ashwin
RE: Job fails if I change HADOOP_USER_NAME
Posted by Rohith Sharma K S <ro...@huawei.com>.
Hi
The below stack trace is generic for any am launcher failed to launch. Can debug on AM container logs, so get proper stacktrace.?
Thanks & Regards
Rohith Sharma K S
From: Ashwin Shankar [mailto:ashwinshankar77@gmail.com]
Sent: 21 March 2014 14:02
To: user@hadoop.apache.org
Subject: Job fails if I change HADOOP_USER_NAME
Hi,
I'm writing a new feature in Fair scheduler and wanted to test it out
by running jobs submitted by different users from my laptop.
My sleep job runs fine as long as the user name is my mac user name.
If I change my hadoop user name by setting HADOOP_USER_NAME,
my jobs fail with the exception org.apache.hadoop.util.Shell$ExitCodeException.
I also tried creating a new user account on my laptop and running a job as that user but I get the same exception.
Please let me know if any of you have come across this.
I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
Here is the stack trace :
Job job_1395389889916_0001 failed with state FAILED due to: Application application_1395389889916_0001 failed 3 times due to AM Container for appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
--
Thanks,
Ashwin
RE: Job fails if I change HADOOP_USER_NAME
Posted by Rohith Sharma K S <ro...@huawei.com>.
Hi
The below stack trace is generic for any am launcher failed to launch. Can debug on AM container logs, so get proper stacktrace.?
Thanks & Regards
Rohith Sharma K S
From: Ashwin Shankar [mailto:ashwinshankar77@gmail.com]
Sent: 21 March 2014 14:02
To: user@hadoop.apache.org
Subject: Job fails if I change HADOOP_USER_NAME
Hi,
I'm writing a new feature in Fair scheduler and wanted to test it out
by running jobs submitted by different users from my laptop.
My sleep job runs fine as long as the user name is my mac user name.
If I change my hadoop user name by setting HADOOP_USER_NAME,
my jobs fail with the exception org.apache.hadoop.util.Shell$ExitCodeException.
I also tried creating a new user account on my laptop and running a job as that user but I get the same exception.
Please let me know if any of you have come across this.
I tried changing ulimits max proc(to 1024),but doesn't solve the problem.
Here is the stack trace :
Job job_1395389889916_0001 failed with state FAILED due to: Application application_1395389889916_0001 failed 3 times due to AM Container for appattempt_1395389889916_0001_000003 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
--
Thanks,
Ashwin