You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Pierre-Francois Laquerre <pi...@nec-labs.com> on 2013/08/01 16:57:26 UTC
incorrect staging area path in 2.0.5
I recently updated from 1.0.4 to 2.0.5. Since then, streaming jobs have
been failing to launch due to what seems like an incorrect staging path:
# /opt/hadoop2/bin/hadoop jar
/opt/hadoop2/share/hadoop/tools/lib/hadoop-streaming-2.0.5-alpha.jar
-input foo -output bar -mapper baz -reducer foobar
13/08/01 10:43:50 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes
where applicable
13/08/01 10:43:50 WARN conf.Configuration: session.id is deprecated.
Instead, use dfs.metrics.session-id
13/08/01 10:43:50 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
13/08/01 10:43:50 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics
with processName=JobTracker, sessionId= - already initialized
13/08/01 10:43:50 INFO mapreduce.JobSubmitter: Cleaning up the staging
area *file:/user/myuser1544460269/.staging/job_local1544460269_0001*
13/08/01 10:43:50 ERROR security.UserGroupInformation:
PriviledgedActionException as:myuser (auth:SIMPLE)
cause:org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot
access `*/user/myuser1544460269/.staging/job_local1544460269_0001*': No
such file or directory
13/08/01 10:43:50 ERROR streaming.StreamJob: Error Launching job :
chmod: cannot access
`/user/myuser1544460269/.staging/job_local1544460269_0001': No such file
or directory
Streaming Command Failed!
This is for a job launched as "myuser". Given that
mapreduce.jobtracker.staging.root.dir is set to /user, I would expect
the staging area to be in /user/myuser/.staging/job_local$jobid.
Instead, it is in /user/myuser$jobid/.staging/job_local$jobid, which
fails since /user/myuser$jobid/ doesn't exist. Has the way
staging.root.dir is used changed in 2.x?
Thank you,
Pierre
Re: incorrect staging area path in 2.0.5
Posted by Pierre-Francois Laquerre <pi...@nec-labs.com>.
Good catch. I hadn't noticed the file:/ instead of hdfs:/. Setting the
framework to yarn got rid of the problem.
Thanks,
Pierre
On 08/01/2013 11:08 AM, Devaraj k wrote:
>
> Hi Pierre,
>
> As per the below information, we see Job is running in local mode and
> trying to use the local file system for staging dir. Could you please
> configure 'mapreduce.framework.name' & 'fs.defaultFS' and check.
>
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html
>
> Thanks
>
> Devaraj k
>
> *From:*Pierre-Francois Laquerre [mailto:pierre.francois@nec-labs.com]
> *Sent:* 01 August 2013 22:57
> *To:* user@hadoop.apache.org
> *Subject:* incorrect staging area path in 2.0.5
>
> I recently updated from 1.0.4 to 2.0.5. Since then, streaming jobs
> have been failing to launch due to what seems like an incorrect
> staging path:
>
> # /opt/hadoop2/bin/hadoop jar
> /opt/hadoop2/share/hadoop/tools/lib/hadoop-streaming-2.0.5-alpha.jar
> -input foo -output bar -mapper baz -reducer foobar
> 13/08/01 10:43:50 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> 13/08/01 10:43:50 WARN conf.Configuration: session.id is deprecated.
> Instead, use dfs.metrics.session-id
> 13/08/01 10:43:50 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId=
> 13/08/01 10:43:50 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics
> with processName=JobTracker, sessionId= - already initialized
> 13/08/01 10:43:50 INFO mapreduce.JobSubmitter: Cleaning up the staging
> area *file:/user/myuser1544460269/.staging/job_local1544460269_0001
> <file:///%5C%5Cuser%5Cmyuser1544460269%5C.staging%5Cjob_local1544460269_0001>*
> 13/08/01 10:43:50 ERROR security.UserGroupInformation:
> PriviledgedActionException as:myuser (auth:SIMPLE)
> cause:org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot
> access `*/user/myuser1544460269/.staging/job_local1544460269_0001*':
> No such file or directory
> 13/08/01 10:43:50 ERROR streaming.StreamJob: Error Launching job :
> chmod: cannot access
> `/user/myuser1544460269/.staging/job_local1544460269_0001': No such
> file or directory
> Streaming Command Failed!
>
> This is for a job launched as "myuser". Given that
> mapreduce.jobtracker.staging.root.dir is set to /user, I would expect
> the staging area to be in /user/myuser/.staging/job_local$jobid.
> Instead, it is in /user/myuser$jobid/.staging/job_local$jobid, which
> fails since /user/myuser$jobid/ doesn't exist. Has the way
> staging.root.dir is used changed in 2.x?
>
> Thank you,
>
> Pierre
>
Re: incorrect staging area path in 2.0.5
Posted by Pierre-Francois Laquerre <pi...@nec-labs.com>.
Good catch. I hadn't noticed the file:/ instead of hdfs:/. Setting the
framework to yarn got rid of the problem.
Thanks,
Pierre
On 08/01/2013 11:08 AM, Devaraj k wrote:
>
> Hi Pierre,
>
> As per the below information, we see Job is running in local mode and
> trying to use the local file system for staging dir. Could you please
> configure 'mapreduce.framework.name' & 'fs.defaultFS' and check.
>
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html
>
> Thanks
>
> Devaraj k
>
> *From:*Pierre-Francois Laquerre [mailto:pierre.francois@nec-labs.com]
> *Sent:* 01 August 2013 22:57
> *To:* user@hadoop.apache.org
> *Subject:* incorrect staging area path in 2.0.5
>
> I recently updated from 1.0.4 to 2.0.5. Since then, streaming jobs
> have been failing to launch due to what seems like an incorrect
> staging path:
>
> # /opt/hadoop2/bin/hadoop jar
> /opt/hadoop2/share/hadoop/tools/lib/hadoop-streaming-2.0.5-alpha.jar
> -input foo -output bar -mapper baz -reducer foobar
> 13/08/01 10:43:50 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> 13/08/01 10:43:50 WARN conf.Configuration: session.id is deprecated.
> Instead, use dfs.metrics.session-id
> 13/08/01 10:43:50 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId=
> 13/08/01 10:43:50 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics
> with processName=JobTracker, sessionId= - already initialized
> 13/08/01 10:43:50 INFO mapreduce.JobSubmitter: Cleaning up the staging
> area *file:/user/myuser1544460269/.staging/job_local1544460269_0001
> <file:///%5C%5Cuser%5Cmyuser1544460269%5C.staging%5Cjob_local1544460269_0001>*
> 13/08/01 10:43:50 ERROR security.UserGroupInformation:
> PriviledgedActionException as:myuser (auth:SIMPLE)
> cause:org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot
> access `*/user/myuser1544460269/.staging/job_local1544460269_0001*':
> No such file or directory
> 13/08/01 10:43:50 ERROR streaming.StreamJob: Error Launching job :
> chmod: cannot access
> `/user/myuser1544460269/.staging/job_local1544460269_0001': No such
> file or directory
> Streaming Command Failed!
>
> This is for a job launched as "myuser". Given that
> mapreduce.jobtracker.staging.root.dir is set to /user, I would expect
> the staging area to be in /user/myuser/.staging/job_local$jobid.
> Instead, it is in /user/myuser$jobid/.staging/job_local$jobid, which
> fails since /user/myuser$jobid/ doesn't exist. Has the way
> staging.root.dir is used changed in 2.x?
>
> Thank you,
>
> Pierre
>
Re: incorrect staging area path in 2.0.5
Posted by Pierre-Francois Laquerre <pi...@nec-labs.com>.
Good catch. I hadn't noticed the file:/ instead of hdfs:/. Setting the
framework to yarn got rid of the problem.
Thanks,
Pierre
On 08/01/2013 11:08 AM, Devaraj k wrote:
>
> Hi Pierre,
>
> As per the below information, we see Job is running in local mode and
> trying to use the local file system for staging dir. Could you please
> configure 'mapreduce.framework.name' & 'fs.defaultFS' and check.
>
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html
>
> Thanks
>
> Devaraj k
>
> *From:*Pierre-Francois Laquerre [mailto:pierre.francois@nec-labs.com]
> *Sent:* 01 August 2013 22:57
> *To:* user@hadoop.apache.org
> *Subject:* incorrect staging area path in 2.0.5
>
> I recently updated from 1.0.4 to 2.0.5. Since then, streaming jobs
> have been failing to launch due to what seems like an incorrect
> staging path:
>
> # /opt/hadoop2/bin/hadoop jar
> /opt/hadoop2/share/hadoop/tools/lib/hadoop-streaming-2.0.5-alpha.jar
> -input foo -output bar -mapper baz -reducer foobar
> 13/08/01 10:43:50 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> 13/08/01 10:43:50 WARN conf.Configuration: session.id is deprecated.
> Instead, use dfs.metrics.session-id
> 13/08/01 10:43:50 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId=
> 13/08/01 10:43:50 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics
> with processName=JobTracker, sessionId= - already initialized
> 13/08/01 10:43:50 INFO mapreduce.JobSubmitter: Cleaning up the staging
> area *file:/user/myuser1544460269/.staging/job_local1544460269_0001
> <file:///%5C%5Cuser%5Cmyuser1544460269%5C.staging%5Cjob_local1544460269_0001>*
> 13/08/01 10:43:50 ERROR security.UserGroupInformation:
> PriviledgedActionException as:myuser (auth:SIMPLE)
> cause:org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot
> access `*/user/myuser1544460269/.staging/job_local1544460269_0001*':
> No such file or directory
> 13/08/01 10:43:50 ERROR streaming.StreamJob: Error Launching job :
> chmod: cannot access
> `/user/myuser1544460269/.staging/job_local1544460269_0001': No such
> file or directory
> Streaming Command Failed!
>
> This is for a job launched as "myuser". Given that
> mapreduce.jobtracker.staging.root.dir is set to /user, I would expect
> the staging area to be in /user/myuser/.staging/job_local$jobid.
> Instead, it is in /user/myuser$jobid/.staging/job_local$jobid, which
> fails since /user/myuser$jobid/ doesn't exist. Has the way
> staging.root.dir is used changed in 2.x?
>
> Thank you,
>
> Pierre
>
Re: incorrect staging area path in 2.0.5
Posted by Pierre-Francois Laquerre <pi...@nec-labs.com>.
Good catch. I hadn't noticed the file:/ instead of hdfs:/. Setting the
framework to yarn got rid of the problem.
Thanks,
Pierre
On 08/01/2013 11:08 AM, Devaraj k wrote:
>
> Hi Pierre,
>
> As per the below information, we see Job is running in local mode and
> trying to use the local file system for staging dir. Could you please
> configure 'mapreduce.framework.name' & 'fs.defaultFS' and check.
>
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html
>
> Thanks
>
> Devaraj k
>
> *From:*Pierre-Francois Laquerre [mailto:pierre.francois@nec-labs.com]
> *Sent:* 01 August 2013 22:57
> *To:* user@hadoop.apache.org
> *Subject:* incorrect staging area path in 2.0.5
>
> I recently updated from 1.0.4 to 2.0.5. Since then, streaming jobs
> have been failing to launch due to what seems like an incorrect
> staging path:
>
> # /opt/hadoop2/bin/hadoop jar
> /opt/hadoop2/share/hadoop/tools/lib/hadoop-streaming-2.0.5-alpha.jar
> -input foo -output bar -mapper baz -reducer foobar
> 13/08/01 10:43:50 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> 13/08/01 10:43:50 WARN conf.Configuration: session.id is deprecated.
> Instead, use dfs.metrics.session-id
> 13/08/01 10:43:50 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId=
> 13/08/01 10:43:50 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics
> with processName=JobTracker, sessionId= - already initialized
> 13/08/01 10:43:50 INFO mapreduce.JobSubmitter: Cleaning up the staging
> area *file:/user/myuser1544460269/.staging/job_local1544460269_0001
> <file:///%5C%5Cuser%5Cmyuser1544460269%5C.staging%5Cjob_local1544460269_0001>*
> 13/08/01 10:43:50 ERROR security.UserGroupInformation:
> PriviledgedActionException as:myuser (auth:SIMPLE)
> cause:org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot
> access `*/user/myuser1544460269/.staging/job_local1544460269_0001*':
> No such file or directory
> 13/08/01 10:43:50 ERROR streaming.StreamJob: Error Launching job :
> chmod: cannot access
> `/user/myuser1544460269/.staging/job_local1544460269_0001': No such
> file or directory
> Streaming Command Failed!
>
> This is for a job launched as "myuser". Given that
> mapreduce.jobtracker.staging.root.dir is set to /user, I would expect
> the staging area to be in /user/myuser/.staging/job_local$jobid.
> Instead, it is in /user/myuser$jobid/.staging/job_local$jobid, which
> fails since /user/myuser$jobid/ doesn't exist. Has the way
> staging.root.dir is used changed in 2.x?
>
> Thank you,
>
> Pierre
>
RE: incorrect staging area path in 2.0.5
Posted by Devaraj k <de...@huawei.com>.
Hi Pierre,
As per the below information, we see Job is running in local mode and trying to use the local file system for staging dir. Could you please configure 'mapreduce.framework.name' & 'fs.defaultFS' and check.
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html
Thanks
Devaraj k
From: Pierre-Francois Laquerre [mailto:pierre.francois@nec-labs.com]
Sent: 01 August 2013 22:57
To: user@hadoop.apache.org
Subject: incorrect staging area path in 2.0.5
I recently updated from 1.0.4 to 2.0.5. Since then, streaming jobs have been failing to launch due to what seems like an incorrect staging path:
# /opt/hadoop2/bin/hadoop jar /opt/hadoop2/share/hadoop/tools/lib/hadoop-streaming-2.0.5-alpha.jar -input foo -output bar -mapper baz -reducer foobar
13/08/01 10:43:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/08/01 10:43:50 WARN conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id
13/08/01 10:43:50 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
13/08/01 10:43:50 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
13/08/01 10:43:50 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/user/myuser1544460269/.staging/job_local1544460269_0001<file:///\\user\myuser1544460269\.staging\job_local1544460269_0001>
13/08/01 10:43:50 ERROR security.UserGroupInformation: PriviledgedActionException as:myuser (auth:SIMPLE) cause:org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access `/user/myuser1544460269/.staging/job_local1544460269_0001': No such file or directory
13/08/01 10:43:50 ERROR streaming.StreamJob: Error Launching job : chmod: cannot access `/user/myuser1544460269/.staging/job_local1544460269_0001': No such file or directory
Streaming Command Failed!
This is for a job launched as "myuser". Given that mapreduce.jobtracker.staging.root.dir is set to /user, I would expect the staging area to be in /user/myuser/.staging/job_local$jobid. Instead, it is in /user/myuser$jobid/.staging/job_local$jobid, which fails since /user/myuser$jobid/ doesn't exist. Has the way staging.root.dir is used changed in 2.x?
Thank you,
Pierre
RE: incorrect staging area path in 2.0.5
Posted by Devaraj k <de...@huawei.com>.
Hi Pierre,
As per the below information, we see Job is running in local mode and trying to use the local file system for staging dir. Could you please configure 'mapreduce.framework.name' & 'fs.defaultFS' and check.
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html
Thanks
Devaraj k
From: Pierre-Francois Laquerre [mailto:pierre.francois@nec-labs.com]
Sent: 01 August 2013 22:57
To: user@hadoop.apache.org
Subject: incorrect staging area path in 2.0.5
I recently updated from 1.0.4 to 2.0.5. Since then, streaming jobs have been failing to launch due to what seems like an incorrect staging path:
# /opt/hadoop2/bin/hadoop jar /opt/hadoop2/share/hadoop/tools/lib/hadoop-streaming-2.0.5-alpha.jar -input foo -output bar -mapper baz -reducer foobar
13/08/01 10:43:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/08/01 10:43:50 WARN conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id
13/08/01 10:43:50 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
13/08/01 10:43:50 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
13/08/01 10:43:50 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/user/myuser1544460269/.staging/job_local1544460269_0001<file:///\\user\myuser1544460269\.staging\job_local1544460269_0001>
13/08/01 10:43:50 ERROR security.UserGroupInformation: PriviledgedActionException as:myuser (auth:SIMPLE) cause:org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access `/user/myuser1544460269/.staging/job_local1544460269_0001': No such file or directory
13/08/01 10:43:50 ERROR streaming.StreamJob: Error Launching job : chmod: cannot access `/user/myuser1544460269/.staging/job_local1544460269_0001': No such file or directory
Streaming Command Failed!
This is for a job launched as "myuser". Given that mapreduce.jobtracker.staging.root.dir is set to /user, I would expect the staging area to be in /user/myuser/.staging/job_local$jobid. Instead, it is in /user/myuser$jobid/.staging/job_local$jobid, which fails since /user/myuser$jobid/ doesn't exist. Has the way staging.root.dir is used changed in 2.x?
Thank you,
Pierre
RE: incorrect staging area path in 2.0.5
Posted by Devaraj k <de...@huawei.com>.
Hi Pierre,
As per the below information, we see Job is running in local mode and trying to use the local file system for staging dir. Could you please configure 'mapreduce.framework.name' & 'fs.defaultFS' and check.
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html
Thanks
Devaraj k
From: Pierre-Francois Laquerre [mailto:pierre.francois@nec-labs.com]
Sent: 01 August 2013 22:57
To: user@hadoop.apache.org
Subject: incorrect staging area path in 2.0.5
I recently updated from 1.0.4 to 2.0.5. Since then, streaming jobs have been failing to launch due to what seems like an incorrect staging path:
# /opt/hadoop2/bin/hadoop jar /opt/hadoop2/share/hadoop/tools/lib/hadoop-streaming-2.0.5-alpha.jar -input foo -output bar -mapper baz -reducer foobar
13/08/01 10:43:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/08/01 10:43:50 WARN conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id
13/08/01 10:43:50 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
13/08/01 10:43:50 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
13/08/01 10:43:50 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/user/myuser1544460269/.staging/job_local1544460269_0001<file:///\\user\myuser1544460269\.staging\job_local1544460269_0001>
13/08/01 10:43:50 ERROR security.UserGroupInformation: PriviledgedActionException as:myuser (auth:SIMPLE) cause:org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access `/user/myuser1544460269/.staging/job_local1544460269_0001': No such file or directory
13/08/01 10:43:50 ERROR streaming.StreamJob: Error Launching job : chmod: cannot access `/user/myuser1544460269/.staging/job_local1544460269_0001': No such file or directory
Streaming Command Failed!
This is for a job launched as "myuser". Given that mapreduce.jobtracker.staging.root.dir is set to /user, I would expect the staging area to be in /user/myuser/.staging/job_local$jobid. Instead, it is in /user/myuser$jobid/.staging/job_local$jobid, which fails since /user/myuser$jobid/ doesn't exist. Has the way staging.root.dir is used changed in 2.x?
Thank you,
Pierre
RE: incorrect staging area path in 2.0.5
Posted by Devaraj k <de...@huawei.com>.
Hi Pierre,
As per the below information, we see Job is running in local mode and trying to use the local file system for staging dir. Could you please configure 'mapreduce.framework.name' & 'fs.defaultFS' and check.
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html
Thanks
Devaraj k
From: Pierre-Francois Laquerre [mailto:pierre.francois@nec-labs.com]
Sent: 01 August 2013 22:57
To: user@hadoop.apache.org
Subject: incorrect staging area path in 2.0.5
I recently updated from 1.0.4 to 2.0.5. Since then, streaming jobs have been failing to launch due to what seems like an incorrect staging path:
# /opt/hadoop2/bin/hadoop jar /opt/hadoop2/share/hadoop/tools/lib/hadoop-streaming-2.0.5-alpha.jar -input foo -output bar -mapper baz -reducer foobar
13/08/01 10:43:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/08/01 10:43:50 WARN conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id
13/08/01 10:43:50 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
13/08/01 10:43:50 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
13/08/01 10:43:50 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/user/myuser1544460269/.staging/job_local1544460269_0001<file:///\\user\myuser1544460269\.staging\job_local1544460269_0001>
13/08/01 10:43:50 ERROR security.UserGroupInformation: PriviledgedActionException as:myuser (auth:SIMPLE) cause:org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access `/user/myuser1544460269/.staging/job_local1544460269_0001': No such file or directory
13/08/01 10:43:50 ERROR streaming.StreamJob: Error Launching job : chmod: cannot access `/user/myuser1544460269/.staging/job_local1544460269_0001': No such file or directory
Streaming Command Failed!
This is for a job launched as "myuser". Given that mapreduce.jobtracker.staging.root.dir is set to /user, I would expect the staging area to be in /user/myuser/.staging/job_local$jobid. Instead, it is in /user/myuser$jobid/.staging/job_local$jobid, which fails since /user/myuser$jobid/ doesn't exist. Has the way staging.root.dir is used changed in 2.x?
Thank you,
Pierre