You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Pradeep Singh <ha...@gmail.com> on 2013/08/27 21:07:55 UTC
Creation of symlink from failed
Hi All ,
I am trying to run a simple steaming command as mentioned below .
bin/hadoop jar
/windows/Hadoop/hadoop-1.2.1/contrib/streaming/hadoop-streaming-1.2.1.jar
-input /usr/pradeep/input/'Good words' -output /usr/pradeep/output mapper
/bin/cat reducer /bin/wc -w
I am getting below as message in the error file
packageJobJar: [/tmp/hadoop-root/hadoop-unjar3249316336279467583/] []
/tmp/streamjob1400561611088699684.jar tmpDir=null
13/08/27 23:55:54 INFO util.NativeCodeLoader: Loaded the native-hadoop
library
13/08/27 23:55:54 WARN snappy.LoadSnappy: Snappy native library not loaded
13/08/27 23:55:54 INFO mapred.FileInputFormat: Total input paths to process
: 1
13/08/27 23:55:55 INFO streaming.StreamJob: getLocalDirs():
[/tmp/hadoop-root/mapred/local]
13/08/27 23:55:55 INFO streaming.StreamJob: Running job:
job_201308272335_0004
13/08/27 23:55:55 INFO streaming.StreamJob: To kill this job, run:
13/08/27 23:55:55 INFO streaming.StreamJob:
/windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job
-Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004
13/08/27 23:55:55 INFO streaming.StreamJob: Tracking URL:
http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004
13/08/27 23:55:56 INFO streaming.StreamJob: map 0% reduce 0%
13/08/27 23:56:21 INFO streaming.StreamJob: map 100% reduce 100%
13/08/27 23:56:21 INFO streaming.StreamJob: To kill this job, run:
13/08/27 23:56:21 INFO streaming.StreamJob:
/windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job
-Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004
13/08/27 23:56:21 INFO streaming.StreamJob: Tracking URL:
http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004
13/08/27 23:56:21 ERROR streaming.StreamJob: Job not successful. Error:
JobCleanup Task Failure, Task: task_201308272335_0004_m_000002
13/08/27 23:56:21 INFO streaming.StreamJob: killJob...
Streaming Command Failed!
I checked the log and this is what i found :
MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201308272335_0004_m_000002"
TASK_ATTEMPT_ID="attempt_201308272335_0004_m_000002_0" TASK_STATUS="FAILED"
FINISH_TIME="1377627971824" HOSTNAME="pradeep-pc"
ERROR="java\.lang\.Throwable: Child Error
at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:271)
Caused by: java\.io\.IOException: Creation of symlink from
/windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
to
/tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
failed\.
at
org\.apache\.hadoop\.mapred\.TaskLog\.createTaskAttemptLogDir(TaskLog\.java:110)
at
org\.apache\.hadoop\.mapred\.DefaultTaskController\.createLogDir(DefaultTaskController\.java:71)
at
org\.apache\.hadoop\.mapred\.TaskRunner\.prepareLogFiles(TaskRunner\.java:316)
at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:228)
Cant understand the reason for such error .Everything seems to be fine
.Would appreicate any help in this respect .
Regards
Pradeep Singh
Re: Creation of symlink from failed
Posted by Pradeep Singh <ha...@gmail.com>.
Hi All ,
I Guess the problem might has something to do with the hadoop setting and
permissions related to Symlink creation .
*Creation of symlink from
/windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
to
/tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
failed\.*
Can anybody throw some light on this .
Regards
Pradeep Singh
On Wed, Aug 28, 2013 at 12:37 AM, Pradeep Singh <ha...@gmail.com>wrote:
> Hi All ,
> I am trying to run a simple steaming command as mentioned below .
>
> bin/hadoop jar
> /windows/Hadoop/hadoop-1.2.1/contrib/streaming/hadoop-streaming-1.2.1.jar
> -input /usr/pradeep/input/'Good words' -output /usr/pradeep/output mapper
> /bin/cat reducer /bin/wc -w
>
> I am getting below as message in the error file
>
> packageJobJar: [/tmp/hadoop-root/hadoop-unjar3249316336279467583/] []
> /tmp/streamjob1400561611088699684.jar tmpDir=null
> 13/08/27 23:55:54 INFO util.NativeCodeLoader: Loaded the native-hadoop
> library
> 13/08/27 23:55:54 WARN snappy.LoadSnappy: Snappy native library not loaded
> 13/08/27 23:55:54 INFO mapred.FileInputFormat: Total input paths to
> process : 1
> 13/08/27 23:55:55 INFO streaming.StreamJob: getLocalDirs():
> [/tmp/hadoop-root/mapred/local]
> 13/08/27 23:55:55 INFO streaming.StreamJob: Running job:
> job_201308272335_0004
> 13/08/27 23:55:55 INFO streaming.StreamJob: To kill this job, run:
> 13/08/27 23:55:55 INFO streaming.StreamJob:
> /windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004
> 13/08/27 23:55:55 INFO streaming.StreamJob: Tracking URL:
> http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004
> 13/08/27 23:55:56 INFO streaming.StreamJob: map 0% reduce 0%
> 13/08/27 23:56:21 INFO streaming.StreamJob: map 100% reduce 100%
> 13/08/27 23:56:21 INFO streaming.StreamJob: To kill this job, run:
> 13/08/27 23:56:21 INFO streaming.StreamJob:
> /windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004
> 13/08/27 23:56:21 INFO streaming.StreamJob: Tracking URL:
> http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004
> 13/08/27 23:56:21 ERROR streaming.StreamJob: Job not successful. Error:
> JobCleanup Task Failure, Task: task_201308272335_0004_m_000002
> 13/08/27 23:56:21 INFO streaming.StreamJob: killJob...
> Streaming Command Failed!
>
> I checked the log and this is what i found :
> MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201308272335_0004_m_000002"
> TASK_ATTEMPT_ID="attempt_201308272335_0004_m_000002_0" TASK_STATUS="FAILED"
> FINISH_TIME="1377627971824" HOSTNAME="pradeep-pc"
> ERROR="java\.lang\.Throwable: Child Error
> at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:271)
> Caused by: java\.io\.IOException: Creation of symlink from
> /windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
> to
> /tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
> failed\.
> at
> org\.apache\.hadoop\.mapred\.TaskLog\.createTaskAttemptLogDir(TaskLog\.java:110)
> at
> org\.apache\.hadoop\.mapred\.DefaultTaskController\.createLogDir(DefaultTaskController\.java:71)
> at
> org\.apache\.hadoop\.mapred\.TaskRunner\.prepareLogFiles(TaskRunner\.java:316)
> at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:228)
>
> Cant understand the reason for such error .Everything seems to be fine
> .Would appreicate any help in this respect .
>
> Regards
> Pradeep Singh
>
Re: Creation of symlink from failed
Posted by Pradeep Singh <ha...@gmail.com>.
Hi All ,
I Guess the problem might has something to do with the hadoop setting and
permissions related to Symlink creation .
*Creation of symlink from
/windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
to
/tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
failed\.*
Can anybody throw some light on this .
Regards
Pradeep Singh
On Wed, Aug 28, 2013 at 12:37 AM, Pradeep Singh <ha...@gmail.com>wrote:
> Hi All ,
> I am trying to run a simple steaming command as mentioned below .
>
> bin/hadoop jar
> /windows/Hadoop/hadoop-1.2.1/contrib/streaming/hadoop-streaming-1.2.1.jar
> -input /usr/pradeep/input/'Good words' -output /usr/pradeep/output mapper
> /bin/cat reducer /bin/wc -w
>
> I am getting below as message in the error file
>
> packageJobJar: [/tmp/hadoop-root/hadoop-unjar3249316336279467583/] []
> /tmp/streamjob1400561611088699684.jar tmpDir=null
> 13/08/27 23:55:54 INFO util.NativeCodeLoader: Loaded the native-hadoop
> library
> 13/08/27 23:55:54 WARN snappy.LoadSnappy: Snappy native library not loaded
> 13/08/27 23:55:54 INFO mapred.FileInputFormat: Total input paths to
> process : 1
> 13/08/27 23:55:55 INFO streaming.StreamJob: getLocalDirs():
> [/tmp/hadoop-root/mapred/local]
> 13/08/27 23:55:55 INFO streaming.StreamJob: Running job:
> job_201308272335_0004
> 13/08/27 23:55:55 INFO streaming.StreamJob: To kill this job, run:
> 13/08/27 23:55:55 INFO streaming.StreamJob:
> /windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004
> 13/08/27 23:55:55 INFO streaming.StreamJob: Tracking URL:
> http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004
> 13/08/27 23:55:56 INFO streaming.StreamJob: map 0% reduce 0%
> 13/08/27 23:56:21 INFO streaming.StreamJob: map 100% reduce 100%
> 13/08/27 23:56:21 INFO streaming.StreamJob: To kill this job, run:
> 13/08/27 23:56:21 INFO streaming.StreamJob:
> /windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004
> 13/08/27 23:56:21 INFO streaming.StreamJob: Tracking URL:
> http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004
> 13/08/27 23:56:21 ERROR streaming.StreamJob: Job not successful. Error:
> JobCleanup Task Failure, Task: task_201308272335_0004_m_000002
> 13/08/27 23:56:21 INFO streaming.StreamJob: killJob...
> Streaming Command Failed!
>
> I checked the log and this is what i found :
> MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201308272335_0004_m_000002"
> TASK_ATTEMPT_ID="attempt_201308272335_0004_m_000002_0" TASK_STATUS="FAILED"
> FINISH_TIME="1377627971824" HOSTNAME="pradeep-pc"
> ERROR="java\.lang\.Throwable: Child Error
> at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:271)
> Caused by: java\.io\.IOException: Creation of symlink from
> /windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
> to
> /tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
> failed\.
> at
> org\.apache\.hadoop\.mapred\.TaskLog\.createTaskAttemptLogDir(TaskLog\.java:110)
> at
> org\.apache\.hadoop\.mapred\.DefaultTaskController\.createLogDir(DefaultTaskController\.java:71)
> at
> org\.apache\.hadoop\.mapred\.TaskRunner\.prepareLogFiles(TaskRunner\.java:316)
> at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:228)
>
> Cant understand the reason for such error .Everything seems to be fine
> .Would appreicate any help in this respect .
>
> Regards
> Pradeep Singh
>
Re: Creation of symlink from failed
Posted by Pradeep Singh <ha...@gmail.com>.
Hi All ,
I Guess the problem might has something to do with the hadoop setting and
permissions related to Symlink creation .
*Creation of symlink from
/windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
to
/tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
failed\.*
Can anybody throw some light on this .
Regards
Pradeep Singh
On Wed, Aug 28, 2013 at 12:37 AM, Pradeep Singh <ha...@gmail.com>wrote:
> Hi All ,
> I am trying to run a simple steaming command as mentioned below .
>
> bin/hadoop jar
> /windows/Hadoop/hadoop-1.2.1/contrib/streaming/hadoop-streaming-1.2.1.jar
> -input /usr/pradeep/input/'Good words' -output /usr/pradeep/output mapper
> /bin/cat reducer /bin/wc -w
>
> I am getting below as message in the error file
>
> packageJobJar: [/tmp/hadoop-root/hadoop-unjar3249316336279467583/] []
> /tmp/streamjob1400561611088699684.jar tmpDir=null
> 13/08/27 23:55:54 INFO util.NativeCodeLoader: Loaded the native-hadoop
> library
> 13/08/27 23:55:54 WARN snappy.LoadSnappy: Snappy native library not loaded
> 13/08/27 23:55:54 INFO mapred.FileInputFormat: Total input paths to
> process : 1
> 13/08/27 23:55:55 INFO streaming.StreamJob: getLocalDirs():
> [/tmp/hadoop-root/mapred/local]
> 13/08/27 23:55:55 INFO streaming.StreamJob: Running job:
> job_201308272335_0004
> 13/08/27 23:55:55 INFO streaming.StreamJob: To kill this job, run:
> 13/08/27 23:55:55 INFO streaming.StreamJob:
> /windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004
> 13/08/27 23:55:55 INFO streaming.StreamJob: Tracking URL:
> http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004
> 13/08/27 23:55:56 INFO streaming.StreamJob: map 0% reduce 0%
> 13/08/27 23:56:21 INFO streaming.StreamJob: map 100% reduce 100%
> 13/08/27 23:56:21 INFO streaming.StreamJob: To kill this job, run:
> 13/08/27 23:56:21 INFO streaming.StreamJob:
> /windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004
> 13/08/27 23:56:21 INFO streaming.StreamJob: Tracking URL:
> http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004
> 13/08/27 23:56:21 ERROR streaming.StreamJob: Job not successful. Error:
> JobCleanup Task Failure, Task: task_201308272335_0004_m_000002
> 13/08/27 23:56:21 INFO streaming.StreamJob: killJob...
> Streaming Command Failed!
>
> I checked the log and this is what i found :
> MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201308272335_0004_m_000002"
> TASK_ATTEMPT_ID="attempt_201308272335_0004_m_000002_0" TASK_STATUS="FAILED"
> FINISH_TIME="1377627971824" HOSTNAME="pradeep-pc"
> ERROR="java\.lang\.Throwable: Child Error
> at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:271)
> Caused by: java\.io\.IOException: Creation of symlink from
> /windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
> to
> /tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
> failed\.
> at
> org\.apache\.hadoop\.mapred\.TaskLog\.createTaskAttemptLogDir(TaskLog\.java:110)
> at
> org\.apache\.hadoop\.mapred\.DefaultTaskController\.createLogDir(DefaultTaskController\.java:71)
> at
> org\.apache\.hadoop\.mapred\.TaskRunner\.prepareLogFiles(TaskRunner\.java:316)
> at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:228)
>
> Cant understand the reason for such error .Everything seems to be fine
> .Would appreicate any help in this respect .
>
> Regards
> Pradeep Singh
>
Re: Creation of symlink from failed
Posted by Pradeep Singh <ha...@gmail.com>.
Hi All ,
I Guess the problem might has something to do with the hadoop setting and
permissions related to Symlink creation .
*Creation of symlink from
/windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
to
/tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
failed\.*
Can anybody throw some light on this .
Regards
Pradeep Singh
On Wed, Aug 28, 2013 at 12:37 AM, Pradeep Singh <ha...@gmail.com>wrote:
> Hi All ,
> I am trying to run a simple steaming command as mentioned below .
>
> bin/hadoop jar
> /windows/Hadoop/hadoop-1.2.1/contrib/streaming/hadoop-streaming-1.2.1.jar
> -input /usr/pradeep/input/'Good words' -output /usr/pradeep/output mapper
> /bin/cat reducer /bin/wc -w
>
> I am getting below as message in the error file
>
> packageJobJar: [/tmp/hadoop-root/hadoop-unjar3249316336279467583/] []
> /tmp/streamjob1400561611088699684.jar tmpDir=null
> 13/08/27 23:55:54 INFO util.NativeCodeLoader: Loaded the native-hadoop
> library
> 13/08/27 23:55:54 WARN snappy.LoadSnappy: Snappy native library not loaded
> 13/08/27 23:55:54 INFO mapred.FileInputFormat: Total input paths to
> process : 1
> 13/08/27 23:55:55 INFO streaming.StreamJob: getLocalDirs():
> [/tmp/hadoop-root/mapred/local]
> 13/08/27 23:55:55 INFO streaming.StreamJob: Running job:
> job_201308272335_0004
> 13/08/27 23:55:55 INFO streaming.StreamJob: To kill this job, run:
> 13/08/27 23:55:55 INFO streaming.StreamJob:
> /windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004
> 13/08/27 23:55:55 INFO streaming.StreamJob: Tracking URL:
> http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004
> 13/08/27 23:55:56 INFO streaming.StreamJob: map 0% reduce 0%
> 13/08/27 23:56:21 INFO streaming.StreamJob: map 100% reduce 100%
> 13/08/27 23:56:21 INFO streaming.StreamJob: To kill this job, run:
> 13/08/27 23:56:21 INFO streaming.StreamJob:
> /windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004
> 13/08/27 23:56:21 INFO streaming.StreamJob: Tracking URL:
> http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004
> 13/08/27 23:56:21 ERROR streaming.StreamJob: Job not successful. Error:
> JobCleanup Task Failure, Task: task_201308272335_0004_m_000002
> 13/08/27 23:56:21 INFO streaming.StreamJob: killJob...
> Streaming Command Failed!
>
> I checked the log and this is what i found :
> MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201308272335_0004_m_000002"
> TASK_ATTEMPT_ID="attempt_201308272335_0004_m_000002_0" TASK_STATUS="FAILED"
> FINISH_TIME="1377627971824" HOSTNAME="pradeep-pc"
> ERROR="java\.lang\.Throwable: Child Error
> at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:271)
> Caused by: java\.io\.IOException: Creation of symlink from
> /windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
> to
> /tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
> failed\.
> at
> org\.apache\.hadoop\.mapred\.TaskLog\.createTaskAttemptLogDir(TaskLog\.java:110)
> at
> org\.apache\.hadoop\.mapred\.DefaultTaskController\.createLogDir(DefaultTaskController\.java:71)
> at
> org\.apache\.hadoop\.mapred\.TaskRunner\.prepareLogFiles(TaskRunner\.java:316)
> at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:228)
>
> Cant understand the reason for such error .Everything seems to be fine
> .Would appreicate any help in this respect .
>
> Regards
> Pradeep Singh
>