You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by "Li, Yonggang" <yo...@hp.com> on 2011/11/04 04:11:54 UTC
someone know how to install hadoop0.20 on hp-ux?
I have installed a hadoop and try to start it with start-all.sh .it hint me : localhost: No such file or directory: bash
Then I modified shell script and re-started successfully .but when I run wordcount example ,it report job fail.
Below is log msg:
adoop@cmcbcsns:/opt/ecip/BMC/hadoopTest/hadoop-0.20.203.0>./bin/hadoop jar hadoop-examples-0.20.203.0.jar wordcount input output
11/11/04 10:47:32 INFO input.FileInputFormat: Total input paths to process : 1
11/11/04 10:47:33 INFO mapred.JobClient: Running job: job_201111041042_0001
11/11/04 10:47:34 INFO mapred.JobClient: map 0% reduce 0%
11/11/04 10:47:42 INFO mapred.JobClient: Task Id : attempt_201111041042_0001_m_000002_0, Status : FAILED
11/11/04 10:47:43 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000002_0&filter=stdout
11/11/04 10:47:43 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000002_0&filter=stderr
11/11/04 10:47:48 INFO mapred.JobClient: Task Id : attempt_201111041042_0001_m_000002_1, Status : FAILED
11/11/04 10:47:48 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000002_1&filter=stdout
11/11/04 10:47:48 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000002_1&filter=stderr
11/11/04 10:47:54 INFO mapred.JobClient: Task Id : attempt_201111041042_0001_m_000002_2, Status : FAILED
11/11/04 10:47:54 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000002_2&filter=stdout
11/11/04 10:47:54 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000002_2&filter=stderr
11/11/04 10:48:05 INFO mapred.JobClient: Task Id : attempt_201111041042_0001_m_000001_0, Status : FAILED
11/11/04 10:48:05 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000001_0&filter=stdout
11/11/04 10:48:06 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000001_0&filter=stderr
2011-11-04 10:42:48,180 INFO org.apache.hadoop.mapred.TaskTracker: FILE_CACHE_SIZE for mapOutputServlet set to : 2000
2011-11-04 10:47:35,653 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201111041042_0001_m_0000
02_0 task's state:UNASSIGNED
2011-11-04 10:47:35,664 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201111041042_0001_m_000002_0 which nee
ds 1 slots
2011-11-04 10:47:35,665 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch atte
mpt_201111041042_0001_m_000002_0 which needs 1 slots
2011-11-04 10:47:36,451 INFO org.apache.hadoop.mapred.JobLocalizer: Initializing user hadoop on this TT.
2011-11-04 10:47:37,239 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201111041042_0001_m_-10927504
18
2011-11-04 10:47:37,239 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201111041042_0001_m_-1092750418 spawned.
2011-11-04 10:47:37,254 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /opt/ecip/BMC/mapred/local/ttprivate/taskT
racker/hadoop/jobcache/job_201111041042_0001/attempt_201111041042_0001_m_000002_0/taskjvm.sh
2011-11-04 10:47:37,262 WARN org.apache.hadoop.mapred.DefaultTaskController: Exit code from task is : 0
2011-11-04 10:47:37,263 INFO org.apache.hadoop.mapred.DefaultTaskController: Output from DefaultTaskController's launchTask follows:
2011-11-04 10:47:37,263 INFO org.apache.hadoop.mapred.TaskController:
2011-11-04 10:47:37,264 INFO org.apache.hadoop.mapred.JvmManager: JVM Not killed jvm_201111041042_0001_m_-1092750418 but just remove
d
2011-11-04 10:47:37,264 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201111041042_0001_m_-1092750418 exited with exit code 0.
Number of tasks it ran: 0
2011-11-04 10:47:40,304 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
2011-11-04 10:47:41,712 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201111041042_0001_m_0000
02_1 task's state:UNASSIGNED
2011-11-04 10:47:41,713 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201111041042_0001_m_000002_1 which nee
ds 1 slots
2011-11-04 10:47:41,714 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch atte
mpt_201111041042_0001_m_000002_1 which needs 1 slots
2011-11-04 10:47:41,779 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201111041042_0001_m_169541285
2011-11-04 10:47:41,780 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201111041042_0001_m_169541285 spawned.
2011-11-04 10:47:41,817 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /opt/ecip/BMC/mapred/local/ttprivate/taskT
racker/hadoop/jobcache/job_201111041042_0001/attempt_201111041042_0001_m_000002_1/taskjvm.sh
2011-11-04 10:47:41,829 WARN org.apache.hadoop.mapred.DefaultTaskController: Exit code from task is : 0
2011-11-04 10:47:41,829 INFO org.apache.hadoop.mapred.DefaultTaskController: Output from DefaultTaskController's launchTask follows:
2011-11-04 10:47:41,829 INFO org.apache.hadoop.mapred.TaskController:
2011-11-04 10:47:41,830 INFO org.apache.hadoop.mapred.JvmManager: JVM Not killed jvm_201111041042_0001_m_169541285 but just removed
2011-11-04 10:47:41,830 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201111041042_0001_m_169541285 exited with exit code 0. N
umber of tasks it ran: 0
2011-11-04 10:47:43,094 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... usin
g builtin-java classes where applicable
2011-11-04 10:47:43,165 WARN org.apache.hadoop.mapred.TaskLog: Failed to retrieve stdout log for task: attempt_201111041042_0001_m_0
00002_0
java.io.FileNotFoundException: /opt/ecip/BMC/hadoopTest/hadoop-0.20.203.0/bin/../logs/userlogs/job_201111041042_0001/attempt_2011110
41042_0001_m_000002_0/log.index (No such file or directory (errno:2))
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:106)
Best Regards
Yonggang Li
Re: someone know how to install hadoop0.20 on hp-ux?
Posted by Joey Echeverria <jo...@cloudera.com>.
You need to create a log directory on your TaskTracker nodes:
/opt/ecip/BMC/hadoopTest/hadoop-0.20.203.0/logs/
Make sure the directory is writable by the mapred user, or which ever
user your TaskTrackers were started as.
-Joey
On Thu, Nov 3, 2011 at 11:11 PM, Li, Yonggang <yo...@hp.com> wrote:
>
> I have installed a hadoop and try to start it with start-all.sh .it hint me : localhost: No such file or directory: bash
> Then I modified shell script and re-started successfully .but when I run wordcount example ,it report job fail.
> Below is log msg:
> adoop@cmcbcsns:/opt/ecip/BMC/hadoopTest/hadoop-0.20.203.0>./bin/hadoop jar hadoop-examples-0.20.203.0.jar wordcount input output
> 11/11/04 10:47:32 INFO input.FileInputFormat: Total input paths to process : 1
> 11/11/04 10:47:33 INFO mapred.JobClient: Running job: job_201111041042_0001
> 11/11/04 10:47:34 INFO mapred.JobClient: map 0% reduce 0%
> 11/11/04 10:47:42 INFO mapred.JobClient: Task Id : attempt_201111041042_0001_m_000002_0, Status : FAILED
> 11/11/04 10:47:43 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000002_0&filter=stdout
> 11/11/04 10:47:43 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000002_0&filter=stderr
> 11/11/04 10:47:48 INFO mapred.JobClient: Task Id : attempt_201111041042_0001_m_000002_1, Status : FAILED
> 11/11/04 10:47:48 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000002_1&filter=stdout
> 11/11/04 10:47:48 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000002_1&filter=stderr
> 11/11/04 10:47:54 INFO mapred.JobClient: Task Id : attempt_201111041042_0001_m_000002_2, Status : FAILED
> 11/11/04 10:47:54 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000002_2&filter=stdout
> 11/11/04 10:47:54 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000002_2&filter=stderr
> 11/11/04 10:48:05 INFO mapred.JobClient: Task Id : attempt_201111041042_0001_m_000001_0, Status : FAILED
> 11/11/04 10:48:05 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000001_0&filter=stdout
> 11/11/04 10:48:06 WARN mapred.JobClient: Error reading task outputhttp://cmcbcsns:50060/tasklog?plaintext=true&attemptid=attempt_201111041042_0001_m_000001_0&filter=stderr
>
>
> 2011-11-04 10:42:48,180 INFO org.apache.hadoop.mapred.TaskTracker: FILE_CACHE_SIZE for mapOutputServlet set to : 2000
> 2011-11-04 10:47:35,653 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201111041042_0001_m_0000
> 02_0 task's state:UNASSIGNED
> 2011-11-04 10:47:35,664 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201111041042_0001_m_000002_0 which nee
> ds 1 slots
> 2011-11-04 10:47:35,665 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch atte
> mpt_201111041042_0001_m_000002_0 which needs 1 slots
> 2011-11-04 10:47:36,451 INFO org.apache.hadoop.mapred.JobLocalizer: Initializing user hadoop on this TT.
> 2011-11-04 10:47:37,239 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201111041042_0001_m_-10927504
> 18
> 2011-11-04 10:47:37,239 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201111041042_0001_m_-1092750418 spawned.
> 2011-11-04 10:47:37,254 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /opt/ecip/BMC/mapred/local/ttprivate/taskT
> racker/hadoop/jobcache/job_201111041042_0001/attempt_201111041042_0001_m_000002_0/taskjvm.sh
> 2011-11-04 10:47:37,262 WARN org.apache.hadoop.mapred.DefaultTaskController: Exit code from task is : 0
> 2011-11-04 10:47:37,263 INFO org.apache.hadoop.mapred.DefaultTaskController: Output from DefaultTaskController's launchTask follows:
> 2011-11-04 10:47:37,263 INFO org.apache.hadoop.mapred.TaskController:
> 2011-11-04 10:47:37,264 INFO org.apache.hadoop.mapred.JvmManager: JVM Not killed jvm_201111041042_0001_m_-1092750418 but just remove
> d
> 2011-11-04 10:47:37,264 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201111041042_0001_m_-1092750418 exited with exit code 0.
> Number of tasks it ran: 0
> 2011-11-04 10:47:40,304 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
> 2011-11-04 10:47:41,712 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201111041042_0001_m_0000
> 02_1 task's state:UNASSIGNED
> 2011-11-04 10:47:41,713 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201111041042_0001_m_000002_1 which nee
> ds 1 slots
> 2011-11-04 10:47:41,714 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch atte
> mpt_201111041042_0001_m_000002_1 which needs 1 slots
> 2011-11-04 10:47:41,779 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201111041042_0001_m_169541285
> 2011-11-04 10:47:41,780 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201111041042_0001_m_169541285 spawned.
> 2011-11-04 10:47:41,817 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /opt/ecip/BMC/mapred/local/ttprivate/taskT
> racker/hadoop/jobcache/job_201111041042_0001/attempt_201111041042_0001_m_000002_1/taskjvm.sh
> 2011-11-04 10:47:41,829 WARN org.apache.hadoop.mapred.DefaultTaskController: Exit code from task is : 0
> 2011-11-04 10:47:41,829 INFO org.apache.hadoop.mapred.DefaultTaskController: Output from DefaultTaskController's launchTask follows:
> 2011-11-04 10:47:41,829 INFO org.apache.hadoop.mapred.TaskController:
> 2011-11-04 10:47:41,830 INFO org.apache.hadoop.mapred.JvmManager: JVM Not killed jvm_201111041042_0001_m_169541285 but just removed
> 2011-11-04 10:47:41,830 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201111041042_0001_m_169541285 exited with exit code 0. N
> umber of tasks it ran: 0
> 2011-11-04 10:47:43,094 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... usin
> g builtin-java classes where applicable
> 2011-11-04 10:47:43,165 WARN org.apache.hadoop.mapred.TaskLog: Failed to retrieve stdout log for task: attempt_201111041042_0001_m_0
> 00002_0
> java.io.FileNotFoundException: /opt/ecip/BMC/hadoopTest/hadoop-0.20.203.0/bin/../logs/userlogs/job_201111041042_0001/attempt_2011110
> 41042_0001_m_000002_0/log.index (No such file or directory (errno:2))
> at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.<init>(FileInputStream.java:106)
> Best Regards
>
> Yonggang Li
>
>
--
Joseph Echeverria
Cloudera, Inc.
443.305.9434