You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by "Yanzan Wu (JIRA)" <ji...@apache.org> on 2012/07/23 09:29:34 UTC
[jira] [Created] (MESOS-238) Trouble running hadoop job on mesos
Yanzan Wu created MESOS-238:
-------------------------------
Summary: Trouble running hadoop job on mesos
Key: MESOS-238
URL: https://issues.apache.org/jira/browse/MESOS-238
Project: Mesos
Issue Type: Bug
Environment: CentOS
Reporter: Yanzan Wu
I was trying to run a hadoop wordcount job on mesos which runs on a single node,here comes the problem:
The running job stays always map 0% reduce 0%
./bin/hadoop jar hadoop-examples-0.20.205.0.jar wordcount
contrib/mesos/src/java/org/apache/hadoop/mapred/ out
12/07/23 12:11:57 INFO input.FileInputFormat: Total input paths to process : 5
12/07/23 12:11:58 INFO mapred.JobClient: Running job: job_201207231207_0002
12/07/23 12:11:59 INFO mapred.JobClient: map 0% reduce 0%
The jobtracker process get the printout as follows:
12/07/23 15:44:05 INFO mapred.FrameworkScheduler: Going to launch map task for setup / cleanup
12/07/23 15:44:05 INFO mapred.FrameworkScheduler: Launching Mesos task 10334 as map on node2.massivedata
12/07/23 15:44:06 INFO mapred.FrameworkScheduler: Task 10334 is TASK_LOST
12/07/23 15:44:06 INFO mapred.FrameworkScheduler: Got resource offer value: "201207231205-1996531904-5050-2701-16821"
12/07/23 15:44:06 INFO mapred.FrameworkScheduler: Going to launch map task for setup / cleanup
12/07/23 15:44:06 INFO mapred.FrameworkScheduler: Launching Mesos task 10335 as map on node2.massivedata
12/07/23 15:44:06 INFO mapred.FrameworkScheduler: Got resource offer value: "201207231205-1996531904-5050-2701-16822"
12/07/23 15:44:07 INFO mapred.FrameworkScheduler: Task 10335 is TASK_LOST
12/07/23 15:44:07 INFO mapred.FrameworkScheduler: Killing tasks that started 20000 milliseconds ago
12/07/23 15:44:07 INFO mapred.FrameworkScheduler: Got resource offer value: "201207231205-1996531904-5050-2701-16823"
12/07/23 15:44:07 INFO mapred.FrameworkScheduler: Going to launch map task for setup / cleanup
12/07/23 15:44:07 INFO mapred.FrameworkScheduler: Launching Mesos task 10336 as map on node2.massivedata
12/07/23 15:44:08 INFO mapred.FrameworkScheduler: Task 10336 is TASK_LOST
12/07/23 15:44:08 INFO mapred.FrameworkScheduler: Got resource offer value: "201207231205-1996531904-5050-2701-16824"
As it show,it repeats getting resource offer while can't launch task!
The jps can make these result:
4283 Jps
3000 NameNode
3417 JobTracker
3140 DataNode
16167 RunJar
3293 SecondaryNameNode
What's the problem?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-238) Trouble running hadoop job on mesos
Posted by "Keith Wyss (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444149#comment-13444149 ]
Keith Wyss commented on MESOS-238:
----------------------------------
This is something that I'm seeing as well. I think it is on the hadoop end, as I'm able to run Spark code successfully. Are you running the CDH3U3 build rather than the one packaged with mesos?
> Trouble running hadoop job on mesos
> -----------------------------------
>
> Key: MESOS-238
> URL: https://issues.apache.org/jira/browse/MESOS-238
> Project: Mesos
> Issue Type: Bug
> Environment: CentOS
> Reporter: Yanzan Wu
> Labels: test
> Original Estimate: 504h
> Remaining Estimate: 504h
>
> I was trying to run a hadoop wordcount job on mesos which runs on a single node,here comes the problem:
> The running job stays always map 0% reduce 0%
> ./bin/hadoop jar hadoop-examples-0.20.205.0.jar wordcount
> contrib/mesos/src/java/org/apache/hadoop/mapred/ out
> 12/07/23 12:11:57 INFO input.FileInputFormat: Total input paths to process : 5
> 12/07/23 12:11:58 INFO mapred.JobClient: Running job: job_201207231207_0002
> 12/07/23 12:11:59 INFO mapred.JobClient: map 0% reduce 0%
> The jobtracker process get the printout as follows:
> 12/07/23 15:44:05 INFO mapred.FrameworkScheduler: Going to launch map task for setup / cleanup
> 12/07/23 15:44:05 INFO mapred.FrameworkScheduler: Launching Mesos task 10334 as map on node2.massivedata
> 12/07/23 15:44:06 INFO mapred.FrameworkScheduler: Task 10334 is TASK_LOST
> 12/07/23 15:44:06 INFO mapred.FrameworkScheduler: Got resource offer value: "201207231205-1996531904-5050-2701-16821"
> 12/07/23 15:44:06 INFO mapred.FrameworkScheduler: Going to launch map task for setup / cleanup
> 12/07/23 15:44:06 INFO mapred.FrameworkScheduler: Launching Mesos task 10335 as map on node2.massivedata
> 12/07/23 15:44:06 INFO mapred.FrameworkScheduler: Got resource offer value: "201207231205-1996531904-5050-2701-16822"
> 12/07/23 15:44:07 INFO mapred.FrameworkScheduler: Task 10335 is TASK_LOST
> 12/07/23 15:44:07 INFO mapred.FrameworkScheduler: Killing tasks that started 20000 milliseconds ago
> 12/07/23 15:44:07 INFO mapred.FrameworkScheduler: Got resource offer value: "201207231205-1996531904-5050-2701-16823"
> 12/07/23 15:44:07 INFO mapred.FrameworkScheduler: Going to launch map task for setup / cleanup
> 12/07/23 15:44:07 INFO mapred.FrameworkScheduler: Launching Mesos task 10336 as map on node2.massivedata
> 12/07/23 15:44:08 INFO mapred.FrameworkScheduler: Task 10336 is TASK_LOST
> 12/07/23 15:44:08 INFO mapred.FrameworkScheduler: Got resource offer value: "201207231205-1996531904-5050-2701-16824"
> As it show,it repeats getting resource offer while can't launch task!
> The jps can make these result:
> 4283 Jps
> 3000 NameNode
> 3417 JobTracker
> 3140 DataNode
> 16167 RunJar
> 3293 SecondaryNameNode
> What's the problem?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira