You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by kwshinsdu <kw...@126.com> on 2014/02/10 08:31:04 UTC

A puzzle about hadoop on mesos!

Dear,Mr/Ms:
I deploy my hadoop on mesos following :https://github.com/mesos/hadoop.I have a puzzly :since my mapreduce program is finished ,a Task_Tracker(sometimes two tasks) is still activity in mesos framework(shown as fig1 and fig2) .In my view, it may be not normal.
Then I reset the variable "mapred.tasktracker.map.tasks.maximum" and "mapred.tasktracker.reduce.tasks.maximum" to 1,when the program is finished（shown as fig3 and fig4）, there is no active tasks in mesos framework.
My mesos version is mesos-0.14.2 and hadoop version is hadoop-1.2.1.
I curious about this.so would you explain this for me, thank you very much!
Best Regards!
WS Kang
 
fig1
fig2
 
fig3
fig4
 
 
 
 
 
 
2014-01-27
kwshinsdu

RE: A puzzle about hadoop on mesos!

Posted by Aaron Gottlieb <aa...@conversantmedia.com>.
It’s possible the tasktracker is still writing data to disk.  A task may report itself 100% complete but still be running because all of its data is not flushed to disk (and replicated in HDFS).  I would recommend checking the actual Hadoop status page for the job.  You may see something like BYTES_WRITTEN increasing every time you refresh the page even if the job says it is already completed.

From: kwshinsdu [mailto:kwshinsdu@126.com]
Sent: Sunday, February 09, 2014 11:31 PM
To: user
Subject: A puzzle about hadoop on mesos!

Dear,Mr/Ms:
I deploy my hadoop on mesos following :https://github.com/mesos/hadoop.I have a puzzly :since my mapreduce program is finished ,a Task_Tracker(sometimes two tasks) is still activity in mesos framework(shown as fig1 and fig2) .In my view, it may be not normal.
Then I reset the variable "mapred.tasktracker.map.tasks.maximum" and "mapred.tasktracker.reduce.tasks.maximum" to 1,when the program is finished（shown as fig3 and fig4）, there is no active tasks in mesos framework.
My mesos version is mesos-0.14.2 and hadoop version is hadoop-1.2.1.
I curious about this.so would you explain this for me, thank you very much!
Best Regards!
WS Kang
[cid:image001.png@01CF264C.2BEB88E0]

fig1
[cid:image002.png@01CF264C.2BEB88E0]
fig2
[cid:image003.png@01CF264C.2BEB88E0]

fig3
[cid:image004.png@01CF264C.2BEB88E0]
fig4






2014-01-27
________________________________
kwshinsdu