You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by skaterQiang <xq...@163.com> on 2011/06/29 14:39:35 UTC

How to debug map reduce in distributed model

Hello guys:
        I want to debug the map reduce tasks, especially org.apache.hadoop.mapred.ReduceTask
I want to understand how the data flow work.
So I add the following in the mapred-site.xml
<property>
        <name>mapred.child.java.opts</name>
        <value>-Xdebug -Xrunjdwp:transport=dt_socket,address=8888,server=y,suspend=y</value>
    </property>
mapreduce.tasktracker.map.tasks.maximum 1
mapreduce.tasktracker.reduce.tasks.maximum 1
But by debugging JvmRunner, the -Xdebug is not added into the JVM start up command.
Why? Is there some where guard this?
Or I have to build hadoop with my debug code?


Regards,
Skater

Re:How to debug map reduce in distributed model

Posted by skaterQiang <xq...@163.com>.

I hate to write these code, although it can work
      String javaOpts = conf.get("mapred.child.java.opts", "-Xdebug -Xrunjdwp:transport=dt_socket,address=8888,server=y,suspend=y");
      javaOpts = "-Xmx200m -Xdebug -Xrunjdwp:transport=dt_socket,address=8888,server=y,suspend=y";//javaOpts.replace("@taskid@", taskid.toString());


At 2011-06-29 20:39:35，skaterQiang <xq...@163.com> wrote:
Hello guys:
        I want to debug the map reduce tasks, especially org.apache.hadoop.mapred.ReduceTask
I want to understand how the data flow work.
So I add the following in the mapred-site.xml
<property>
        <name>mapred.child.java.opts</name>
        <value>-Xdebug -Xrunjdwp:transport=dt_socket,address=8888,server=y,suspend=y</value>
    </property>
mapreduce.tasktracker.map.tasks.maximum 1
mapreduce.tasktracker.reduce.tasks.maximum 1
But by debugging JvmRunner, the -Xdebug is not added into the JVM start up command.
Why? Is there some where guard this?
Or I have to build hadoop with my debug code?


Regards,
Skater

Re:How to debug map reduce in distributed model

Posted by skaterQiang <xq...@163.com>.

I hate to write these code, although it can work
      String javaOpts = conf.get("mapred.child.java.opts", "-Xdebug -Xrunjdwp:transport=dt_socket,address=8888,server=y,suspend=y");
      javaOpts = "-Xmx200m -Xdebug -Xrunjdwp:transport=dt_socket,address=8888,server=y,suspend=y";//javaOpts.replace("@taskid@", taskid.toString());


At 2011-06-29 20:39:35，skaterQiang <xq...@163.com> wrote:
Hello guys:
        I want to debug the map reduce tasks, especially org.apache.hadoop.mapred.ReduceTask
I want to understand how the data flow work.
So I add the following in the mapred-site.xml
<property>
        <name>mapred.child.java.opts</name>
        <value>-Xdebug -Xrunjdwp:transport=dt_socket,address=8888,server=y,suspend=y</value>
    </property>
mapreduce.tasktracker.map.tasks.maximum 1
mapreduce.tasktracker.reduce.tasks.maximum 1
But by debugging JvmRunner, the -Xdebug is not added into the JVM start up command.
Why? Is there some where guard this?
Or I have to build hadoop with my debug code?


Regards,
Skater