You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Jimmy Wan <ji...@indeed.com> on 2009/04/02 01:55:51 UTC

Profiling with Hadoop 0.17.2.1

I'm trying to profile my map/reduce processes under Hadoop 0.17.2.
>From looking at the hadoop-default.xml, the property
"mapred.task.profile.params" did not yet exist back then, so I'm
trying to add to the property "mapred.child.java.opts" with
-Xmx512m -verbose:gc
-Xrunhprof:cpu=samples,depth=6,thread=y,file=/tmp/@taskid@.txt
-Xloggc:/tmp/@taskid@.gc

The resulting JVMs won't have the hprof parameters when I look at them
via PS, the files are never created and there is no mention of dumping
stats in the logs.

Am I missing something?

I'd switch to 0.19.1, but I haven't had time to setup a migration plan
for my data yet.

Jimmy Wan