You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Shubham Chopra <sh...@gmail.com> on 2011/06/27 19:44:03 UTC

profiling mappers using hprof through pig

Hi,

I wanted to profile the jobs being created by my pig-script. As per the
hadoop documentation, I can use hprof and set suitable parameters.

So I added the following lines to my pig-script:
set mapred.task.profile true
set mapred.task.profile.params
-agentlib:hprof=heap=sites,file=output_hprof.txt

I do see that the first three mappers are significantly slower after the
addition of these two lines, implying that the profiler is working. I,
however, do not see any output in the directory I ran pig from. Does anyone
know the location of profiler output? Has anyone used hprof with pig to
profile the mappers before?

Thanks,
Shubham.