You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Yaron Gonen <ya...@gmail.com> on 2013/02/06 22:50:08 UTC
Profiling the Mapper using hprof on Hadoop 0.20.205
Hi,
I wish to profile my mapper, so I've set the properties mapred.task.profileand
mapred.task.profile.maps in mapred-site.xml.
At the end of the job I'm getting a profile.out file, however I think it's
not the profile of the mapper... none of the methods I use in the mapper
appear on any of the traces. Am I doing something wrong?
I'm using a small cluster with 4 machines.
Thanks,
Yaron
RE: Profiling the Mapper using hprof on Hadoop 0.20.205
Posted by Yaron Gonen <ya...@gmail.com>.
I tried various range values.
I'm not even seeing the map method and the recordreader methods.
On Feb 7, 2013 3:58 AM, "java8964 java8964" <ja...@hotmail.com> wrote:
> What range you gave it for mapred.task.profile.maps? And you sure your
> mapper will invoke the methods you expect in the traces?
>
> Yong
>
> ------------------------------
> Date: Wed, 6 Feb 2013 23:50:08 +0200
> Subject: Profiling the Mapper using hprof on Hadoop 0.20.205
> From: yaron.gonen@gmail.com
> To: user@hadoop.apache.org
>
> Hi,
> I wish to profile my mapper, so I've set the properties
> mapred.task.profile and mapred.task.profile.maps in mapred-site.xml.
> At the end of the job I'm getting a profile.out file, however I think
> it's not the profile of the mapper... none of the methods I use in the
> mapper appear on any of the traces. Am I doing something wrong?
> I'm using a small cluster with 4 machines.
>
> Thanks,
> Yaron
>
RE: Profiling the Mapper using hprof on Hadoop 0.20.205
Posted by Yaron Gonen <ya...@gmail.com>.
I tried various range values.
I'm not even seeing the map method and the recordreader methods.
On Feb 7, 2013 3:58 AM, "java8964 java8964" <ja...@hotmail.com> wrote:
> What range you gave it for mapred.task.profile.maps? And you sure your
> mapper will invoke the methods you expect in the traces?
>
> Yong
>
> ------------------------------
> Date: Wed, 6 Feb 2013 23:50:08 +0200
> Subject: Profiling the Mapper using hprof on Hadoop 0.20.205
> From: yaron.gonen@gmail.com
> To: user@hadoop.apache.org
>
> Hi,
> I wish to profile my mapper, so I've set the properties
> mapred.task.profile and mapred.task.profile.maps in mapred-site.xml.
> At the end of the job I'm getting a profile.out file, however I think
> it's not the profile of the mapper... none of the methods I use in the
> mapper appear on any of the traces. Am I doing something wrong?
> I'm using a small cluster with 4 machines.
>
> Thanks,
> Yaron
>
RE: Profiling the Mapper using hprof on Hadoop 0.20.205
Posted by Yaron Gonen <ya...@gmail.com>.
I tried various range values.
I'm not even seeing the map method and the recordreader methods.
On Feb 7, 2013 3:58 AM, "java8964 java8964" <ja...@hotmail.com> wrote:
> What range you gave it for mapred.task.profile.maps? And you sure your
> mapper will invoke the methods you expect in the traces?
>
> Yong
>
> ------------------------------
> Date: Wed, 6 Feb 2013 23:50:08 +0200
> Subject: Profiling the Mapper using hprof on Hadoop 0.20.205
> From: yaron.gonen@gmail.com
> To: user@hadoop.apache.org
>
> Hi,
> I wish to profile my mapper, so I've set the properties
> mapred.task.profile and mapred.task.profile.maps in mapred-site.xml.
> At the end of the job I'm getting a profile.out file, however I think
> it's not the profile of the mapper... none of the methods I use in the
> mapper appear on any of the traces. Am I doing something wrong?
> I'm using a small cluster with 4 machines.
>
> Thanks,
> Yaron
>
RE: Profiling the Mapper using hprof on Hadoop 0.20.205
Posted by Yaron Gonen <ya...@gmail.com>.
I tried various range values.
I'm not even seeing the map method and the recordreader methods.
On Feb 7, 2013 3:58 AM, "java8964 java8964" <ja...@hotmail.com> wrote:
> What range you gave it for mapred.task.profile.maps? And you sure your
> mapper will invoke the methods you expect in the traces?
>
> Yong
>
> ------------------------------
> Date: Wed, 6 Feb 2013 23:50:08 +0200
> Subject: Profiling the Mapper using hprof on Hadoop 0.20.205
> From: yaron.gonen@gmail.com
> To: user@hadoop.apache.org
>
> Hi,
> I wish to profile my mapper, so I've set the properties
> mapred.task.profile and mapred.task.profile.maps in mapred-site.xml.
> At the end of the job I'm getting a profile.out file, however I think
> it's not the profile of the mapper... none of the methods I use in the
> mapper appear on any of the traces. Am I doing something wrong?
> I'm using a small cluster with 4 machines.
>
> Thanks,
> Yaron
>
RE: Profiling the Mapper using hprof on Hadoop 0.20.205
Posted by java8964 java8964 <ja...@hotmail.com>.
What range you gave it for mapred.task.profile.maps? And you sure your mapper will invoke the methods you expect in the traces?
Yong
Date: Wed, 6 Feb 2013 23:50:08 +0200
Subject: Profiling the Mapper using hprof on Hadoop 0.20.205
From: yaron.gonen@gmail.com
To: user@hadoop.apache.org
Hi,I wish to profile my mapper, so I've set the properties mapred.task.profile and mapred.task.profile.maps in mapred-site.xml.
At the end of the job I'm getting a profile.out file, however I think it's not the profile of the mapper... none of the methods I use in the mapper appear on any of the traces. Am I doing something wrong?
I'm using a small cluster with 4 machines.
Thanks,Yaron
RE: Profiling the Mapper using hprof on Hadoop 0.20.205
Posted by java8964 java8964 <ja...@hotmail.com>.
What range you gave it for mapred.task.profile.maps? And you sure your mapper will invoke the methods you expect in the traces?
Yong
Date: Wed, 6 Feb 2013 23:50:08 +0200
Subject: Profiling the Mapper using hprof on Hadoop 0.20.205
From: yaron.gonen@gmail.com
To: user@hadoop.apache.org
Hi,I wish to profile my mapper, so I've set the properties mapred.task.profile and mapred.task.profile.maps in mapred-site.xml.
At the end of the job I'm getting a profile.out file, however I think it's not the profile of the mapper... none of the methods I use in the mapper appear on any of the traces. Am I doing something wrong?
I'm using a small cluster with 4 machines.
Thanks,Yaron
RE: Profiling the Mapper using hprof on Hadoop 0.20.205
Posted by java8964 java8964 <ja...@hotmail.com>.
What range you gave it for mapred.task.profile.maps? And you sure your mapper will invoke the methods you expect in the traces?
Yong
Date: Wed, 6 Feb 2013 23:50:08 +0200
Subject: Profiling the Mapper using hprof on Hadoop 0.20.205
From: yaron.gonen@gmail.com
To: user@hadoop.apache.org
Hi,I wish to profile my mapper, so I've set the properties mapred.task.profile and mapred.task.profile.maps in mapred-site.xml.
At the end of the job I'm getting a profile.out file, however I think it's not the profile of the mapper... none of the methods I use in the mapper appear on any of the traces. Am I doing something wrong?
I'm using a small cluster with 4 machines.
Thanks,Yaron
RE: Profiling the Mapper using hprof on Hadoop 0.20.205
Posted by java8964 java8964 <ja...@hotmail.com>.
What range you gave it for mapred.task.profile.maps? And you sure your mapper will invoke the methods you expect in the traces?
Yong
Date: Wed, 6 Feb 2013 23:50:08 +0200
Subject: Profiling the Mapper using hprof on Hadoop 0.20.205
From: yaron.gonen@gmail.com
To: user@hadoop.apache.org
Hi,I wish to profile my mapper, so I've set the properties mapred.task.profile and mapred.task.profile.maps in mapred-site.xml.
At the end of the job I'm getting a profile.out file, however I think it's not the profile of the mapper... none of the methods I use in the mapper appear on any of the traces. Am I doing something wrong?
I'm using a small cluster with 4 machines.
Thanks,Yaron