You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Shrinivas Joshi <js...@gmail.com> on 2010/06/17 04:50:52 UTC

JIT debug symbols and OProfile under Hadoop

Sorry if this is repeat email for you, I did send this to common-dev list as
well.

Hello,
I am trying to get profiles for a workload running on top of Hadoop 0.20.2
framework. The workload jars and Hadoop jars have been compiled with debug
symbols enabled. I could see local variable tables and line number tables in
the jar class files using javap. I am passing the right value for *agentpath
* flag pointing to oprofile JVMTI agent and I am also seeing it passed
correctly to map and reduce JVMs from *ps *output. This is done through
conf/mapred-site.xml properties. However, in the profiles I am unable to see
meaningful samples for JITed code; everything is getting attributed to anon
samples.

Profiling a test app works just fine with the same JVM that is being used by
Hadoop. I can see the right debug symbols being attributed in the resulting
profiling. In fact, I even tried profiling a JVM process that was spawned
using *ProcessBuilder* API, the way Hadoop spawns its map and reduce JVMs,
that seem to work fine too.

Does anybody have any pointers as to what I might be missing here? Has
anybody successfully collected profiles showing JIT debug symbols for any
Hadoop workload? I am using OProfile 0.9.6.

Thanks,

-Shrinivas

Re: JIT debug symbols and OProfile under Hadoop

Posted by Shrinivas Joshi <js...@gmail.com>.
Hi All,

I just wanted to check if anybody had a comment on this query.

Thanks,
-Shrinivas

On Wed, Jun 16, 2010 at 9:50 PM, Shrinivas Joshi <js...@gmail.com>wrote:

> Sorry if this is repeat email for you, I did send this to common-dev list
> as well.
>
> Hello,
> I am trying to get profiles for a workload running on top of Hadoop 0.20.2
> framework. The workload jars and Hadoop jars have been compiled with debug
> symbols enabled. I could see local variable tables and line number tables in
> the jar class files using javap. I am passing the right value for *
> agentpath* flag pointing to oprofile JVMTI agent and I am also seeing it
> passed correctly to map and reduce JVMs from *ps *output. This is done
> through conf/mapred-site.xml properties. However, in the profiles I am
> unable to see meaningful samples for JITed code; everything is getting
> attributed to anon samples.
>
> Profiling a test app works just fine with the same JVM that is being used
> by Hadoop. I can see the right debug symbols being attributed in the
> resulting profiling. In fact, I even tried profiling a JVM process that was
> spawned using *ProcessBuilder* API, the way Hadoop spawns its map and
> reduce JVMs, that seem to work fine too.
>
> Does anybody have any pointers as to what I might be missing here? Has
> anybody successfully collected profiles showing JIT debug symbols for any
> Hadoop workload? I am using OProfile 0.9.6.
>
> Thanks,
>
> -Shrinivas
>

Re: JIT debug symbols and OProfile under Hadoop

Posted by Shrinivas Joshi <js...@gmail.com>.
Hi All,

I just wanted to check if anybody had a comment on this query.

Thanks,
-Shrinivas

On Wed, Jun 16, 2010 at 9:50 PM, Shrinivas Joshi <js...@gmail.com>wrote:

> Sorry if this is repeat email for you, I did send this to common-dev list
> as well.
>
> Hello,
> I am trying to get profiles for a workload running on top of Hadoop 0.20.2
> framework. The workload jars and Hadoop jars have been compiled with debug
> symbols enabled. I could see local variable tables and line number tables in
> the jar class files using javap. I am passing the right value for *
> agentpath* flag pointing to oprofile JVMTI agent and I am also seeing it
> passed correctly to map and reduce JVMs from *ps *output. This is done
> through conf/mapred-site.xml properties. However, in the profiles I am
> unable to see meaningful samples for JITed code; everything is getting
> attributed to anon samples.
>
> Profiling a test app works just fine with the same JVM that is being used
> by Hadoop. I can see the right debug symbols being attributed in the
> resulting profiling. In fact, I even tried profiling a JVM process that was
> spawned using *ProcessBuilder* API, the way Hadoop spawns its map and
> reduce JVMs, that seem to work fine too.
>
> Does anybody have any pointers as to what I might be missing here? Has
> anybody successfully collected profiles showing JIT debug symbols for any
> Hadoop workload? I am using OProfile 0.9.6.
>
> Thanks,
>
> -Shrinivas
>