development setup for Htrace

[Nisala Mendis <ni...@gmail.com>]

Hi all,

I was able to setup HDFS cluster as per [1] and HTrace service up and
running could access Htrace webapp. I would like to know the following,

1. How do I remote debug Htrace maven component ( Htrace-Core etc ) when
the code is executed by htrace service?
2. How do I modify a Jar associated with Htrace maven component (
Htrace-Core etc ). lets say I have made a modification to Htace core and
build the component using maven. How do I patch the htrace service to
reflect the change to Htrace-core? Where do I need to put that jar?

[1]
http://blog.cloudera.com/blog/2015/12/new-in-cloudera-labs-apache-htrace-incubating/

Regards
Nisala

Re: development setup for Htrace

[Colin McCabe <cm...@apache.org>]

Good questions!
 
On Tue, May 31, 2016, at 07:22, Nisala Mendis wrote:
> Hi all,
> I was able to setup HDFS cluster as per [1] and HTrace service up and
> running could access Htrace webapp. I would like to know the
> following,
> 1. How do I remote debug Htrace maven component ( Htrace-Core etc )
>    when the code is executed by htrace service?
 
I find log4j messages very useful when debugging things like htrace span
receivers.  You can see a lot of log messages sprinkled throughout the
code for exactly this reason.  If you set your log4j level to TRACE, you
will see all the log messages.
 
There are also "traditional" debuggers like jdb that attach to the
process and let you single-step, and so forth.  But jdb only attaches to
one process at a time.  In a distributed system, other things will be
going on while you have stopped that one process.  So it tends to be
less useful than you might at first think.
 
> 2. How do I modify a Jar associated with Htrace maven component (
>    Htrace-Core etc ). lets say I have made a modification to Htace
>    core and build the component using maven. How do I patch the
>    htrace service to reflect the change to Htrace-core? Where do I
>    need to put that jar?
 
The htrace-core4 component is a special case, since it gets published to
the Sonatype repository and pulled in to the Hadoop build.  It would be
somewhat complex to use a different version of this-- you'd have to
build your different version, then rebuild Hadoop against it, then
reinstall Hadoop.
 
Since you are mainly interested in developing a new span receiver, the
procedure is a lot simpler.  Just build your new span receiver in your
local htrace repo.  It should generate a jar file.  For example, if you
are dealing with the htrace-htraced span receiver, it will generate
this jar file:
 
./htrace-htraced/target/htrace-htraced-4.2.0-incubating-SNAPSHOT.jar
 
You can verify that it contains your span receiver classes:
 
$ cmccabe@mirabilis:~/htrace> jar tvf ./htrace-htraced/target/htrace-htraced-4.2.0-incubating-
SNAPSHOT.jar | grep SpanReceiver
11635 Mon May 09 11:26:12 PDT 2016
org/apache/htrace/impl/HTracedSpanReceiver.class
7644 Mon May 09 11:26:12 PDT 2016
org/apache/htrace/impl/HTracedSpanReceiver$PostSpansThread.class
886 Mon May 09 11:26:12 PDT 2016
org/apache/htrace/impl/HTracedSpanReceiver$FaultInjector.class
 
Then copy this jar to your CLASSPATH.  In your case, it would probably
be: /opt/cloudera/parcels/CDH/lib/hadoop
 
If you are developing a new span receiver, of course you'll want to
create your own top-level directory.  But that's the general idea.
 
best,
Colin
 
> [1] http://blog.cloudera.com/blog/2015/12/new-in-cloudera-labs-apache-htrace-incubating/
>
> Regards
> Nisala