You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@htrace.apache.org by Colin McCabe <cm...@alumni.cmu.edu> on 2015/01/28 04:23:19 UTC
HTrace build products in Maven Central?
Hi guys,
I apologize in advance that there are probably some dumb questions
here. I don't know much about packaging.
What's the plan for pushing HTrace 3.1.0 build products to Maven
Central (once 3.1.0 is accepted)?
It seems like we have 3 options:
1. fatjar that includes all dependencies in shaded form
2. fatjar that includes jetty, jackson, and a few other things in
shaded form, but not zipkin / flume / hbase
3. separate jars for each subproject (htrace-core, htrace-hbase, etc.)
I don't think option #1 is really viable. Some of our submodules
depend on lots of stuff... like htrace-hbase pulls in all the hbase
jars, htrace-flume pulls in all the flume jars, and so forth. That
seems too big to upload to Maven Central, and nobody would want to put
such an enormous jar on the classpath.
Option #2 might make sense. I think this jar would have a reasonable size.
Option #3 is the most modular. But it puts the burden on users to
manually add the jars for the HTrace collector subproject they want to
their CLASSPATH.
Strictly speaking, the only dependency on HTrace that Hadoop *needs*
is a dependency on htrace-core. This contains the Span,
SpanReceiverBuilder, and other core classes that are needed to
integrate with HTrace.
But in a world where the Hadoop CLASSPATH just has htrace-core.jar on
it, users won't be able to use Span receivers like the hbase one
without manually adding more jars. That seems cumbersome.
I think maybe solution #2 is the best. What do you think?
On a related topic, I'm kind of troubled by some of the dependencies I
see in htrace-hbase and htrace-zipkin. The Guava "provided"
dependency seems like a grenade waiting to go off. We should just
remove Guava from those subprojects to avoid any chance of version
collisions between our version and the version used by the HBase /
Hadoop / whatever dependencies. Also, htrace-hbase is pulling in
protobuf as a "provided" dependency (sad face). It's only a matter of
time until Hadoop bumps its protobuf dependency again (the 2.4.1 ->
2.5 transition caused havoc.) We should fix that.
best,
Colin
Re: HTrace build products in Maven Central?
Posted by Stack <st...@duboce.net>.
On Tue, Jan 27, 2015 at 7:23 PM, Colin McCabe <cm...@alumni.cmu.edu>
wrote:
> Hi guys,
>
> I apologize in advance that there are probably some dumb questions
> here. I don't know much about packaging.
>
> What's the plan for pushing HTrace 3.1.0 build products to Maven
> Central (once 3.1.0 is accepted)?
>
It seems like we have 3 options:
> 1. fatjar that includes all dependencies in shaded form
> 2. fatjar that includes jetty, jackson, and a few other things in
> shaded form, but not zipkin / flume / hbase
> 3. separate jars for each subproject (htrace-core, htrace-hbase, etc.)
>
>
It is up in mvn central now in form #3:
http://mvnrepository.com/artifact/org.apache.htrace
http://search.maven.org/#artifactdetails%7Corg.apache.htrace%7Chtrace%7C3.1.0-incubating%7Cpom
> I don't think option #1 is really viable. Some of our submodules
> depend on lots of stuff... like htrace-hbase pulls in all the hbase
> jars, htrace-flume pulls in all the flume jars, and so forth. That
> seems too big to upload to Maven Central, and nobody would want to put
> such an enormous jar on the classpath.
>
> Option #2 might make sense. I think this jar would have a reasonable size.
>
> Option #3 is the most modular. But it puts the burden on users to
> manually add the jars for the HTrace collector subproject they want to
> their CLASSPATH.
>
> Strictly speaking, the only dependency on HTrace that Hadoop *needs*
> is a dependency on htrace-core. This contains the Span,
> SpanReceiverBuilder, and other core classes that are needed to
> integrate with HTrace.
>
> But in a world where the Hadoop CLASSPATH just has htrace-core.jar on
> it, users won't be able to use Span receivers like the hbase one
> without manually adding more jars. That seems cumbersome.
>
> I think maybe solution #2 is the best. What do you think?
>
>
#2 presumes the receiver they want to use in the ordained set.
No harm in including on CLASSPATH some basic receiver I'd say so can try
simple tracing... . Could be as distinct jar or bundled up as #2. If want
to invest more heavily in tracing, then could bring the big guns by doing a
restart to add zipkin or hbase to the CLASSPATH.
> On a related topic, I'm kind of troubled by some of the dependencies I
> see in htrace-hbase and htrace-zipkin. The Guava "provided"
> dependency seems like a grenade waiting to go off. We should just
> remove Guava from those subprojects to avoid any chance of version
> collisions between our version and the version used by the HBase /
> Hadoop / whatever dependencies. Also, htrace-hbase is pulling in
> protobuf as a "provided" dependency (sad face). It's only a matter of
> time until Hadoop bumps its protobuf dependency again (the 2.4.1 ->
> 2.5 transition caused havoc.) We should fix that.
>
>
hbase has same disease as hadoop; it needs a dependency cleanup. Shouldn't
be part of the base htrace I'd say. Ditto zipkin. These projects are
usually at core, not the guests of another process and their dependency
list shows it.
St.Ack
Re: HTrace build products in Maven Central?
Posted by Colin McCabe <cm...@alumni.cmu.edu>.
I filed HTRACE-90 to fix the dependency issues. C.
On Tue, Jan 27, 2015 at 7:23 PM, Colin McCabe <cm...@alumni.cmu.edu> wrote:
> Hi guys,
>
> I apologize in advance that there are probably some dumb questions
> here. I don't know much about packaging.
>
> What's the plan for pushing HTrace 3.1.0 build products to Maven
> Central (once 3.1.0 is accepted)?
>
> It seems like we have 3 options:
> 1. fatjar that includes all dependencies in shaded form
> 2. fatjar that includes jetty, jackson, and a few other things in
> shaded form, but not zipkin / flume / hbase
> 3. separate jars for each subproject (htrace-core, htrace-hbase, etc.)
>
> I don't think option #1 is really viable. Some of our submodules
> depend on lots of stuff... like htrace-hbase pulls in all the hbase
> jars, htrace-flume pulls in all the flume jars, and so forth. That
> seems too big to upload to Maven Central, and nobody would want to put
> such an enormous jar on the classpath.
>
> Option #2 might make sense. I think this jar would have a reasonable size.
>
> Option #3 is the most modular. But it puts the burden on users to
> manually add the jars for the HTrace collector subproject they want to
> their CLASSPATH.
>
> Strictly speaking, the only dependency on HTrace that Hadoop *needs*
> is a dependency on htrace-core. This contains the Span,
> SpanReceiverBuilder, and other core classes that are needed to
> integrate with HTrace.
>
> But in a world where the Hadoop CLASSPATH just has htrace-core.jar on
> it, users won't be able to use Span receivers like the hbase one
> without manually adding more jars. That seems cumbersome.
>
> I think maybe solution #2 is the best. What do you think?
>
> On a related topic, I'm kind of troubled by some of the dependencies I
> see in htrace-hbase and htrace-zipkin. The Guava "provided"
> dependency seems like a grenade waiting to go off. We should just
> remove Guava from those subprojects to avoid any chance of version
> collisions between our version and the version used by the HBase /
> Hadoop / whatever dependencies. Also, htrace-hbase is pulling in
> protobuf as a "provided" dependency (sad face). It's only a matter of
> time until Hadoop bumps its protobuf dependency again (the 2.4.1 ->
> 2.5 transition caused havoc.) We should fix that.
>
> best,
> Colin