You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kudu.apache.org by Boris Tyukin <bo...@boristyukin.com> on 2019/01/02 19:44:08 UTC
kudu-client dependencies
Hi guys,
sorry for a dumb question but why kudu-client.jar does not include async
and slf4j-api and slf4j-simple libs? I need to call Kudu API from a simple
groovy script and had to add 3 other jars explicitly.
I see these libs were excluded on purpose:
https://github.com/apache/kudu/blob/master/java/kudu-client/build.gradle
Kafka client, for example, is a single jar.
My challenge now which version of these libs to pick, how to support them
so they won't break in future etc.
Even on CDH cluster, while kudu client is shipped with parcel, one has to
know exact versions of other 3 jars for client to work.
Maybe I am missing something here and there is an easy way, especially on
CDH since it ships already with Kudu.
Boris
Re: kudu-client dependencies
Posted by Boris Tyukin <bo...@boristyukin.com>.
thanks for the explanation, Mike.
Boris
On Wed, Jan 2, 2019 at 6:32 PM Mike Percy <mp...@apache.org> wrote:
> Hi Boris,
> kudu-client is a client API library designed to be embedded in a client
> application, and it specifies its dependencies via a Maven pom. Typically
> one would only want one version of a given dep on the classpath at runtime
> and so shipping a fat jar usually isn't done for client libraries.
>
> We shade all dependencies that are not exposed via the public API except
> slf4j and related bindings since those are typically provided by the
> application (e.g. slf4j-log4j). Since async appears in the public Kudu
> Client API we can't shade it.
>
> kudu-client-tools is not a library but a set of command-line tools, so it
> has to carry all of its dependencies in the jar.
>
> I'm not sure how most people handle dependency management in the Groovy
> world, but a quick Google search turned up Grape
> <http://docs.groovy-lang.org/latest/html/documentation/grape.html>, so
> maybe that's worth looking into.
>
> Regards,
> Mike
>
>
> On Wed, Jan 2, 2019 at 12:37 PM Boris Tyukin <bo...@boristyukin.com>
> wrote:
>
>> ok we just figured out that we need another jar - kudu-client-tools.jar.
>> that one bundled with a proper version async lib and slf4j-api.
>>
>> slf4j-simple.jar has to be added separately but you do not have to do it
>> if it is okay to suppress kudu client logs.
>>
>> kudu-client.jar and kudu-client-tools.jar are symlinked to a proper
>> version of jars for CDH parcel.
>>
>> /opt/cloudera/parcels/CDH/lib/kudu/kudu-client.jar
>> /opt/cloudera/parcels/CDH/lib/kudu/kudu-client-tools.jar
>> /opt/cloudera/parcels/CDH/jars/slf4j-simple-1.7.5.jar
>>
>>
>>
>> On Wed, Jan 2, 2019 at 2:44 PM Boris Tyukin <bo...@boristyukin.com>
>> wrote:
>>
>>> Hi guys,
>>>
>>> sorry for a dumb question but why kudu-client.jar does not include async
>>> and slf4j-api and slf4j-simple libs? I need to call Kudu API from a simple
>>> groovy script and had to add 3 other jars explicitly.
>>>
>>> I see these libs were excluded on purpose:
>>> https://github.com/apache/kudu/blob/master/java/kudu-client/build.gradle
>>>
>>> Kafka client, for example, is a single jar.
>>>
>>> My challenge now which version of these libs to pick, how to support
>>> them so they won't break in future etc.
>>>
>>> Even on CDH cluster, while kudu client is shipped with parcel, one has
>>> to know exact versions of other 3 jars for client to work.
>>>
>>> Maybe I am missing something here and there is an easy way, especially
>>> on CDH since it ships already with Kudu.
>>>
>>> Boris
>>>
>>
Re: kudu-client dependencies
Posted by Mike Percy <mp...@apache.org>.
Hi Boris,
kudu-client is a client API library designed to be embedded in a client
application, and it specifies its dependencies via a Maven pom. Typically
one would only want one version of a given dep on the classpath at runtime
and so shipping a fat jar usually isn't done for client libraries.
We shade all dependencies that are not exposed via the public API except
slf4j and related bindings since those are typically provided by the
application (e.g. slf4j-log4j). Since async appears in the public Kudu
Client API we can't shade it.
kudu-client-tools is not a library but a set of command-line tools, so it
has to carry all of its dependencies in the jar.
I'm not sure how most people handle dependency management in the Groovy
world, but a quick Google search turned up Grape
<http://docs.groovy-lang.org/latest/html/documentation/grape.html>, so
maybe that's worth looking into.
Regards,
Mike
On Wed, Jan 2, 2019 at 12:37 PM Boris Tyukin <bo...@boristyukin.com> wrote:
> ok we just figured out that we need another jar - kudu-client-tools.jar.
> that one bundled with a proper version async lib and slf4j-api.
>
> slf4j-simple.jar has to be added separately but you do not have to do it
> if it is okay to suppress kudu client logs.
>
> kudu-client.jar and kudu-client-tools.jar are symlinked to a proper
> version of jars for CDH parcel.
>
> /opt/cloudera/parcels/CDH/lib/kudu/kudu-client.jar
> /opt/cloudera/parcels/CDH/lib/kudu/kudu-client-tools.jar
> /opt/cloudera/parcels/CDH/jars/slf4j-simple-1.7.5.jar
>
>
>
> On Wed, Jan 2, 2019 at 2:44 PM Boris Tyukin <bo...@boristyukin.com> wrote:
>
>> Hi guys,
>>
>> sorry for a dumb question but why kudu-client.jar does not include async
>> and slf4j-api and slf4j-simple libs? I need to call Kudu API from a simple
>> groovy script and had to add 3 other jars explicitly.
>>
>> I see these libs were excluded on purpose:
>> https://github.com/apache/kudu/blob/master/java/kudu-client/build.gradle
>>
>> Kafka client, for example, is a single jar.
>>
>> My challenge now which version of these libs to pick, how to support them
>> so they won't break in future etc.
>>
>> Even on CDH cluster, while kudu client is shipped with parcel, one has to
>> know exact versions of other 3 jars for client to work.
>>
>> Maybe I am missing something here and there is an easy way, especially on
>> CDH since it ships already with Kudu.
>>
>> Boris
>>
>
Re: kudu-client dependencies
Posted by Boris Tyukin <bo...@boristyukin.com>.
ok we just figured out that we need another jar - kudu-client-tools.jar.
that one bundled with a proper version async lib and slf4j-api.
slf4j-simple.jar has to be added separately but you do not have to do it if
it is okay to suppress kudu client logs.
kudu-client.jar and kudu-client-tools.jar are symlinked to a proper version
of jars for CDH parcel.
/opt/cloudera/parcels/CDH/lib/kudu/kudu-client.jar
/opt/cloudera/parcels/CDH/lib/kudu/kudu-client-tools.jar
/opt/cloudera/parcels/CDH/jars/slf4j-simple-1.7.5.jar
On Wed, Jan 2, 2019 at 2:44 PM Boris Tyukin <bo...@boristyukin.com> wrote:
> Hi guys,
>
> sorry for a dumb question but why kudu-client.jar does not include async
> and slf4j-api and slf4j-simple libs? I need to call Kudu API from a simple
> groovy script and had to add 3 other jars explicitly.
>
> I see these libs were excluded on purpose:
> https://github.com/apache/kudu/blob/master/java/kudu-client/build.gradle
>
> Kafka client, for example, is a single jar.
>
> My challenge now which version of these libs to pick, how to support them
> so they won't break in future etc.
>
> Even on CDH cluster, while kudu client is shipped with parcel, one has to
> know exact versions of other 3 jars for client to work.
>
> Maybe I am missing something here and there is an easy way, especially on
> CDH since it ships already with Kudu.
>
> Boris
>