You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by danehammer <da...@gmail.com> on 2012/11/22 00:07:24 UTC
When to manage transitive dependencies
I have two uberjars that I build with the maven-shade-plugin, we'll call them
"batch" and "realtime". They have a set of common dependencies that I want
to keep in sync (our data processing jars). But they have dependencies
themselves that are completely different between these two jars runtimes.
Sometimes there is an intersection between their runtime's dependencies, and
the data processing dependencies. That has to be managed and tested.
What's a best practice, or just an approach others have tried, for lessening
the likelihood that I have issues down the road? I need the data processing
jars to stay in sync of course, and that's easy. But when those jars
dependencies (that is, my uberjars' transitive dependencies) intersect with
the uberjar's runtime's dependencies, I can have issues.
If a more concrete example helps here it is. My data processing dependencies
are running in two runtimes, batch processing (Map/Reduce) and realtime
processing (storm). Map/Reduce, with one of its hadoop libs, uses asm 3.x.
Storm depends on kryo, which uses asm 4.x. This could be managed separately
maybe, but it gets even more fun when one of my data processing dependencies
uses groovy 2.x, which uses asm 4.x.
--
View this message in context: http://maven.40175.n5.nabble.com/When-to-manage-transitive-dependencies-tp5732405.html
Sent from the Maven - Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: When to manage transitive dependencies
Posted by danehammer <da...@gmail.com>.
What do you mean by "plain dependency management"? At the end of the day,
what else do you do to get dependencies available at runtime for
dependencies not provided by your container?
--
View this message in context: http://maven.40175.n5.nabble.com/When-to-manage-transitive-dependencies-tp5732405p5733592.html
Sent from the Maven - Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: When to manage transitive dependencies
Posted by Barrie Treloar <ba...@gmail.com>.
Stop doing shade and uberjars?
Seriously look at using plain dependency management and your life
should be a lot easier.
Your case is complicated so maybe shade is your only option.
Plain dependencies will enable you to spot collisions easily and to
manage/lock down versions and use enforcer to help that.
Can you make your data processing independent of its runtime (batch or
realtime)?
Alternatively you could patch Map/Reduce and hadoop to bump the asm
libs to 4.x to align your versions.
On Thu, Nov 22, 2012 at 2:03 PM, Ron Wheeler
<rw...@artifact-software.com> wrote:
> It looks a bit complex for my taste but it seems that "excludes" on
> transitive dependencies should eliminate at least one of your problems.
>
> Ron
>
>
> On 21/11/2012 6:07 PM, danehammer wrote:
>>
>> I have two uberjars that I build with the maven-shade-plugin, we'll call
>> them
>> "batch" and "realtime". They have a set of common dependencies that I want
>> to keep in sync (our data processing jars). But they have dependencies
>> themselves that are completely different between these two jars runtimes.
>> Sometimes there is an intersection between their runtime's dependencies,
>> and
>> the data processing dependencies. That has to be managed and tested.
>>
>> What's a best practice, or just an approach others have tried, for
>> lessening
>> the likelihood that I have issues down the road? I need the data
>> processing
>> jars to stay in sync of course, and that's easy. But when those jars
>> dependencies (that is, my uberjars' transitive dependencies) intersect
>> with
>> the uberjar's runtime's dependencies, I can have issues.
>>
>> If a more concrete example helps here it is. My data processing
>> dependencies
>> are running in two runtimes, batch processing (Map/Reduce) and realtime
>> processing (storm). Map/Reduce, with one of its hadoop libs, uses asm 3.x.
>> Storm depends on kryo, which uses asm 4.x. This could be managed
>> separately
>> maybe, but it gets even more fun when one of my data processing
>> dependencies
>> uses groovy 2.x, which uses asm 4.x.
>>
>>
>>
>> --
>> View this message in context:
>> http://maven.40175.n5.nabble.com/When-to-manage-transitive-dependencies-tp5732405.html
>> Sent from the Maven - Users mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>
>
> --
> Ron Wheeler
> President
> Artifact Software Inc
> email: rwheeler@artifact-software.com
> skype: ronaldmwheeler
> phone: 866-970-2435, ext 102
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: When to manage transitive dependencies
Posted by Ron Wheeler <rw...@artifact-software.com>.
It looks a bit complex for my taste but it seems that "excludes" on
transitive dependencies should eliminate at least one of your problems.
Ron
On 21/11/2012 6:07 PM, danehammer wrote:
> I have two uberjars that I build with the maven-shade-plugin, we'll call them
> "batch" and "realtime". They have a set of common dependencies that I want
> to keep in sync (our data processing jars). But they have dependencies
> themselves that are completely different between these two jars runtimes.
> Sometimes there is an intersection between their runtime's dependencies, and
> the data processing dependencies. That has to be managed and tested.
>
> What's a best practice, or just an approach others have tried, for lessening
> the likelihood that I have issues down the road? I need the data processing
> jars to stay in sync of course, and that's easy. But when those jars
> dependencies (that is, my uberjars' transitive dependencies) intersect with
> the uberjar's runtime's dependencies, I can have issues.
>
> If a more concrete example helps here it is. My data processing dependencies
> are running in two runtimes, batch processing (Map/Reduce) and realtime
> processing (storm). Map/Reduce, with one of its hadoop libs, uses asm 3.x.
> Storm depends on kryo, which uses asm 4.x. This could be managed separately
> maybe, but it gets even more fun when one of my data processing dependencies
> uses groovy 2.x, which uses asm 4.x.
>
>
>
> --
> View this message in context: http://maven.40175.n5.nabble.com/When-to-manage-transitive-dependencies-tp5732405.html
> Sent from the Maven - Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
--
Ron Wheeler
President
Artifact Software Inc
email: rwheeler@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org