hive-exec shaded jar a bad idea?

[Kristof Vanbecelaere <kr...@gmail.com>]

Hi,

we are using gradle as our build tool. I need hive-exec to use class
OrcFile. As soon as I pull in hive-exec 0.12.0 as a dependency my build
starts to fail with

java.lang.NoClassDefFoundError: Could not initialize class
org.gradle.internal.SystemProperties"

This class uses ImmutableSet.of() for static initialization. I found out
that hive-exec actually includes guava and so I guess the classloaders for
both copies of ImmutableSet are different leading to the above exception.

Is it really necessary to bundle guava (and many others) in the exec jar?
Can you provide all classes in regular jars and make a separate maven
module for the shaded jar? If not, then can you please relocate those
classes as described in
http://maven.apache.org/plugins/maven-shade-plugin/examples/class-relocation.html

Is there a way to solve this in the short term? Going back to 0.11 seems to
be the best solution for now.

Thanks

Re: hive-exec shaded jar a bad idea?

[Owen O'Malley <om...@apache.org>]

There is already a jira for it:
https://issues.apache.org/jira/browse/HIVE-5725


On Thu, Nov 21, 2013 at 10:25 AM, Owen O'Malley <om...@apache.org> wrote:

> I think we should create a ql jar that contains the ql code and no
> dependencies. We can still make an exec jar that bundles the dependencies.
>
>
> On Thu, Nov 21, 2013 at 6:50 AM, Edward Capriolo <ed...@gmail.com>wrote:
>
>> That is a good idea. I have also considered jar jar, as well as stripping
>> these things from hive-exec. Guava provides some nice functions for sure
>> but when it causes classpath issues with other things, you say to your
>> self, crud all these problems just for Preconditions.checkNotNull() ?
>>
>>
>> On Thu, Nov 21, 2013 at 7:55 AM, Kristof Vanbecelaere <
>> kristof.vanbecelaere@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> we are using gradle as our build tool. I need hive-exec to use class
>>> OrcFile. As soon as I pull in hive-exec 0.12.0 as a dependency my build
>>> starts to fail with
>>>
>>> java.lang.NoClassDefFoundError: Could not initialize class
>>> org.gradle.internal.SystemProperties"
>>>
>>> This class uses ImmutableSet.of() for static initialization. I found out
>>> that hive-exec actually includes guava and so I guess the classloaders for
>>> both copies of ImmutableSet are different leading to the above exception.
>>>
>>> Is it really necessary to bundle guava (and many others) in the exec
>>> jar? Can you provide all classes in regular jars and make a separate maven
>>> module for the shaded jar? If not, then can you please relocate those
>>> classes as described in
>>> http://maven.apache.org/plugins/maven-shade-plugin/examples/class-relocation.html
>>>
>>> Is there a way to solve this in the short term? Going back to 0.11 seems
>>> to be the best solution for now.
>>>
>>> Thanks
>>>
>>
>>
>

Re: hive-exec shaded jar a bad idea?

[Owen O'Malley <om...@apache.org>]

I think we should create a ql jar that contains the ql code and no
dependencies. We can still make an exec jar that bundles the dependencies.


On Thu, Nov 21, 2013 at 6:50 AM, Edward Capriolo <ed...@gmail.com>wrote:

> That is a good idea. I have also considered jar jar, as well as stripping
> these things from hive-exec. Guava provides some nice functions for sure
> but when it causes classpath issues with other things, you say to your
> self, crud all these problems just for Preconditions.checkNotNull() ?
>
>
> On Thu, Nov 21, 2013 at 7:55 AM, Kristof Vanbecelaere <
> kristof.vanbecelaere@gmail.com> wrote:
>
>> Hi,
>>
>> we are using gradle as our build tool. I need hive-exec to use class
>> OrcFile. As soon as I pull in hive-exec 0.12.0 as a dependency my build
>> starts to fail with
>>
>> java.lang.NoClassDefFoundError: Could not initialize class
>> org.gradle.internal.SystemProperties"
>>
>> This class uses ImmutableSet.of() for static initialization. I found out
>> that hive-exec actually includes guava and so I guess the classloaders for
>> both copies of ImmutableSet are different leading to the above exception.
>>
>> Is it really necessary to bundle guava (and many others) in the exec jar?
>> Can you provide all classes in regular jars and make a separate maven
>> module for the shaded jar? If not, then can you please relocate those
>> classes as described in
>> http://maven.apache.org/plugins/maven-shade-plugin/examples/class-relocation.html
>>
>> Is there a way to solve this in the short term? Going back to 0.11 seems
>> to be the best solution for now.
>>
>> Thanks
>>
>
>

Re: hive-exec shaded jar a bad idea?

[Edward Capriolo <ed...@gmail.com>]

That is a good idea. I have also considered jar jar, as well as stripping
these things from hive-exec. Guava provides some nice functions for sure
but when it causes classpath issues with other things, you say to your
self, crud all these problems just for Preconditions.checkNotNull() ?


On Thu, Nov 21, 2013 at 7:55 AM, Kristof Vanbecelaere <
kristof.vanbecelaere@gmail.com> wrote:

> Hi,
>
> we are using gradle as our build tool. I need hive-exec to use class
> OrcFile. As soon as I pull in hive-exec 0.12.0 as a dependency my build
> starts to fail with
>
> java.lang.NoClassDefFoundError: Could not initialize class
> org.gradle.internal.SystemProperties"
>
> This class uses ImmutableSet.of() for static initialization. I found out
> that hive-exec actually includes guava and so I guess the classloaders for
> both copies of ImmutableSet are different leading to the above exception.
>
> Is it really necessary to bundle guava (and many others) in the exec jar?
> Can you provide all classes in regular jars and make a separate maven
> module for the shaded jar? If not, then can you please relocate those
> classes as described in
> http://maven.apache.org/plugins/maven-shade-plugin/examples/class-relocation.html
>
> Is there a way to solve this in the short term? Going back to 0.11 seems
> to be the best solution for now.
>
> Thanks
>