You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Andrew Bayer (Created) (JIRA)" <ji...@apache.org> on 2011/12/12 19:03:30 UTC
[jira] [Created] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
----------------------------------------------------------------------------
Key: HIVE-2646
URL: https://issues.apache.org/jira/browse/HIVE-2646
Project: Hive
Issue Type: Bug
Components: Build Infrastructure
Reporter: Andrew Bayer
The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Andrew Bayer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Bayer updated HIVE-2646:
-------------------------------
Priority: Critical (was: Major)
Seeing more problems as a result of the non-standard tarball dependency when trying to build against Hadoop with the 0.23.1 layout. Until this is fixed and Hive properly uses Ivy to depend on jars directly, issues will keep popping up, which will keep requiring hacks to the Hive build classpaths, the Hadoop tarball, or both, none of which would occur if Hive behaved correctly.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Andrew Bayer
> Priority: Critical
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227046#comment-13227046 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
There are several changes to build/dist/lib that may not be intended. In particular there should be no hadoop bundled and other versions not downgrade. Should review to ensure we don't bundle things that are already part of hadoop (such as slf4j etc.)
```
readfactline-lm:hive thw$ ls trunk-before/build/dist/lib/ | sort > oldlib.txt
readfactline-lm:hive thw$ ls trunk/build/dist/lib/ | sort > newlib.txt
readfactline-lm:hive thw$ diff oldlib.txt newlib.txt
8,9d7
< commons-beanutils-1.7.0.jar
< commons-beanutils-core-1.8.0.jar
12d9
< commons-codec-1.4.jar
14d10
< commons-configuration-1.6.jar
16d11
< commons-digester-1.8.jar
19a15
> commons-lang-2.5.jar
23d18
< commons-math-2.1.jar
31a27,29
> ftplet-api-1.0.0.jar
> ftpserver-core-1.0.0.jar
> ftpserver-deprecated-1.0.0-M2.jar
32a31,33
> hadoop-core-0.20.2.jar
> hadoop-test-0.20.2.jar
> hadoop-tools-0.20.2.jar
52,53c53,54
< jackson-core-asl-1.0.1.jar
< jackson-mapper-asl-1.0.1.jar
---
> httpclient-4.0.1.jar
> httpcore-4.0.1.jar
58,59c59,60
< jetty-6.1.26.jar
< jetty-util-6.1.26.jar
---
> jetty-6.1.14.jar
> jetty-util-6.1.14.jar
64d64
< junit-3.8.1.jar
65a66
> junit-4.5.jar
71a73
> mina-core-2.0.0-M5.jar
76d77
< servlet-api-2.5-20081211.jar
77a79,80
> servlet-api-2.5.jar
> slf4j-api-1.5.2.jar
```
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Andrew Bayer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Bayer updated HIVE-2646:
-------------------------------
Attachment: BIGTOP-649.diff.txt
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt, HIVE-2646_LANG.patch
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226940#comment-13226940 ]
Phabricator commented on HIVE-2646:
-----------------------------------
abayer has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
shims/build.xml:28 Nope - will remove.
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225665#comment-13225665 ]
Phabricator commented on HIVE-2646:
-----------------------------------
cwsteinbach has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
shims/ivy.xml:34 Seems like we can get the same benefits by ordering the dependencies listed in each ivy.xml file based on configuration.
@Andrew: Can you look into doing this? Thanks.
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Andrew Bayer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253642#comment-13253642 ]
Andrew Bayer commented on HIVE-2646:
------------------------------------
With the latest patch, those two tests pass.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255173#comment-13255173 ]
Phabricator commented on HIVE-2646:
-----------------------------------
cwsteinbach has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
alter_table_wrong_regex and create_table_wrong_regex both belong in contrib/src/test/queries/clientnegative/. It was a mistake in the first place for them to get committed to the ql directory since nothing else in Hive is supposed to have a dependency on contrib (which is pretty much the whole point of contrib).
The same thing goes for udfnull.q. The point of this test (which you would never be able to tell by looking at it since the original author failed to include any documentation) is that query execution will fail (despite query compilation succeeding) if the query references a UDF which is on the local classpath but not on the cluster classpath, which usually happens when the user forgets to include an 'ADD JAR ...' statement. In order to get this test to pass we need to ensure that hive-contrib*.jar is not on bin/hadoop's classpath.
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226674#comment-13226674 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
also tried
arc export --unified --revision 2133
and then patch command, same result
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Thomas Weise (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise updated HIVE-2646:
-------------------------------
Attachment: HIVE-2646-fixtests.patch
Patch file produced using
arc export --unified --revision 2883
then replaced ivy/common-configurations section with output from
svn diff ivy/common-configurations.xml
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach resolved HIVE-2646.
----------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Committed to trunk. Thanks Andrew and Thomas!
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise updated HIVE-2646:
-------------------------------
Attachment: HIVE-2646-fixtests.patch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251161#comment-13251161 ]
Phabricator commented on HIVE-2646:
-----------------------------------
abayer has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
'k, think I found the problem with hive-builtins - since its jar is built outside of the build-common.xml jar target, it wasn't getting published to the local repo. re: the shims configuration names - they've got to be consistent with the other strings in shims/build.xml, so unless those can change too (and I'm not sure if they can), they need to stay as they are.
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226688#comment-13226688 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
shims/build.xml:28 Is this here still needed?
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.15.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA, cwsteinbach
Exclude hive contrib from test classpaths, bump up commons-codec
version to 1.4 in ivy/libraries.properties.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226685#comment-13226685 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
build.xml conflict is gone, ivy/common-configurations.xml.rej:
```
***************
*** 18,23 ****
<!--these match the Maven configurations-->
<conf name="default" extends="master,compile"/>
<conf name="master" description="contains the artifact but no dependencies"/>
- <conf name="compile" description="contains the artifact but no dependencies"/>
<conf name="runtime" description="runtime but not the artifact"/>
</configurations>
--- 18,31 ----
<!--these match the Maven configurations-->
<conf name="default" extends="master,compile"/>
<conf name="master" description="contains the artifact but no dependencies"/>
+ <conf name="compile" extends="hadoop${hadoop.mr.rev}" description="contains the artifact but no dependencies" visibility="private"/>
<conf name="runtime" description="runtime but not the artifact"/>
+ <conf name="test" extends="hadoop${hadoop.mr.rev}test,compile" visibility="private" />
+ <conf name="hadoop20" visibility="private"/>
+ <conf name="hadoop23" visibility="private"/>
+ <conf name="hadoop20test" visibility="private"/>
+ <conf name="hadoop23test" visibility="private"/>
+ <conf name="hadoop0.20.shim" visibility="private"/>
+ <conf name="hadoop0.20S.shim" visibility="private"/>
+ <conf name="hadoop0.23.shim" visibility="private"/>
</configurations>
```
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Carl Steinbach (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258454#comment-13258454 ]
Carl Steinbach commented on HIVE-2646:
--------------------------------------
@Thomas: I'm going to try to fix the pdk problem later today. In the meantime I was wondering if you could look at the eclipse templates and make sure that they're up to date? Thanks.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227830#comment-13227830 ]
Phabricator commented on HIVE-2646:
-----------------------------------
abayer has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
build-common.xml:384 Ok, still working on getting this bit right. It's worth mentioning that the above as it is in this patch here does NOT include hive-contrib*.jar in the classpath, so that's not an issue.
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256722#comment-13256722 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Carl, thanks for the clarification. I have the 3 tests working now after including contrib into the test classpath and excluding it from the MR classpath.
Looks like there is one more test failing (but not showing up as failure in the test report?):
test:
[junit] Running org.apache.hive.pdk.PluginTest
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 9.877 sec
That's because it does not get the ${test.hadoop.bin.path} setting. I'm trying to fix that. Will see to put up a new patch once that is done.
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise updated HIVE-2646:
-------------------------------
Attachment: (was: HIVE-2646-fixtests.patch)
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2646:
---------------------------------
Fix Version/s: (was: 0.9.0)
0.10.0
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254347#comment-13254347 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
The following 3 tests fail:
testNegativeCliDriver_alter_table_wrong_regex
testNegativeCliDriver_create_table_wrong_regex
testNegativeCliDriver_udfnull
Looks like SerDe/UDFs are not found:
```Caused by: org.apache.hadoop.hive.serde2.SerDeException: SerDe org.apache.hadoop.hive.contrib.serde2.RegexSerDe does not exist
at org.apache.hadoop.hive.serde2.SerDeUtils.lookupDeserializer(SerDeUtils.java:85)
at org.apache.hadoop.hive.ql.exec.DDLTask.validateSerDe(DDLTask.java:3085)```
...
```2012-04-15 00:59:58,643 INFO exec.FunctionTask (FunctionTask.java:createFunction(87)) - create function: java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.udf.example.UDFExampleArraySum
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
```
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225502#comment-13225502 ]
Phabricator commented on HIVE-2646:
-----------------------------------
cwsteinbach has requested changes to the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
build-common.xml:68 Please move this to build.properties
build-common.xml:86 Is this specific to 0.23? If so please add a comment.
build-common.xml:148 Since Hadoop 1.0.0 is now a reality, I think it would be good to always prefix any occurrences of version numbers with the major number, e.g. "0.20" instead of "20"
build-common.xml:154 We should be consistent about the naming scheme in relation to versions, e.g. "ivy-retrieve-hadoop-0.20", "ivy-resolve-hadoop-0.20", "hadoop-0.20-shim", etc.
build.properties:20 Please add some comments explaining what this property does.
testutils/hadoop:48 We should probably just insist that $HIVE_TEST_CLASSPATH is set and error out if it isn't.
build.xml:700 This target is broken.
build.xml:740 This target is broken.
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Andrew Bayer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Bayer updated HIVE-2646:
-------------------------------
Status: Patch Available (was: Open)
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Carl Steinbach (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258452#comment-13258452 ]
Carl Steinbach commented on HIVE-2646:
--------------------------------------
@Thomas: I tried running the tests on Jenkins and noticed that the build is failing despite all tests passing. It looks like this is because compilation of the pdk subproject is failing:
{noformat}
BUILD FAILED
/var/lib/hudson/production/workspace/carl-HIVE-patch-test1/build.xml:316: The following error occurred while executing this line:
/var/lib/hudson/production/workspace/carl-HIVE-patch-test1/build.xml:321: The following error occurred while executing this line:
/var/lib/hudson/production/workspace/carl-HIVE-patch-test1/pdk/build.xml:52: The following error occurred while executing this line:
/var/lib/hudson/production/workspace/carl-HIVE-patch-test1/build/dist/scripts/pdk/build-plugin.xml:58: /var/lib/hudson/production/workspace/carl-HIVE-patch-test1/build/pdk/test-plugin/${build.ivy.lib.dir}/default does not exist.
at org.apache.tools.ant.types.AbstractFileSet.getDirectoryScanner(AbstractFileSet.java:474)
at org.apache.tools.ant.types.FileSet.iterator(FileSet.java:69)
at org.apache.tools.ant.types.resources.Union.getCollection(Union.java:123)
at org.apache.tools.ant.types.resources.Union.getCollection(Union.java:107)
at org.apache.tools.ant.types.resources.BaseResourceCollectionContainer.cacheCollection(BaseResourceCollectionContainer.java:265)
at org.apache.tools.ant.types.resources.BaseResourceCollectionContainer.iterator(BaseResourceCollectionContainer.java:142)
at org.apache.tools.ant.types.Path.iterator(Path.java:704)
at org.apache.tools.ant.types.Path.iterator(Path.java:698)
at org.apache.tools.ant.types.resources.Union.getCollection(Union.java:123)
at org.apache.tools.ant.types.resources.Union.list(Union.java:86)
at org.apache.tools.ant.types.Path.list(Path.java:372)
at org.apache.tools.ant.types.Path.addExisting(Path.java:331)
at org.apache.tools.ant.types.Path.addExisting(Path.java:319)
at org.apache.tools.ant.types.Path.concatSpecialPath(Path.java:566)
at org.apache.tools.ant.types.Path.concatSystemClasspath(Path.java:526)
at org.apache.tools.ant.taskdefs.compilers.DefaultCompilerAdapter.getCompileClasspath(DefaultCompilerAdapter.java:155)
at org.apache.tools.ant.taskdefs.compilers.DefaultCompilerAdapter.setupJavacCommandlineSwitches(DefaultCompilerAdapter.java:183)
at org.apache.tools.ant.taskdefs.compilers.DefaultCompilerAdapter.setupModernJavacCommandlineSwitches(DefaultCompilerAdapter.java:321)
at org.apache.tools.ant.taskdefs.compilers.DefaultCompilerAdapter.setupModernJavacCommand(DefaultCompilerAdapter.java:368)
at org.apache.tools.ant.taskdefs.compilers.Javac13.execute(Javac13.java:48)
at org.apache.tools.ant.taskdefs.Javac.compile(Javac.java:1097)
at org.apache.tools.ant.taskdefs.Javac.execute(Javac.java:906)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:68)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.taskdefs.MacroInstance.execute(MacroInstance.java:398)
at net.sf.antcontrib.logic.ForTask.doSequentialIteration(ForTask.java:259)
at net.sf.antcontrib.logic.ForTask.doToken(ForTask.java:268)
at net.sf.antcontrib.logic.ForTask.doTheTasks(ForTask.java:324)
at net.sf.antcontrib.logic.ForTask.execute(ForTask.java:244)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
at org.apache.tools.ant.Project.executeTarget(Project.java:1366)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
at org.apache.tools.ant.Main.runBuild(Main.java:801)
at org.apache.tools.ant.Main.startAnt(Main.java:218)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
{noformat}
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254252#comment-13254252 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
ivy/common-configurations.xml:32 Applying the patch (still) fails for this file and after download from here I still need to remove that last line " No newline ..." - Can you eliminate this from the patch?
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2646:
-----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to trunk and 0.9 branch. Thanks, Thomas!
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt, HIVE-2646_LANG.patch
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Andrew Bayer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Bayer updated HIVE-2646:
-------------------------------
Affects Version/s: 0.8.0
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.2.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA
Was missing testutils/hadoop.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270922#comment-13270922 ]
Hudson commented on HIVE-2646:
------------------------------
Integrated in Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #10 (See [https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/10/])
HIVE-2646 : merged in from trunk. Fix for LANG setting for test environment. (Thomas Weise via Ashutosh Chauhan) (Revision 1335774)
Result = FAILURE
hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1335774
Files :
* /hive/branches/branch-0.9
* /hive/branches/branch-0.9/build-common.xml
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt, HIVE-2646_LANG.patch
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226650#comment-13226650 ]
Phabricator commented on HIVE-2646:
-----------------------------------
abayer has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
shims/ivy.xml:34 Sure thing.
build-common.xml:86 It is - will do so.
EDIT: actually, no, it isn't. It was originally but is general now. Will remove the misleading comment.
build-common.xml:154 ivy-{resolve,retrieve}-hadoop20 is an obsolete bit of cruft - removing it. I've also replaced the ivy-{resolve,retrieve}-hadoop0.{20,20S,23}-shim targets with ivy-{resolve,retrieve}-hadoop-shim, which is called with a parameter that specifies the shim conf to get.
build.xml:700 Working for me?
build.xml:740 Again - works for me, with "ant clean binary"
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225579#comment-13225579 ]
Phabricator commented on HIVE-2646:
-----------------------------------
cwsteinbach has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
build.xml:542 This patch breaks the eclipse classpath template. Can you please investigate the feasibility of intregrating IvyDE into the build? HIVE-2739 covers that task, but it would be great to get this all done in one patch. Note that ZK recently added support for IvyDE in ZOOKEEPER-1178, so you may be able to look at that patch for hints.
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225629#comment-13225629 ]
Phabricator commented on HIVE-2646:
-----------------------------------
cwsteinbach has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
shims/ivy.xml:34 Aren't configurations the designated mechanism for handling situations like this in Ivy? I don't see how breaking this into multiple files will improve the situation. Won't that mean that we'll have to update multiple files every time a new dependency is added instead of just one?
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise updated HIVE-2646:
-------------------------------
Fix Version/s: 0.9.1
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Andrew Bayer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234900#comment-13234900 ]
Andrew Bayer commented on HIVE-2646:
------------------------------------
With the latest patch (I'll have a rebase on top of trunk posted later today, but there were no differences in build results), we've still got just two consistent test failures (org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_udfnull and org.apache.hadoop.hive.serde2.lazy.TestLazySimpleSerDe.testLazySimpleSerDe). What more do we need to do to go live with this?
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise reopened HIVE-2646:
--------------------------------
Reopening for backport to 0.9 branch.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Thomas Weise (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257474#comment-13257474 ]
Thomas Weise commented on HIVE-2646:
------------------------------------
The patch needs to be applied through arc:
arc patch D2883
chmod u+x testutils/hadoop
svn propset svn:executable yes testutils/hadoop
Al least for me, arc patch will choke on ivy/common-configurations.xml - download the file from phabricator directly. I had the problem with Andrew's patch also, so I'm not sure whether this is just me or an arc issue.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.12.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA, cwsteinbach
Rebased onto latest trunk.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise updated HIVE-2646:
-------------------------------
Status: Patch Available (was: Reopened)
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt, HIVE-2646_LANG.patch
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise reopened HIVE-2646:
--------------------------------
Reopening to fix test failure on 0.9 branch.
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/7/testReport/
Earlier changes erased the LANG setting, while trunk build masks this by exporting it, 0.9.1 build does not. Thanks to Ashutosh for diagnosing the issue.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt, HIVE-2646_LANG.patch
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227044#comment-13227044 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
compile-test fails when running
ant clean package test
```
compile-test:
[echo] Project: contrib
[javac] Compiling 2 source files to /Users/thw/java/hive/trunk/build/contrib/test/classes
[javac] /Users/thw/java/hive/trunk/contrib/src/test/org/apache/hadoop/hive/contrib/mr/TestGenericMR.java:117: cannot find symbol
[javac] symbol : class Mapper
[javac] location: class org.apache.hadoop.hive.contrib.mr.TestGenericMR
[javac] private Mapper identityMapper() {
[javac] ^
...
[javac] ^
[javac] 15 errors
BUILD FAILED
/Users/thw/java/hive/trunk/build.xml:306: The following error occurred while executing this line:
/Users/thw/java/hive/trunk/build.xml:117: The following error occurred while executing this line:
/Users/thw/java/hive/trunk/build-common.xml:269: Compile failed; see the compiler error output for details.
```
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2883.1.patch
thw requested code review of "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA
https://issues.apache.org/jira/browse/HIVE-2646
Update to fix test failures.
The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D2883
AFFECTED FILES
shims/ivy.xml
shims/build.xml
builtins/ivy.xml
builtins/build.xml
build.properties
hbase-handler/ivy.xml
hbase-handler/build.xml
build.xml
testutils/hadoop
jdbc/ivy.xml
jdbc/build.xml
metastore/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
build-common.xml
hwi/ivy.xml
hwi/build.xml
common/ivy.xml
service/ivy.xml
service/build.xml
contrib/ivy.xml
contrib/build.xml
serde/ivy.xml
cli/ivy.xml
ql/ivy.xml
ql/build.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/6567/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.7.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA, cwsteinbach
Changes to test classpaths to get compilation working properly for
shims and contrib tests.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226691#comment-13226691 ]
Phabricator commented on HIVE-2646:
-----------------------------------
abayer has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Not sure why that doesn't apply cleanly.
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226943#comment-13226943 ]
Phabricator commented on HIVE-2646:
-----------------------------------
abayer has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
build.xml:542 This may take a bit for me to figure out, as I'm not at all familiar with the eclipse stuff.
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254493#comment-13254493 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Only udfnull fails after adding back contrib into the test class path (resolves other 2 failures).
```
ant test -Dtestcase=TestNegativeCliDriver -Dqfile=udfnull.q -Dtest.silent=false
```
It looks from the hive history though as if the udfnull queries complete. Could not find out what is wrong with it yet.
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Carl Steinbach (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252905#comment-13252905 ]
Carl Steinbach commented on HIVE-2646:
--------------------------------------
bq. org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_udfnull
TestNegativeCliDriver.udfnull fails consistently for me. The cause of the failure is that the hive-contrib and hive_contrib JARs are getting included on the HIVE_HADOOP_TEST_CLASSPATH.
@Andrew: Can you please modify the patch so that the hive-contrib and hive_contrib JARs are not included in HIVE_HADOOP_TEST_CLASSPATH? Thanks.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258706#comment-13258706 ]
Thomas Weise commented on HIVE-2646:
------------------------------------
To note, all of that under the radar of testreport?
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise updated HIVE-2646:
-------------------------------
Attachment: HIVE-2646_LANG.patch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt, HIVE-2646_LANG.patch
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227771#comment-13227771 ]
Phabricator commented on HIVE-2646:
-----------------------------------
abayer has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
build-common.xml:384 Adding a new hadoop.test.classpath path + hadoop.testcp property that'll use the shims Ivy configuration for the version of Hadoop we're testing against (so that we actually get all the jars properly) and the Hive jars. We can add to that if needed.
build-common.xml:79 Done.
build-common.xml:409 Done.
ivy/common-configurations.xml:24 Done.
pdk/scripts/build-plugin.xml:36 Done.
testutils/hadoop:47 Done.
testutils/hadoop:50 Done.
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223791#comment-13223791 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
The tarball mechanics would extract the tarball into a version specific subdirectory of build/hadoopcore. The ivy equivalent could ivy-retrieve the hadoop artifacts into a similar folder structure and then reference it from elsewhere as it was.
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226673#comment-13226673 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
arc patch --revision 2133
gives me two rejects (applying to trunk head)?
patching file build-common.xml
patching file build.properties
patching file build.xml
Hunk #1 FAILED at 684.
1 out of 1 hunk FAILED -- saving rejects to file build.xml.rej
patching file builtins/build.xml
patching file builtins/ivy.xml
patching file cli/ivy.xml
patching file common/ivy.xml
patching file contrib/build.xml
patching file contrib/ivy.xml
patching file hbase-handler/build.xml
patching file hbase-handler/ivy.xml
patching file hwi/build.xml
patching file hwi/ivy.xml
patching file ivy/common-configurations.xml
Hunk #1 FAILED at 18.
1 out of 1 hunk FAILED -- saving rejects to file ivy/common-configurations.xml.rej
patching file ivy/ivysettings.xml
patching file ivy/libraries.properties
patching file jdbc/build.xml
patching file jdbc/ivy.xml
patching file metastore/ivy.xml
patching file pdk/ivy.xml
patching file pdk/scripts/build-plugin.xml
patching file ql/build.xml
patching file ql/ivy.xml
patching file serde/ivy.xml
patching file service/build.xml
patching file service/ivy.xml
patching file shims/build.xml
patching file shims/ivy.xml
patching file testutils/hadoop
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.9.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA, cwsteinbach
Working on test failures - still seeing a good number, but making progress.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228416#comment-13228416 ]
Phabricator commented on HIVE-2646:
-----------------------------------
abayer has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Latest patch had four test failures, only one of which I've seen more than once so far. That one is org.apache.hadoop.hive.serde2.lazy.TestLazySimpleSerDe.testLazySimpleSerDe. The others are org.apache.hadoop.hive.metastore.TestMarkPartition.testMarkingPartitionSet, org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample6, and org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_udfnull.
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.1.patch
abayer requested code review of "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA
HIVE-2646, HIVE-2745. Rework Hive dependency management to depend on jars directly, rather than via tarballs.
The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/4647/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254417#comment-13254417 ]
Phabricator commented on HIVE-2646:
-----------------------------------
abayer has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Looks like the ql tests *do* need hive contrib in the classpath, at least for those tests. Hrm. Carl, any thoughts?
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271041#comment-13271041 ]
Hudson commented on HIVE-2646:
------------------------------
Integrated in Hive-trunk-h0.21 #1419 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1419/])
HIVE-2646 : Fix for LANG setting while running tests. (Thomas Weise via Ashutosh Chauhan) (Revision 1335771)
Result = SUCCESS
hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1335771
Files :
* /hive/trunk/build-common.xml
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt, HIVE-2646_LANG.patch
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2883.2.patch
thw updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA
Fix for pdk test target.
REVISION DETAIL
https://reviews.facebook.net/D2883
AFFECTED FILES
shims/ivy.xml
shims/build.xml
builtins/ivy.xml
builtins/build.xml
build.properties
hbase-handler/ivy.xml
hbase-handler/build.xml
build.xml
testutils/hadoop
jdbc/ivy.xml
jdbc/build.xml
metastore/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
build-common.xml
hwi/ivy.xml
hwi/build.xml
common/ivy.xml
service/ivy.xml
service/build.xml
contrib/ivy.xml
contrib/build.xml
serde/ivy.xml
cli/ivy.xml
ql/ivy.xml
ql/build.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
pdk/build.xml
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Thomas Weise (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257833#comment-13257833 ]
Thomas Weise commented on HIVE-2646:
------------------------------------
All test pass in latest run:
Tests Failures Errors Success rate Time
1678 0 0 100.00% 19245.326
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise updated HIVE-2646:
-------------------------------
Attachment: HIVE-2646-fixtests.patch
Patch with fix for pdk test.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258695#comment-13258695 ]
Thomas Weise commented on HIVE-2646:
------------------------------------
With latest patch, the build failure is gone. However, what I see now is that the same test runs twice, it passes at the beginning and then later it fails (different ant target chain).
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251140#comment-13251140 ]
Phabricator commented on HIVE-2646:
-----------------------------------
cwsteinbach has requested changes to the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Executing the following ant command results in a failure:
% ant clean package test -Dtestcase=TestNegativeCliDriver -Dqfile=autolocal1.q
WARNINGS
[ivy:resolve] module not found: org.apache.hive#hive-builtins;0.10.0-SNAPSHOT
INLINE COMMENTS
ivy/common-configurations.xml:28 Is it possible to make these names consistent, e.g. change "hadoop0.20.shim" to "hadoop20.shim"?
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.5.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA, cwsteinbach
Rebased on trunk - got the build.xml conflict when rebasing, but
didn't see any issues with ivy/common-configurations.xml.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.6.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA, cwsteinbach
Removing obsolete notclasspath from shims/build.xml. Also removed
shims/build.xml's own classpath definition, as it was redundant.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260134#comment-13260134 ]
Hudson commented on HIVE-2646:
------------------------------
Integrated in Hive-trunk-h0.21 #1391 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1391/])
HIVE-2646. Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs (Andrew Bayer and Thomas Weise) (Revision 1329381)
Result = FAILURE
cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1329381
Files :
* /hive/trunk/build-common.xml
* /hive/trunk/build.properties
* /hive/trunk/build.xml
* /hive/trunk/builtins/build.xml
* /hive/trunk/builtins/ivy.xml
* /hive/trunk/cli/ivy.xml
* /hive/trunk/common/ivy.xml
* /hive/trunk/contrib/build.xml
* /hive/trunk/contrib/ivy.xml
* /hive/trunk/hbase-handler/build.xml
* /hive/trunk/hbase-handler/ivy.xml
* /hive/trunk/hwi/build.xml
* /hive/trunk/hwi/ivy.xml
* /hive/trunk/ivy/common-configurations.xml
* /hive/trunk/ivy/ivysettings.xml
* /hive/trunk/ivy/libraries.properties
* /hive/trunk/jdbc/build.xml
* /hive/trunk/jdbc/ivy.xml
* /hive/trunk/metastore/ivy.xml
* /hive/trunk/pdk/build.xml
* /hive/trunk/pdk/ivy.xml
* /hive/trunk/pdk/scripts/build-plugin.xml
* /hive/trunk/ql/build.xml
* /hive/trunk/ql/ivy.xml
* /hive/trunk/serde/ivy.xml
* /hive/trunk/service/build.xml
* /hive/trunk/service/ivy.xml
* /hive/trunk/shims/build.xml
* /hive/trunk/shims/ivy.xml
* /hive/trunk/testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Thomas Weise (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise updated HIVE-2646:
-------------------------------
Attachment: (was: HIVE-2646-fixtests.patch)
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Andrew Bayer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Bayer updated HIVE-2646:
-------------------------------
Attachment: HIVE-2646.diff.txt
This should apply cleanly to trunk, and I think it also encompasses the fixes for HIVE-2745.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.11.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA, cwsteinbach
A bunch of test failures fixed, a few issues addressed. Still passing
the whole test.classpath to testutils/hadoop - things went weird if I didn't.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Eli Collins (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins reassigned HIVE-2646:
---------------------------------
Assignee: Andrew Bayer
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2883.3.patch
thw updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA
Fix pdk/PluginTest
REVISION DETAIL
https://reviews.facebook.net/D2883
AFFECTED FILES
shims/ivy.xml
shims/build.xml
builtins/ivy.xml
builtins/build.xml
build.properties
hbase-handler/ivy.xml
hbase-handler/build.xml
build.xml
testutils/hadoop
jdbc/ivy.xml
jdbc/build.xml
metastore/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
build-common.xml
hwi/ivy.xml
hwi/build.xml
common/ivy.xml
service/ivy.xml
service/build.xml
contrib/ivy.xml
contrib/build.xml
serde/ivy.xml
cli/ivy.xml
ql/ivy.xml
ql/build.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
pdk/build.xml
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.10.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA, cwsteinbach
Got a bunch more tests passing, especially in TestCLIDriver.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Andrew Bayer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223424#comment-13223424 ]
Andrew Bayer commented on HIVE-2646:
------------------------------------
fwiw, Carl actually implemented a reworking of the test execution to work around the bin/hadoop from the tarball issue, and that's part of this patch.
This doesn't change anything about building shims - rather, it changes how the classpath for building the shims is assembled. Instead of blowing up a tarball to get the jars for the various Hadoop versions, it just gets the jars.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255175#comment-13255175 ]
Phabricator commented on HIVE-2646:
-----------------------------------
cwsteinbach has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
build-common.xml:367 Indentation.
build-common.xml:379 ${testcp} is not defined.
build-common.xml:385 $HIVE_HADOOP_TEST_CLASSPATH is supposed to be the classpath that bin/hadoop would construct for itself without any outside input, e.g. hadoop-* JAR files and any dependencies. It should not include any Hive JARs or dependencies.
testutils/hadoop:53 If $HADOOP_CLASSPATH is defined, we should tack it onto the end of the CLASSPATH at this point.
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267201#comment-13267201 ]
Hudson commented on HIVE-2646:
------------------------------
Integrated in Hive-0.9.1-SNAPSHOT-h0.21 #3 (See [https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/3/])
HIVE-2646 Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs. Merged in from trunk. (Andrew Bayer and Thomas Weise) (Revision 1333094)
Result = ABORTED
hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1333094
Files :
* /hive/branches/branch-0.9
* /hive/branches/branch-0.9/build-common.xml
* /hive/branches/branch-0.9/build.properties
* /hive/branches/branch-0.9/build.xml
* /hive/branches/branch-0.9/builtins/build.xml
* /hive/branches/branch-0.9/builtins/ivy.xml
* /hive/branches/branch-0.9/cli/ivy.xml
* /hive/branches/branch-0.9/common/ivy.xml
* /hive/branches/branch-0.9/contrib/build.xml
* /hive/branches/branch-0.9/contrib/ivy.xml
* /hive/branches/branch-0.9/hbase-handler/build.xml
* /hive/branches/branch-0.9/hbase-handler/ivy.xml
* /hive/branches/branch-0.9/hwi/build.xml
* /hive/branches/branch-0.9/hwi/ivy.xml
* /hive/branches/branch-0.9/ivy/common-configurations.xml
* /hive/branches/branch-0.9/ivy/ivysettings.xml
* /hive/branches/branch-0.9/ivy/libraries.properties
* /hive/branches/branch-0.9/jdbc/build.xml
* /hive/branches/branch-0.9/jdbc/ivy.xml
* /hive/branches/branch-0.9/metastore/ivy.xml
* /hive/branches/branch-0.9/pdk/build.xml
* /hive/branches/branch-0.9/pdk/ivy.xml
* /hive/branches/branch-0.9/pdk/scripts/build-plugin.xml
* /hive/branches/branch-0.9/ql/build.xml
* /hive/branches/branch-0.9/ql/ivy.xml
* /hive/branches/branch-0.9/serde/ivy.xml
* /hive/branches/branch-0.9/service/build.xml
* /hive/branches/branch-0.9/service/ivy.xml
* /hive/branches/branch-0.9/shims/build.xml
* /hive/branches/branch-0.9/shims/ivy.xml
* /hive/branches/branch-0.9/testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225646#comment-13225646 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
shims/ivy.xml:34 My comment refers only to the hadoop dependencies below. You will never have hadoop23 and hadoop20 configuration at the same time. Below is what we did in Pig, but it gets really cluttered because now you have the stuff for 0.20 and 0.23, which is mutually exclusive, in one file.
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254255#comment-13254255 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
testutils/hadoop:1 had to make this file executable after the patch
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258867#comment-13258867 ]
Thomas Weise commented on HIVE-2646:
------------------------------------
ant clean package package test
BUILD SUCCESSFUL
Total time: 315 minutes 27 seconds
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Kanna Karanam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291928#comment-13291928 ]
Kanna Karanam commented on HIVE-2646:
-------------------------------------
I downloaded the Hadoop & Hive source from apache and made some changes in Hadoop internally. Now I am trying to run hive unit tests against my changes (internal Hadoop). Are there any configuration settings in hive (unit tests) to configure and run against local Hadoop jars.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt, HIVE-2646_LANG.patch
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266729#comment-13266729 ]
Hudson commented on HIVE-2646:
------------------------------
Integrated in Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #3 (See [https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/3/])
HIVE-2646 Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs. Merged in from trunk. (Andrew Bayer and Thomas Weise) (Revision 1333094)
Result = FAILURE
hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1333094
Files :
* /hive/branches/branch-0.9
* /hive/branches/branch-0.9/build-common.xml
* /hive/branches/branch-0.9/build.properties
* /hive/branches/branch-0.9/build.xml
* /hive/branches/branch-0.9/builtins/build.xml
* /hive/branches/branch-0.9/builtins/ivy.xml
* /hive/branches/branch-0.9/cli/ivy.xml
* /hive/branches/branch-0.9/common/ivy.xml
* /hive/branches/branch-0.9/contrib/build.xml
* /hive/branches/branch-0.9/contrib/ivy.xml
* /hive/branches/branch-0.9/hbase-handler/build.xml
* /hive/branches/branch-0.9/hbase-handler/ivy.xml
* /hive/branches/branch-0.9/hwi/build.xml
* /hive/branches/branch-0.9/hwi/ivy.xml
* /hive/branches/branch-0.9/ivy/common-configurations.xml
* /hive/branches/branch-0.9/ivy/ivysettings.xml
* /hive/branches/branch-0.9/ivy/libraries.properties
* /hive/branches/branch-0.9/jdbc/build.xml
* /hive/branches/branch-0.9/jdbc/ivy.xml
* /hive/branches/branch-0.9/metastore/ivy.xml
* /hive/branches/branch-0.9/pdk/build.xml
* /hive/branches/branch-0.9/pdk/ivy.xml
* /hive/branches/branch-0.9/pdk/scripts/build-plugin.xml
* /hive/branches/branch-0.9/ql/build.xml
* /hive/branches/branch-0.9/ql/ivy.xml
* /hive/branches/branch-0.9/serde/ivy.xml
* /hive/branches/branch-0.9/service/build.xml
* /hive/branches/branch-0.9/service/ivy.xml
* /hive/branches/branch-0.9/shims/build.xml
* /hive/branches/branch-0.9/shims/ivy.xml
* /hive/branches/branch-0.9/testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.14.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA, cwsteinbach
Added ivy:publish for builtins, rebased on trunk.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Thomas Weise (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise updated HIVE-2646:
-------------------------------
Fix Version/s: 0.9.0
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Edward Capriolo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223318#comment-13223318 ]
Edward Capriolo commented on HIVE-2646:
---------------------------------------
Hive-2745 is really baked in. It is more then just built changes in the hive code we build a bin/hadoop jar command and fork (this is really lame). The non-standard tarball dependency is mostly caused by the fact that upstream changes effect Hive. Hive does heavy work to maintain backwards compatibility all the way to hadoop 0.20.2. Pig takes the other approach but you almost always need to compile pig for your version of hadoop. You are right to say that as long as upstream keeps changing we have to keep hacking. I will look at the patch I am not sure what is the best way to handle supporting versions of hadoop that are not even released yet.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise updated HIVE-2646:
-------------------------------
Attachment: (was: HIVE-2646-fixtests.patch)
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Andrew Bayer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Bayer updated HIVE-2646:
-------------------------------
Attachment: (was: BIGTOP-649.diff.txt)
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt, HIVE-2646_LANG.patch
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258705#comment-13258705 ]
Thomas Weise commented on HIVE-2646:
------------------------------------
Here is the test execution failure (for the second execution of org.apache.hive.pdk.PluginTest)
{code}
Failed to load Hive builtin functions
java.lang.RuntimeException: Failed to load Hive builtin functions
at org.apache.hadoop.hive.ql.session.SessionState.<init>(SessionState.java:205)
at org.apache.hadoop.hive.cli.CliSessionState.<init>(CliSessionState.java:81)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:579)
at org.apache.hive.pdk.PluginTest.runHive(PluginTest.java:70)
at org.apache.hive.pdk.PluginTest$PluginGlobalSetup.setUp(PluginTest.java:177)
at junit.extensions.TestSetup$1.protect(TestSetup.java:22)
at junit.extensions.TestSetup.run(TestSetup.java:27)
Caused by: java.lang.ClassNotFoundException: org.apache.hive.builtins.BuiltinUtils
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
{code}
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2646:
---------------------------------
Status: Open (was: Patch Available)
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Thomas Weise (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270140#comment-13270140 ]
Thomas Weise commented on HIVE-2646:
------------------------------------
See HIVE-2811. Test: ant test -Dtestcase=TestCliDriver -Dqfile=inputddl5.q -Dtest.silent=false
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt, HIVE-2646_LANG.patch
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.4.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA, cwsteinbach
Redid the shim ivy targets, clarified a few things.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225635#comment-13225635 ]
Phabricator commented on HIVE-2646:
-----------------------------------
abayer has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
I'll rework the ivy-{retrieve,resolve}-hadoopX targets get generalized - that should be easy. And I agree - the configuration switch is the right way to handle this.
I'll take a look at IvyDE.
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271149#comment-13271149 ]
Hudson commented on HIVE-2646:
------------------------------
Integrated in Hive-0.9.1-SNAPSHOT-h0.21 #10 (See [https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/10/])
HIVE-2646 : merged in from trunk. Fix for LANG setting for test environment. (Thomas Weise via Ashutosh Chauhan) (Revision 1335774)
Result = FAILURE
hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1335774
Files :
* /hive/branches/branch-0.9
* /hive/branches/branch-0.9/build-common.xml
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt, HIVE-2646_LANG.patch
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227746#comment-13227746 ]
Phabricator commented on HIVE-2646:
-----------------------------------
cwsteinbach has requested changes to the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
testutils/hadoop:50 If $HIVE_TEST_CLASSPATH is not set we should throw an error and exit instead of using a hardcoded default classpath.
testutils/hadoop:47 I think we should change the name of this env var to something like HIVE_HADOOP_TEST_CLASSPATH in order to make it clear that we're using it with bin/hadoop.
build-common.xml:409 Let's make it possible to override this value by adding test.hadoop.bin.path=${hive.root}/testutils/hadoop to build.properties, and then reference ${test.hadoop.bin.path} here.
build-common.xml:384 This is probably causing some test failures since some tests expect hive-contrib-*.jar to *not* be on the classpath. I think we should create a classpath specifically for passing to the bin/hadoop consisting of the hive jars, the hadoop jars, and relevant dependencies, and not include hive-contrib-*.jar.
build-common.xml:79 alphabetical order?
ivy/common-configurations.xml:24 Are 'hadoop20' and 'hadoop23' compile configurations? If so can we change the name to 'hadoop20.compile' and 'hadoop23.compile', and change the names of the test configurations to "hadoop20.test" and "hadoop23.test", or something that's consistent?
pdk/scripts/build-plugin.xml:36 Please define ivy.lib.default.dir and reference that instead.
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Andrew Bayer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253643#comment-13253643 ]
Andrew Bayer commented on HIVE-2646:
------------------------------------
...but make sure to nuke your ~/.ivy2/cache/org.apache.hive before rebuilding, or you'll end up with stale Ivy artifacts pointing to commons-codec 1.3.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Thomas Weise (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Weise updated HIVE-2646:
-------------------------------
Attachment: HIVE-2646-fixtests.patch
Patch to address unit test failures. This should apply cleanly to trunk (arc patch doesn't).
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Andrew Bayer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259845#comment-13259845 ]
Andrew Bayer commented on HIVE-2646:
------------------------------------
+1 indeed. =)
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223784#comment-13223784 ]
Phabricator commented on HIVE-2646:
-----------------------------------
abayer has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Re: the dependency on Hadoop being repeated - I don't remember what was happening, but when I made the shims dependency not have transitive="false", things went bad, thereby requiring the hadoop dependency get specified everywhere. I'll play with that.
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.3.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA
Bumped Hadoop 0.20 and 0.23 versions to ones with properly published
jars. Reworked shims dependency logic to not require if/else. Switched
to transient dependencies being used to pick up hadoop dependencies
downstream of hive-common.
Note that "ant very-clean" needs to be run when switching from
building against 0.20, 1.0, or 0.23 to another on the same machine.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223769#comment-13223769 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
It can't be good to repeat the hadoop ivy dependencies for every module, when before that was all in one location (the shims module). The shims module should continue to deal with the hadoop dependency magic and everything else depend on shims.
Within shims, it would be nice to not see if-then-else for shim versions, but instead parameterized includes so that each shim version has a separate file for its hadoop dependencies (one ivy file for 0.20, one for 0.20S and one for 0.23).
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Andrew Bayer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Bayer updated HIVE-2646:
-------------------------------
Attachment: HIVE-2646-fixtests.txt
Reuploading patch with ASF license checked.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Carl Steinbach (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252929#comment-13252929 ]
Carl Steinbach commented on HIVE-2646:
--------------------------------------
Here's the exception stack from the TestLazySimpleSerDe test failure:
{noformat}
[junit] Running org.apache.hadoop.hive.serde2.lazy.TestLazySimpleSerDe
[junit] java.lang.ArrayIndexOutOfBoundsException: -65
[junit] at org.apache.commons.codec.binary.Base64.isBase64(Base64.java:137)
[junit] at org.apache.commons.codec.binary.Base64.isArrayByteBase64(Base64.java:163)
[junit] at org.apache.hadoop.hive.serde2.lazy.LazyBinary.init(LazyBinary.java:52)
[junit] at org.apache.hadoop.hive.serde2.lazy.LazyStruct.uncheckedGetField(LazyStruct.java:219)
[junit] at org.apache.hadoop.hive.serde2.lazy.LazyStruct.getField(LazyStruct.java:192)
[junit] at org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldData(LazySimpleStructObjectInspector.java:188)
[junit] at org.apache.hadoop.hive.serde2.lazy.TestLazySimpleSerDe.deserializeAndSerialize(TestLazySimpleSerDe.java:92)
[junit] at org.apache.hadoop.hive.serde2.lazy.TestLazySimpleSerDe.testLazySimpleSerDe(TestLazySimpleSerDe.java:73)
{noformat}
The ArrayIndexOutOfBoundsException thrown by Base64.isBase64() is due to a bug in commons-codec 1.3. This was fixed in CODEC-22 in commons-codec 1.4. Right now the test classpath contains copies of versions 1.3 and 1.4 of commons-codec, and the version 1.3 JAR precedes version 1.4. Ideally we can prevent the 1.3 JAR from getting pulled down entirely, but at the least we need to make sure it appears on the classpath after the v1.4 JAR.
@Andrew: can you please look into this? Thanks.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259842#comment-13259842 ]
Carl Steinbach commented on HIVE-2646:
--------------------------------------
+1.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan resolved HIVE-2646.
------------------------------------
Resolution: Fixed
Merged in 0.9 branch.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-2646-fixtests.patch, HIVE-2646-fixtests.txt, HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.15.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.D2883.1.patch, HIVE-2646.D2883.2.patch, HIVE-2646.D2883.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227051#comment-13227051 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
As for the reject for ivy/common-configurations.xml (which is still an issue in the latest patch), after applying the patch with arc I downloaded the raw file from here (view options). Then created new diff and applied it to a clean working copy - works w/o rejects. Perhaps you could try the same locally and see whether that would produce a different patch?
REVISION DETAIL
https://reviews.facebook.net/D2133
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop
should depend on jars directly, not tarballs
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225581#comment-13225581 ]
Phabricator commented on HIVE-2646:
-----------------------------------
thw has commented on the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
INLINE COMMENTS
shims/ivy.xml:34 Can we make separate ivy files for 0.20 and 0.23 and include dynamically? This would eliminate the need for the hadoopXXX configuration names.
build-common.xml:154 Please make a single ivy-retrieve-hadoop target or macro and parameterize the shim name.
REVISION DETAIL
https://reviews.facebook.net/D2133
BRANCH
HIVE-2646-dev-branch
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.8.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA, cwsteinbach
A number of test compilation/execution tweaks, modifying the
build/ivy/lib/default contents (and therefore the build/dist/lib
contents) to not include transitive dependencies of Hadoop, and
excluding hadoop-*.jar from build/dist/lib.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Priority: Critical
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2646) Hive Ivy dependencies on Hadoop should
depend on jars directly, not tarballs
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2646:
------------------------------
Attachment: HIVE-2646.D2133.13.patch
abayer updated the revision "HIVE-2646 [jira] Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs".
Reviewers: JIRA, cwsteinbach
Rebasing on trunk again.
REVISION DETAIL
https://reviews.facebook.net/D2133
AFFECTED FILES
build-common.xml
build.properties
build.xml
builtins/build.xml
builtins/ivy.xml
cli/ivy.xml
common/ivy.xml
contrib/build.xml
contrib/ivy.xml
hbase-handler/build.xml
hbase-handler/ivy.xml
hwi/build.xml
hwi/ivy.xml
ivy/common-configurations.xml
ivy/ivysettings.xml
ivy/libraries.properties
jdbc/build.xml
jdbc/ivy.xml
metastore/ivy.xml
pdk/ivy.xml
pdk/scripts/build-plugin.xml
ql/build.xml
ql/ivy.xml
serde/ivy.xml
service/build.xml
service/ivy.xml
shims/build.xml
shims/ivy.xml
testutils/hadoop
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>
> Key: HIVE-2646
> URL: https://issues.apache.org/jira/browse/HIVE-2646
> Project: Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.8.0
> Reporter: Andrew Bayer
> Assignee: Andrew Bayer
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch, HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.2.patch, HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch, HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
>
>
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending on the tarball and extracting the jars from there, rather than depending on the jars directly. It'd be great if this was fixed to actually have the jar dependencies defined directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira