You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Carl Steinbach (JIRA)" <ji...@apache.org> on 2010/02/23 19:32:27 UTC
[jira] Created: (HIVE-1192) Build fails when hadoop.version=0.20.1
Build fails when hadoop.version=0.20.1
--------------------------------------
Key: HIVE-1192
URL: https://issues.apache.org/jira/browse/HIVE-1192
Project: Hadoop Hive
Issue Type: Bug
Components: Build Infrastructure
Reporter: Carl Steinbach
Setting hadoop.version=0.20.1 causes the build to fail since
mirror.facebook.net/facebook/hive-deps does not have 0.20.1
(only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
Suggested fix:
* remove/ignore the hadoop.version configuration parameter
or
* Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
* Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861996#action_12861996 ]
John Sichi commented on HIVE-1192:
----------------------------------
0.20.1 is there now...Bill, can you try it?
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "Leo Alekseyev (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897003#action_12897003 ]
Leo Alekseyev commented on HIVE-1192:
-------------------------------------
I just ran into this issue building the latest hive checkout. Ivy somehow fails to retrieve the sources for 0.20.1 (notice how it's not in the list below):
{code}
[ivy:retrieve] confs: [default]
[ivy:retrieve] found hadoop#core;0.17.2.1 in hadoop-source
[ivy:retrieve] found hadoop#core;0.18.3 in hadoop-source
[ivy:retrieve] found hadoop#core;0.19.0 in hadoop-source
[ivy:retrieve] found hadoop#core;0.20.0 in hadoop-source
[ivy:retrieve] :: resolution report :: resolve 5143ms :: artifacts dl 5ms
{code}
I could get the build to work by doing wget http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.1/hadoop-0.20.1.tar.gz and uncompressing into $HIVE_SRC/build/hadoopcore, but this is an unsatisfactory solution since we are trying to automate all builds. Any ideas on a better fix?
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
> Attachments: hadoop-0.20.1.tar.gz.md5
>
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861922#action_12861922 ]
John Sichi commented on HIVE-1192:
----------------------------------
Hmmm, looks like the mirrors have been updated to 0.20.2. Facebook hasn't yet gotten around to provisioning more archival versions under mirror.facebook.net/facebook/hive-deps.
If you can't use 0.20.2, you could
* retry with archive.apache.org (sometimes it is under too much load)
* or see if you can google for a mirror which didn't get refreshed
* or else try downloading it through some other means such as wget (ivy seems to be very sensitive to network problems)
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "Bill Au (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861889#action_12861889 ]
Bill Au commented on HIVE-1192:
-------------------------------
I am trying to build 0.5.0 for Hadoop 0.20.1. It looks like 0.20.1 is only available on http://archive.apache.org/dist. So I set hadoop.mirror to that but I am getting the same error as in HIVE-984:
https://issues.apache.org/jira/browse/HIVE-984
I can connect and download the file but there is a problem with the checksum.
What do I need to do to proceed?
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837387#action_12837387 ]
John Sichi commented on HIVE-1192:
----------------------------------
I'll get 0.20.1 added to the FB mirror. Any others?
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837409#action_12837409 ]
John Sichi commented on HIVE-1192:
----------------------------------
I don't think we should drop the patch number, since someone may want a specific version. But we can improve the wiki docs about which versions can be obtained where and what to do if you don't care about patch number.
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861926#action_12861926 ]
John Sichi commented on HIVE-1192:
----------------------------------
Meanwhile, I'm pinging Facebook ops to see if we can get 0.20.1 added to mirror.facebook.net/facebook/hive-deps.
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862326#action_12862326 ]
John Sichi commented on HIVE-1192:
----------------------------------
Thanks Carl. I'm working on getting the corrected checksum uploaded.
Note that in the past, the checksums have actually been needed for detecting bad downloads from archive.apache.org when it was overloaded.
Is there a JIRA open for Apache to keep the release directories ivy-friendly? Or maybe it has already been corrected for later releases?
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
> Attachments: hadoop-0.20.1.tar.gz.md5
>
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862054#action_12862054 ]
Carl Steinbach commented on HIVE-1192:
--------------------------------------
@John: I attached a correctly formatted version of the md5 checksum file for the 0.20.1 tarball.
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
> Attachments: hadoop-0.20.1.tar.gz.md5
>
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837395#action_12837395 ]
Carl Steinbach commented on HIVE-1192:
--------------------------------------
bq. I'll get 0.20.1 added to the FB mirror. Any others?
Yes: 0.17.0, 0.17.1, 0.17.2, 0.18.1, 0.18.2, 0.19.1, and 0.19.2. I think that anything listed [here|http://archive.apache.org/dist/hadoop/core/] with a minor version >= 17 is fair game.
What do you think of my suggestion that we repackage these tarballs using names that drop the patch number?
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1192) Build fails when hadoop.version=0.20.1
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach resolved HIVE-1192.
----------------------------------
Resolution: Invalid
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
> Attachments: hadoop-0.20.1.tar.gz.md5
>
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862033#action_12862033 ]
John Sichi commented on HIVE-1192:
----------------------------------
If someone regenerates the correctly formatted .md5 file and tests it, I can get the bad one replaced on mirror.facebook.net.
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1192) Build fails when hadoop.version=0.20.1
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-1192:
---------------------------------
Attachment: hadoop-0.20.1.tar.gz.md5
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
> Attachments: hadoop-0.20.1.tar.gz.md5
>
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837420#action_12837420 ]
Carl Steinbach commented on HIVE-1192:
--------------------------------------
For this particular application I don't think the patch numbers matter. We download
the Hadoop tarballs in order to obtain the jars which are referenced when the
shims are built in order to satisfy the compiler. Since the API does not change between
patch versions there is no difference between using 0.20.1 and 0.20.0, etc for this
step. Subsequently, the contents of these tarballs are not referenced at all --
we expect users to supply their own installation of Hadoop. I think it's a bug
that our build currently thinks there is a difference between 0.20.0 and 0.20.1.
Am I missing something?
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862005#action_12862005 ]
Carl Steinbach commented on HIVE-1192:
--------------------------------------
John/Bill: Ivy can't download the 0.20.1 tarball off of archive.apache.org because
the associated md5 checksum file does not adhere to convention, i.e. it was not created
by running 'md5sum' on the tarball.
For example, compare the following two files:
http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.0/hadoop-0.20.0.tar.gz.md5
http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.1/hadoop-0.20.1.tar.gz.md5
The checksum file for 0.20.0 contains the output of md5sum. The checksum file for 0.20.1
contains a bunch of checksums generated using some other tool. Ivy barfs when it tries to located
the md5 checksum in this mess.
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862433#action_12862433 ]
Carl Steinbach commented on HIVE-1192:
--------------------------------------
@John: As far as I know 0.20.1 is the only release on archive.apache.org with a bogus md5 file.
I raised this issue on common-user a couple months ago and got no response. I just filed
HADOOP-6737 to track this problem.
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
> Attachments: hadoop-0.20.1.tar.gz.md5
>
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879161#action_12879161 ]
John Sichi commented on HIVE-1192:
----------------------------------
I think we can close this one now?
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
> Attachments: hadoop-0.20.1.tar.gz.md5
>
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862882#action_12862882 ]
John Sichi commented on HIVE-1192:
----------------------------------
Checksum file has been corrected; please retry.
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
> Attachments: hadoop-0.20.1.tar.gz.md5
>
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862282#action_12862282 ]
Carl Steinbach commented on HIVE-1192:
--------------------------------------
A temporary workaround for this problem is to set the property ivy.checksums="" in ivy/ivysettings.xml:
{code}
<property name="ivy.checksums" value=""/>
{code}
This has the affect of disabling checksum checks.
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
> Attachments: hadoop-0.20.1.tar.gz.md5
>
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837445#action_12837445 ]
John Sichi commented on HIVE-1192:
----------------------------------
Wiki docs updated here:
http://wiki.apache.org/hadoop/Hive/HowToContribute#Hadoop_Dependencies
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837436#action_12837436 ]
John Sichi commented on HIVE-1192:
----------------------------------
hadoop.version is used for running tests too (not just for building shims).
>From "test" target in build-common.xml:
<env key="HADOOP_HOME" value="${hadoop.root}"/>
>From build.properties:
hadoop.version.ant-internal=${hadoop.version}
hadoop.root.default=${build.dir.hadoop}/hadoop-${hadoop.version.ant-internal}
hadoop.root=${hadoop.root.default}
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when
hadoop.version=0.20.1
Posted by "Bill Au (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862221#action_12862221 ]
Bill Au commented on HIVE-1192:
-------------------------------
As Carl has pointed out, I have no problem downloading the file. But ivy fails because of the bad checksum.
> Build fails when hadoop.version=0.20.1
> --------------------------------------
>
> Key: HIVE-1192
> URL: https://issues.apache.org/jira/browse/HIVE-1192
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
> Attachments: hadoop-0.20.1.tar.gz.md5
>
>
> Setting hadoop.version=0.20.1 causes the build to fail since
> mirror.facebook.net/facebook/hive-deps does not have 0.20.1
> (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0).
> Suggested fix:
> * remove/ignore the hadoop.version configuration parameter
> or
> * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz.
> * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.