You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Zheng Shao (JIRA)" <ji...@apache.org> on 2009/09/03 07:36:32 UTC
[jira] Created: (HIVE-812) Hive compiled with "ant package" should
work with hadoop 0.17
Hive compiled with "ant package" should work with hadoop 0.17
-------------------------------------------------------------
Key: HIVE-812
URL: https://issues.apache.org/jira/browse/HIVE-812
Project: Hadoop Hive
Issue Type: Bug
Affects Versions: 0.4.0, 0.5.0
Reporter: Zheng Shao
Assignee: Zheng Shao
The goal of shims is to make Hive compiled with "ant package" to work with any version of hadoop.
However currently it is not working with hadoop 0.17 for a minor issue:
org.apache.hadoop.io.Text.compareTo was inherited (and NOT overriden) from org.apache.hadoop.io.BinaryComparable.compareTo in hadoop 0.18 and above. When compiling Hive by default, we will compile it against hadoop 0.19.0. Java compiler will automatically replace org.apache.hadoop.io.Text.compareTo with org.apache.hadoop.io.BinaryComparable.compareTo in some places, and that caused the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-812) Hive compiled with "ant package"
should work with hadoop 0.17
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750913#action_12750913 ]
Zheng Shao commented on HIVE-812:
---------------------------------
HIVE-812.2.patch: fixed some other places that also called Text.compareTo(Text)
> Hive compiled with "ant package" should work with hadoop 0.17
> -------------------------------------------------------------
>
> Key: HIVE-812
> URL: https://issues.apache.org/jira/browse/HIVE-812
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.4.0, 0.5.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-812.1.patch, HIVE-812.2.patch
>
>
> The goal of shims is to make Hive compiled with "ant package" to work with any version of hadoop.
> However currently it is not working with hadoop 0.17 for a minor issue:
> org.apache.hadoop.io.Text.compareTo was inherited (and NOT overriden) from org.apache.hadoop.io.BinaryComparable.compareTo in hadoop 0.18 and above. When compiling Hive by default, we will compile it against hadoop 0.19.0. Java compiler will automatically replace org.apache.hadoop.io.Text.compareTo with org.apache.hadoop.io.BinaryComparable.compareTo in some places, and that caused the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-812) Hive compiled with "ant package"
should work with hadoop 0.17
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751150#action_12751150 ]
Namit Jain commented on HIVE-812:
---------------------------------
I did run the tests with all versions
> Hive compiled with "ant package" should work with hadoop 0.17
> -------------------------------------------------------------
>
> Key: HIVE-812
> URL: https://issues.apache.org/jira/browse/HIVE-812
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.4.0, 0.5.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-812.1.patch, HIVE-812.2.patch
>
>
> The goal of shims is to make Hive compiled with "ant package" to work with any version of hadoop.
> However currently it is not working with hadoop 0.17 for a minor issue:
> org.apache.hadoop.io.Text.compareTo was inherited (and NOT overriden) from org.apache.hadoop.io.BinaryComparable.compareTo in hadoop 0.18 and above. When compiling Hive by default, we will compile it against hadoop 0.19.0. Java compiler will automatically replace org.apache.hadoop.io.Text.compareTo with org.apache.hadoop.io.BinaryComparable.compareTo in some places, and that caused the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-812) Hive compiled with "ant package" should
work with hadoop 0.17
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-812:
----------------------------
Attachment: HIVE-812.2.patch
> Hive compiled with "ant package" should work with hadoop 0.17
> -------------------------------------------------------------
>
> Key: HIVE-812
> URL: https://issues.apache.org/jira/browse/HIVE-812
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.4.0, 0.5.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-812.1.patch, HIVE-812.2.patch
>
>
> The goal of shims is to make Hive compiled with "ant package" to work with any version of hadoop.
> However currently it is not working with hadoop 0.17 for a minor issue:
> org.apache.hadoop.io.Text.compareTo was inherited (and NOT overriden) from org.apache.hadoop.io.BinaryComparable.compareTo in hadoop 0.18 and above. When compiling Hive by default, we will compile it against hadoop 0.19.0. Java compiler will automatically replace org.apache.hadoop.io.Text.compareTo with org.apache.hadoop.io.BinaryComparable.compareTo in some places, and that caused the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-812) Hive compiled with "ant package" should
work with hadoop 0.17
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-812:
----------------------------
Attachment: HIVE-812.1.patch
HIVE-812.1.patch works for both trunk and branch-0.4.
> Hive compiled with "ant package" should work with hadoop 0.17
> -------------------------------------------------------------
>
> Key: HIVE-812
> URL: https://issues.apache.org/jira/browse/HIVE-812
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.4.0, 0.5.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-812.1.patch
>
>
> The goal of shims is to make Hive compiled with "ant package" to work with any version of hadoop.
> However currently it is not working with hadoop 0.17 for a minor issue:
> org.apache.hadoop.io.Text.compareTo was inherited (and NOT overriden) from org.apache.hadoop.io.BinaryComparable.compareTo in hadoop 0.18 and above. When compiling Hive by default, we will compile it against hadoop 0.19.0. Java compiler will automatically replace org.apache.hadoop.io.Text.compareTo with org.apache.hadoop.io.BinaryComparable.compareTo in some places, and that caused the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-812) Hive compiled with "ant package"
should work with hadoop 0.17
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751100#action_12751100 ]
Namit Jain commented on HIVE-812:
---------------------------------
We dont have any shim specific unit tests right now - I will run unit tests for all 4 hadoop versions before committing this.
But, making it mandatory will lead to a long time to run tests - can we make it run daily as part of hudson or something like that ?
> Hive compiled with "ant package" should work with hadoop 0.17
> -------------------------------------------------------------
>
> Key: HIVE-812
> URL: https://issues.apache.org/jira/browse/HIVE-812
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.4.0, 0.5.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-812.1.patch, HIVE-812.2.patch
>
>
> The goal of shims is to make Hive compiled with "ant package" to work with any version of hadoop.
> However currently it is not working with hadoop 0.17 for a minor issue:
> org.apache.hadoop.io.Text.compareTo was inherited (and NOT overriden) from org.apache.hadoop.io.BinaryComparable.compareTo in hadoop 0.18 and above. When compiling Hive by default, we will compile it against hadoop 0.19.0. Java compiler will automatically replace org.apache.hadoop.io.Text.compareTo with org.apache.hadoop.io.BinaryComparable.compareTo in some places, and that caused the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-812) Hive compiled with "ant package" should
work with hadoop 0.17
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-812:
--------------------------------
Fix Version/s: 0.4.0
Affects Version/s: (was: 0.4.0)
(was: 0.5.0)
Component/s: Build Infrastructure
> Hive compiled with "ant package" should work with hadoop 0.17
> -------------------------------------------------------------
>
> Key: HIVE-812
> URL: https://issues.apache.org/jira/browse/HIVE-812
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-812.1.patch, HIVE-812.2.patch
>
>
> The goal of shims is to make Hive compiled with "ant package" to work with any version of hadoop.
> However currently it is not working with hadoop 0.17 for a minor issue:
> org.apache.hadoop.io.Text.compareTo was inherited (and NOT overriden) from org.apache.hadoop.io.BinaryComparable.compareTo in hadoop 0.18 and above. When compiling Hive by default, we will compile it against hadoop 0.19.0. Java compiler will automatically replace org.apache.hadoop.io.Text.compareTo with org.apache.hadoop.io.BinaryComparable.compareTo in some places, and that caused the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-812) Hive compiled with "ant package"
should work with hadoop 0.17
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751129#action_12751129 ]
Zheng Shao commented on HIVE-812:
---------------------------------
I ran the tests in the following order, and it all passed:
{code}
ant clean package
ant test -Dhadoop.version=0.17.2.1
ant test -Dhadoop.version=0.18.3
ant test -Dhadoop.version=0.19.0
ant test -Dhadoop.version=0.20.0
{code}
We are already running tests in parallel using a little different way. The test for one hadoop version takes around 18 minutes with 4 parallel tests, on a machine with 16GB memory and 4 cpu cores.
> Hive compiled with "ant package" should work with hadoop 0.17
> -------------------------------------------------------------
>
> Key: HIVE-812
> URL: https://issues.apache.org/jira/browse/HIVE-812
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.4.0, 0.5.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-812.1.patch, HIVE-812.2.patch
>
>
> The goal of shims is to make Hive compiled with "ant package" to work with any version of hadoop.
> However currently it is not working with hadoop 0.17 for a minor issue:
> org.apache.hadoop.io.Text.compareTo was inherited (and NOT overriden) from org.apache.hadoop.io.BinaryComparable.compareTo in hadoop 0.18 and above. When compiling Hive by default, we will compile it against hadoop 0.19.0. Java compiler will automatically replace org.apache.hadoop.io.Text.compareTo with org.apache.hadoop.io.BinaryComparable.compareTo in some places, and that caused the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-812) Hive compiled with "ant package"
should work with hadoop 0.17
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750916#action_12750916 ]
Raghotham Murthy commented on HIVE-812:
---------------------------------------
Do we have any unit tests for shims? Can we run tests using a specific version of hadoop? And, before committing any change, we should run tests on all versions of hadoop that hive plans to support.
> Hive compiled with "ant package" should work with hadoop 0.17
> -------------------------------------------------------------
>
> Key: HIVE-812
> URL: https://issues.apache.org/jira/browse/HIVE-812
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.4.0, 0.5.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-812.1.patch, HIVE-812.2.patch
>
>
> The goal of shims is to make Hive compiled with "ant package" to work with any version of hadoop.
> However currently it is not working with hadoop 0.17 for a minor issue:
> org.apache.hadoop.io.Text.compareTo was inherited (and NOT overriden) from org.apache.hadoop.io.BinaryComparable.compareTo in hadoop 0.18 and above. When compiling Hive by default, we will compile it against hadoop 0.19.0. Java compiler will automatically replace org.apache.hadoop.io.Text.compareTo with org.apache.hadoop.io.BinaryComparable.compareTo in some places, and that caused the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-812) Hive compiled with "ant package" should
work with hadoop 0.17
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain resolved HIVE-812.
-----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Committed in both trunk and 0.4 - Thanks Zheng
> Hive compiled with "ant package" should work with hadoop 0.17
> -------------------------------------------------------------
>
> Key: HIVE-812
> URL: https://issues.apache.org/jira/browse/HIVE-812
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.4.0, 0.5.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-812.1.patch, HIVE-812.2.patch
>
>
> The goal of shims is to make Hive compiled with "ant package" to work with any version of hadoop.
> However currently it is not working with hadoop 0.17 for a minor issue:
> org.apache.hadoop.io.Text.compareTo was inherited (and NOT overriden) from org.apache.hadoop.io.BinaryComparable.compareTo in hadoop 0.18 and above. When compiling Hive by default, we will compile it against hadoop 0.19.0. Java compiler will automatically replace org.apache.hadoop.io.Text.compareTo with org.apache.hadoop.io.BinaryComparable.compareTo in some places, and that caused the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-812) Hive compiled with "ant package"
should work with hadoop 0.17
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751102#action_12751102 ]
Raghotham Murthy commented on HIVE-812:
---------------------------------------
It will make it longer to run tests only before commit. Its probably a good hit to take. We can try to reduce the test time by using ant's parallel task - http://ant.apache.org/manual/CoreTasks/parallel.html.
> Hive compiled with "ant package" should work with hadoop 0.17
> -------------------------------------------------------------
>
> Key: HIVE-812
> URL: https://issues.apache.org/jira/browse/HIVE-812
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.4.0, 0.5.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-812.1.patch, HIVE-812.2.patch
>
>
> The goal of shims is to make Hive compiled with "ant package" to work with any version of hadoop.
> However currently it is not working with hadoop 0.17 for a minor issue:
> org.apache.hadoop.io.Text.compareTo was inherited (and NOT overriden) from org.apache.hadoop.io.BinaryComparable.compareTo in hadoop 0.18 and above. When compiling Hive by default, we will compile it against hadoop 0.19.0. Java compiler will automatically replace org.apache.hadoop.io.Text.compareTo with org.apache.hadoop.io.BinaryComparable.compareTo in some places, and that caused the problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.