You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Sushanth Sowmyan (JIRA)" <ji...@apache.org> on 2012/08/07 21:19:11 UTC
[jira] [Created] (HIVE-3341) Making hive tests run against
different MR versions
Sushanth Sowmyan created HIVE-3341:
--------------------------------------
Summary: Making hive tests run against different MR versions
Key: HIVE-3341
URL: https://issues.apache.org/jira/browse/HIVE-3341
Project: Hive
Issue Type: Bug
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sushanth Sowmyan updated HIVE-3341:
-----------------------------------
Attachment: HIVE-3341.patch
Attaching patch mimicing rev2 of the patch on reviewboard
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434286#comment-13434286 ]
Sushanth Sowmyan commented on HIVE-3341:
----------------------------------------
Namit, the issue you face is the issue reported in https://issues.apache.org/jira/browse/HIVE-3156 that I referred to in the initial bug report. Since we used to earlier have other hadoop jars in the test compile classpath, that'd succeed, but fail during runtime. With this patch, the visibility of that problem moved from test runtime to test compiletime for 23.
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433988#comment-13433988 ]
Namit Jain commented on HIVE-3341:
----------------------------------
I tried to run some tests for hadoop 23: by following steps above::
ant clean && ant very-clean && ant package
ant test -Dhadoop.mr.rev=23 -Dtest.print.classpath=true -Dhadoop.version=2.0.0-alpha -Dhadoop.security.version=2.0.0-alpha
I got the following error::
compile-test:
[echo] Project: ql
[javac] Compiling 59 source files to /Users/njain/hive/hive_commit3/build/ql/test/classes
[javac] /Users/njain/hive/hive_commit3/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java:79: cannot find symbol
[javac] symbol : class MiniMRCluster
[javac] location: package org.apache.hadoop.mapred
[javac] import org.apache.hadoop.mapred.MiniMRCluster;
[javac] ^
[javac] /Users/njain/hive/hive_commit3/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java:115: cannot find symbo\
l
[javac] symbol : class MiniMRCluster
[javac] location: class org.apache.hadoop.hive.ql.QTestUtil
[javac] private MiniMRCluster mr = null;
[javac] ^
[javac] /Users/njain/hive/hive_commit3/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java:278: cannot find symbo\
l
[javac] symbol : class MiniMRCluster
Can you put the exact instructions of how to run a particular test for 23 ?
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434429#comment-13434429 ]
Sushanth Sowmyan commented on HIVE-3341:
----------------------------------------
@Namit - Also, I've attached a .patch file on HIVE-3385 that fixes the specific issue you faced. The issue mentioned in HIVE-3156 still exists for runtime - should I keep the scope of that jira minimal so we fix this issue alone, or do we want to track the others as well on 3385?
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-3341:
---------------------------------
Status: Open (was: Patch Available)
@Sushanth: I left some comments on phabricator. Thanks.
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433991#comment-13433991 ]
Namit Jain commented on HIVE-3341:
----------------------------------
[~ashutoshc] ??
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13432153#comment-13432153 ]
Carl Steinbach commented on HIVE-3341:
--------------------------------------
@Sushanth: Left some comments on phabricator.
@Ashutosh: I got an error when I tried to cancel this patch. It also looks like I can't create new tickets. Have you encountered the same problems?
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433412#comment-13433412 ]
Ashutosh Chauhan commented on HIVE-3341:
----------------------------------------
+1 Running tests.
@Carl Yeah.. apache jira had problems over last few days.
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434422#comment-13434422 ]
Sushanth Sowmyan commented on HIVE-3341:
----------------------------------------
@Namit, I've opened up HIVE-3385.
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430651#comment-13430651 ]
Sushanth Sowmyan commented on HIVE-3341:
----------------------------------------
Phabricator review request posted at : https://reviews.facebook.net/D4557
With the patch attached, I compile/build as follows:
{noformat}
ant clean && ant very-clean && ant package
{noformat}
To test against 20:
{noformat}
ant test -Dhadoop.mr.rev=20 -Dtest.print.classpath=true
{noformat}
To test against 20S:
{noformat}
ant test -Dhadoop.mr.rev=20S -Dtest.print.classpath=true
{noformat}
To test against 23:
{noformat}
ant test -Dhadoop.mr.rev=23 -Dtest.print.classpath=true -Dhadoop.version=2.0.0-alpha -Dhadoop.security.version=2.0.0-alpha
{noformat}
( Note that 23 currently fails due to the MiniMR issue raised in HIVE-3156.)
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sushanth Sowmyan updated HIVE-3341:
-----------------------------------
Status: Patch Available (was: Open)
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434345#comment-13434345 ]
Sushanth Sowmyan commented on HIVE-3341:
----------------------------------------
Actually, looking in to it a bit more, I see that although I expected a failure because of the other jira, that shouldn't cause a compile time failure with the UnsupportedException route they've taken. This is failing because we're missing a hadoop-test jar in the 23 shims directory. Looking in to it.
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-3341:
-----------------------------------
Resolution: Fixed
Fix Version/s: 0.10.0
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks, Sushanth!
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach resolved HIVE-3341.
----------------------------------
Resolution: Fixed
Closing this again since the patch was committed.
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13432065#comment-13432065 ]
Ashutosh Chauhan commented on HIVE-3341:
----------------------------------------
@Sushanth,
Can you upload the patch here. Phabricator doesnt let you download a patch.
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434739#comment-13434739 ]
Hudson commented on HIVE-3341:
------------------------------
Integrated in Hive-trunk-h0.21 #1605 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1605/])
HIVE-3341 [jira] Making hive tests run against different MR versions
(Sushanth Sowmyan via Ashutosh Chauhan)
Summary:
HIVE-3341 Making hive tests run against different MR versions
After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
With this patch, we can set ant variable hadoop.mr.rev to 20, 20S or 23 to test against a particular version.
Test Plan: current tests continue to work - this is a build system change
Reviewers: JIRA, ashutoshc, cwsteinbach
Reviewed By: ashutoshc
Differential Revision: https://reviews.facebook.net/D4557 (Revision 1372745)
Result = SUCCESS
hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1372745
Files :
* /hive/trunk/build-common.xml
* /hive/trunk/build.properties
* /hive/trunk/ivy/common-configurations.xml
* /hive/trunk/shims/build.xml
* /hive/trunk/shims/src/common-secure/test
* /hive/trunk/shims/src/common-secure/test/org
* /hive/trunk/shims/src/common-secure/test/org/apache
* /hive/trunk/shims/src/common-secure/test/org/apache/hadoop
* /hive/trunk/shims/src/common-secure/test/org/apache/hadoop/hive
* /hive/trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift
* /hive/trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java
* /hive/trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java
* /hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java
* /hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain reopened HIVE-3341:
------------------------------
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434354#comment-13434354 ]
Namit Jain commented on HIVE-3341:
----------------------------------
@Sushanth, do you want to file a follow-up jira for this ? No point in opening this one again.
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433997#comment-13433997 ]
Namit Jain commented on HIVE-3341:
----------------------------------
@carl, @ashutosh, can you update the instructions for running tests on 23 ?
If it does not work, I would have to do -1 on this, and this patch needs to be backed out, or a
followup needs to be filed for 23.
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3341) Making hive tests run against
different MR versions
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sushanth Sowmyan updated HIVE-3341:
-----------------------------------
Status: Patch Available (was: Open)
> Making hive tests run against different MR versions
> ---------------------------------------------------
>
> Key: HIVE-3341
> URL: https://issues.apache.org/jira/browse/HIVE-3341
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-3341.patch
>
>
> After we build hive, we want to have the ability to run unit tests against specific hadoop versions. Currently, the classpath constructed has multiple hadoop jars, which makes compiling okay, but running non-deterministic.
> An example is HIVE-3156, where running against 0.23 shows issues with a couple of tests (which should either be shimmed out, or separated into directories the way it's done in the shims/ directory) - It would also be nice to find these issues out at test-compile time itself, rather than having them fail at test runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira