You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2012/09/07 17:26:07 UTC
[jira] [Created] (MAPREDUCE-4644) mapreduce-client-jobclient-tests
do not run from dist tarball
Jason Lowe created MAPREDUCE-4644:
-------------------------------------
Summary: mapreduce-client-jobclient-tests do not run from dist tarball
Key: MAPREDUCE-4644
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: build, test
Affects Versions: 2.2.0-alpha
Reporter: Jason Lowe
Priority: Blocker
The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455826#comment-13455826 ]
Robert Joseph Evans commented on MAPREDUCE-4644:
------------------------------------------------
We need to be in the process of separating out true unit test from system and integration tests. Unit tests should run fast and are something that we can do for all components as part of the pre-commit build. If the test is full featured enough that it could be called as a "tool" then there is no way that it is a true unit test. I am +1 for moving those out to be part of a tools or examples package somewhere.
We also need to look at cleaning up our classpaths. Separating each tool out into a directory with a full list of its dependencies seems like a reasonable solution. It is what Oozie asks users to do for their work flows and seems to work fairly well. But that starts to sound like a larger effort then moving a few classes around and splitting a launcher into two. I think it is something that needs to be done, but perhaps needs some design work, especially in relation to how we may want to do dependency isolation in the future with OSGi or something else. Alejandro, I know you have been looking at and thinking about the classpath issue with YARN/MR, and how we should package things a lot already. MAPREDUCE-3745, HADOOP-7935, and MAPREDUCE-4421 do we need another JIRA explicitly for tools? How do we handle the case of a tool having a map/reduce dependency now that the MR code is going to be separated out so that we can use a different version of MR? Does that mean that they have to provide their own tools with a MR dependency and a config to point to them? It just seems like a change like this needs a full design.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455287#comment-13455287 ]
Alejandro Abdelnur commented on MAPREDUCE-4644:
-----------------------------------------------
The problem was that junit was ending up in the cluster classpath, and in ALL jobs classpath.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455374#comment-13455374 ]
Jason Lowe commented on MAPREDUCE-4644:
---------------------------------------
But then don't we have the issues of the tests polluting the other tools, or am I missing something? When TOOL_PATH is added to the classpath, as is done when running tools like archive and distcp, the tool could potentially encounter the same kinds of issues with tests in the classpath that we're trying to avoid.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Thomas Graves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455281#comment-13455281 ]
Thomas Graves commented on MAPREDUCE-4644:
------------------------------------------
Alejandro, was the junit.jar being included breaking something?
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450717#comment-13450717 ]
Jason Lowe commented on MAPREDUCE-4644:
---------------------------------------
Error traceback when trying to launch the tests jar:
{noformat}
$ hadoop jar hadoop-*-SNAPSHOT/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-*-SNAPSHOT-tests.jar
java.lang.NoClassDefFoundError: junit/framework/TestCase
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at org.apache.hadoop.test.MapredTestDriver.<init>(MapredTestDriver.java:60)
at org.apache.hadoop.test.MapredTestDriver.<init>(MapredTestDriver.java:54)
at org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.ClassNotFoundException: junit.framework.TestCase
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 20 more
An example program must be given as the first argument.
Valid program names are:
{noformat}
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.2.0-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455994#comment-13455994 ]
Alejandro Abdelnur commented on MAPREDUCE-4644:
-----------------------------------------------
Opened MAPREDUCE-4644
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455294#comment-13455294 ]
Jason Lowe commented on MAPREDUCE-4644:
---------------------------------------
Seems like we need to move the *-tests jars along with their specific dependencies out of the classpath, along the lines of what HADOOP-8723 was trying to do. The patch there only moves the *-tests jars out, so it would need to be enhanced to also move out dependencies that are only needed by those jars. Does that sound reasonable?
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455793#comment-13455793 ]
Tom White commented on MAPREDUCE-4644:
--------------------------------------
bq. If desired we could separate these into "tests that are really tools" which would go into tools/ and shouldn't rely on junit or other test framework stuff and "tests that are really unit tests" that go into something like tests/.
+1. Many of the "tests that are really tools" are benchmarks so we could call them that.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450972#comment-13450972 ]
Jason Lowe commented on MAPREDUCE-4644:
---------------------------------------
HADOOP-8738 explicitly removed the junit jar from the distro which lead to the jobclient tests breakage.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.2.0-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455444#comment-13455444 ]
Alejandro Abdelnur commented on MAPREDUCE-4644:
-----------------------------------------------
Well, each tool should have its own lib/ dir, that would solve the problem. Until now tools don't depend on additional JARs than then ones provided in the Hadoop cluster, thus never was a need for that.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455365#comment-13455365 ]
Alejandro Abdelnur commented on MAPREDUCE-4644:
-----------------------------------------------
(the testcase has been overloaded as a tool).
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455345#comment-13455345 ]
Jason Lowe commented on MAPREDUCE-4644:
---------------------------------------
Agreed it's risky to blindly pull in all test scope JARs, so maybe it needs to be an explicit list of jars to include.
bq. are you suggesting to create a new libtest/ dir? Wouldn't make more sense to have a tools module for this? this seems like a tool usecase.
I'm not sure it's best to clutter the tools directory with the unit test jars. They're not really related, and it can lead to some of the same problems we're trying to avoid when someone runs tools. IMHO the test stuff shouldn't be in the classpath unless you're running tests, and tools aren't tests.
So yes, I'm proposing a separate place to store the tests and their dependencies. This implies the user needs to modify the classpath to run the tests, and we could make this easier by providing a "hadoop tests-classpath" tool or something similar to aid them in doing so.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455782#comment-13455782 ]
Jason Lowe commented on MAPREDUCE-4644:
---------------------------------------
Actually I'm thinking of cases where the test jars themselves cause the problems, see HDFS-3831. There are a lot of things in these tests jars besides the items that are invoked by ToolRunner, and not all test jars even use ToolRunner. If desired we could separate these into "tests that are really tools" which would go into tools/ and shouldn't rely on junit or other test framework stuff and "tests that are really unit tests" that go into something like tests/. That would make running the "tests that are tools" a bit easier since we hopefully don't need a separate classpath beyond TOOL_PATH, but the junit test cases are completely out of the way.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467122#comment-13467122 ]
Harsh J commented on MAPREDUCE-4644:
------------------------------------
There is a workaround for this right? To place it manually on classpath and via -libjars for each app?
I mean to ask: Should this really be a blocker?
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455325#comment-13455325 ]
Colin Patrick McCabe commented on MAPREDUCE-4644:
-------------------------------------------------
Is there a workaround? Not being able to run the mapreduce client tests is frustrating.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455307#comment-13455307 ]
Alejandro Abdelnur commented on MAPREDUCE-4644:
-----------------------------------------------
Jason, out where? are you suggesting to create a new libtest/ dir? Wouldn't make more sense to have a tools module for this? this seems like a tool usecase. Also, if you bring in all test scope JARs, you have to make sure they all have compatible licenses (ie, if JDIFF endups there we are in trouble).
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467166#comment-13467166 ]
Arun C Murthy commented on MAPREDUCE-4644:
------------------------------------------
This is really a blocker - the inability to run tests (system tests) on a release is a blocker.
For now I think we should revert HADOOP-8738 and revisit it once we have a fix.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455364#comment-13455364 ]
Alejandro Abdelnur commented on MAPREDUCE-4644:
-----------------------------------------------
I'd argue that if you are running TestDFSIO in a cluster you are using a testcase as a tool to determine something. And this, IMO, applies to all tests used in this manner.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455340#comment-13455340 ]
Jason Lowe commented on MAPREDUCE-4644:
---------------------------------------
Couple of workarounds:
# copy the junit jar from the .m2 repository to {{share/hadoop/mapreduce/lib/}}
# Edit hadoop-mapreduce-project/pom.xml and remove {{<scope>test</scope>}} from the junit dependency
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-4644) mapreduce-client-jobclient-tests
do not run from dist tarball
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy resolved MAPREDUCE-4644.
--------------------------------------
Resolution: Not A Problem
Resolving since we've reverted HADOOP-8738.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4644)
mapreduce-client-jobclient-tests do not run from dist tarball
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455900#comment-13455900 ]
Alejandro Abdelnur commented on MAPREDUCE-4644:
-----------------------------------------------
Moving each tools JARs into separate lib/ dirs it is quite easy (modifying a single assembly). What we should think is a common bootstrap script for that so each tool does not have to duplicate (and get wrong) such script. I'll open a JIRA for that.
> mapreduce-client-jobclient-tests do not run from dist tarball
> -------------------------------------------------------------
>
> Key: MAPREDUCE-4644
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4644
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
> Affects Versions: 2.0.2-alpha
> Reporter: Jason Lowe
> Priority: Blocker
>
> The mapreduce jobclient tests rely on junit which is missing from the dist tarball. This prevents running often-used tests like sleep jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira