You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Kihwal Lee (JIRA)" <ji...@apache.org> on 2012/07/23 23:57:34 UTC
[jira] [Created] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Kihwal Lee created MAPREDUCE-4470:
-------------------------------------
Summary: Fix TestCombineFileInputFormat.testForEmptyFile
Key: MAPREDUCE-4470
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.0.0-alpha
Reporter: Kihwal Lee
Fix For: 2.1.0-alpha, 3.0.0
TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13431996#comment-13431996 ]
Hadoop QA commented on MAPREDUCE-4470:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12540068/TestFileInputFormat.java
against trunk revision .
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2721//console
This message is automatically generated.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470.patch, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423265#comment-13423265 ]
Todd Lipcon commented on MAPREDUCE-4470:
----------------------------------------
I think we should fix this on the InputFormat side, if we expect an empty job to have 1 empty split. I think the underlying fix in Common is indeed an improvement.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Mariappan Asokan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422360#comment-13422360 ]
Mariappan Asokan commented on MAPREDUCE-4470:
---------------------------------------------
I think it does break other InputFormat implementations. For example, FileInputFormat returns number of splits as 1 for an empty input file and the method getLocations() in FileSplit returns a 0 length array. This can cause array index out of bound exception if one is not careful enough to check the array length.
There is also another potential problem. When an MR job is run with an empty input, if the number of splits is 0, the number of mappers will be 0 and number of reducers can be non-zero. I am not sure whether the MR job will run successfully. My preference is to have all InputFormat implementations return 1 split of size 0 when the input is empty.
On a side note, when this test failed, the assertion message
{code}
testForEmptyFile(org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat): expected:<0> but was:<1>
{code}
was confusing. It should be:
{code}
testForEmptyFile(org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat): expected:<1> but was:<0>
{code}
Upon examining the test, there are several places where the parameter order is wrong to assertEquals(). I raised MAPREDUCE-4479 and posted a patch there.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Mariappan Asokan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mariappan Asokan updated MAPREDUCE-4470:
----------------------------------------
Attachment: TestFileInputFormat.java
I think a proper fix should address all InputFormat implementations. Tests for empty input should be added for all input formats. For example, I added a test in TestFileInputFormat.java to test for empty input. It is also failing.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470.patch, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Ilya Katsov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ilya Katsov updated MAPREDUCE-4470:
-----------------------------------
Status: Patch Available (was: Open)
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470.patch
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated MAPREDUCE-4470:
------------------------------------------
Resolution: Fixed
Fix Version/s: (was: 2.1.0-alpha)
(was: 3.0.0)
2.2.0-alpha
Assignee: Ilya Katsov
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Iilya. Committed to trunk and branch-2. Would you please open a follow up JIRA for the other inputformats?
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Assignee: Ilya Katsov
> Fix For: 2.2.0-alpha
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Ilya Katsov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ilya Katsov updated MAPREDUCE-4470:
-----------------------------------
Attachment: MAPREDUCE-4470-2.patch
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Mariappan Asokan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423306#comment-13423306 ]
Mariappan Asokan commented on MAPREDUCE-4470:
---------------------------------------------
Todd,
In the meantime, do you have any suggestion on how I can get a clean trunk build? Also, after this is fixed, can you please commit MAPREDUCE-4479 to the trunk?
Thanks.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Mariappan Asokan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13432088#comment-13432088 ]
Mariappan Asokan commented on MAPREDUCE-4470:
---------------------------------------------
Sorry about the file upload. I did not mean it to be a patch:(
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470.patch, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439958#comment-13439958 ]
Hadoop QA commented on MAPREDUCE-4470:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12541014/MAPREDUCE-4470-2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2760//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2760//console
This message is automatically generated.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434639#comment-13434639 ]
Hadoop QA commented on MAPREDUCE-4470:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12540941/TestFileInputFormat.java
against trunk revision .
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2731//console
This message is automatically generated.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440158#comment-13440158 ]
Hudson commented on MAPREDUCE-4470:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #2625 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2625/])
MAPREDUCE-4470. Fix TestCombineFileInputFormat.testForEmptyFile (ikatsov via tucu) (Revision 1376325)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1376325
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/CombineFileInputFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFileInputFormat.java
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Assignee: Ilya Katsov
> Fix For: 2.2.0-alpha
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Ilya Katsov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ilya Katsov updated MAPREDUCE-4470:
-----------------------------------
Attachment: MAPREDUCE-4470.patch
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470.patch
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Ilya Katsov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13432694#comment-13432694 ]
Ilya Katsov commented on MAPREDUCE-4470:
----------------------------------------
Mariappan,
Thank you for a test. Could you please clarify what is the correct way to obtain split locations in InputFormat?
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470.patch, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439734#comment-13439734 ]
Alejandro Abdelnur commented on MAPREDUCE-4470:
-----------------------------------------------
With and without the patch I see the following failure:
{code}
$ mvn clean test -Dtest=TestCombineFileInputFormat
...
Running org.apache.hadoop.mapred.TestCombineFileInputFormat
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.673 sec
Running org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat
Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.481 sec <<< FAILURE!
Results :
Tests in error:
testMissingBlocks(org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat): org.apache.hadoop.fs.BlockLocation
...
{code}
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Mariappan Asokan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435245#comment-13435245 ]
Mariappan Asokan commented on MAPREDUCE-4470:
---------------------------------------------
Hi Ilya Katsov,
You are right. Your patch should fix the affected test. However, I think NLineInputFormat, DBInputFormat, and CompositeInputFormat implementations may fail on empty input for different reasons. There is no test for empty input for them:( I think this can be addressed in a different Jira.
By the way, is there a committer who is going to commit your patch?
Thanks.
-- Asokan
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Ilya Katsov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435784#comment-13435784 ]
Ilya Katsov commented on MAPREDUCE-4470:
----------------------------------------
>> By the way, is there a committer who is going to commit your patch?
No.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440140#comment-13440140 ]
Hudson commented on MAPREDUCE-4470:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2689 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2689/])
MAPREDUCE-4470. Fix TestCombineFileInputFormat.testForEmptyFile (ikatsov via tucu) (Revision 1376325)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1376325
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/CombineFileInputFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFileInputFormat.java
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Assignee: Ilya Katsov
> Fix For: 2.2.0-alpha
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Mariappan Asokan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mariappan Asokan updated MAPREDUCE-4470:
----------------------------------------
Attachment: TestFileInputFormat.java
Hi Ilya Katsov,
Sorry, I was not able to get back ASAP since Jira was down. I looked at the code for FileInputFormat. I am sorry, I misled you. The method getLocations() in InputSplit CAN return a 0 length array for an empty file since there is no physical block of storage associated with the file. I have attached the new version of TestFileInputFormat.java. I feel that it is good to have a test for an empty file since it tests a boundary condition.
Cheers,
-- Asokan
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436369#comment-13436369 ]
Ahmed Radwan commented on MAPREDUCE-4470:
-----------------------------------------
+1, the latest patch looks good to me. I have also locally applied the patch and I am no longer seeing the failure. Thanks!
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13431744#comment-13431744 ]
Hadoop QA commented on MAPREDUCE-4470:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12540006/MAPREDUCE-4470.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2719//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2719//console
This message is automatically generated.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470.patch
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed MAPREDUCE-4470.
------------------------------------
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Assignee: Ilya Katsov
> Fix For: 2.0.2-alpha
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440248#comment-13440248 ]
Hudson commented on MAPREDUCE-4470:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #1143 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1143/])
MAPREDUCE-4470. Fix TestCombineFileInputFormat.testForEmptyFile (ikatsov via tucu) (Revision 1376325)
Result = FAILURE
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1376325
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/CombineFileInputFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFileInputFormat.java
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Assignee: Ilya Katsov
> Fix For: 2.2.0-alpha
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434934#comment-13434934 ]
Hadoop QA commented on MAPREDUCE-4470:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12541014/MAPREDUCE-4470-2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2733//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2733//console
This message is automatically generated.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Ilya Katsov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434857#comment-13434857 ]
Ilya Katsov commented on MAPREDUCE-4470:
----------------------------------------
Mariappan,
I've added this test to the patch. As far as I see, getSplits depends directly on FileSystem.getFileBlockLocations only in FileInputFormat and CombinedFileInputFormat, so other InputFormat implementations should be OK. Please let me know if I'm wrong on this.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439922#comment-13439922 ]
Aaron T. Myers commented on MAPREDUCE-4470:
-------------------------------------------
I've just committed the fix for the ArrayStoreException in TestCombineFileInputFormat as described in MAPREDUCE-4577. We should be good to go on this JIRA.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439968#comment-13439968 ]
Alejandro Abdelnur commented on MAPREDUCE-4470:
-----------------------------------------------
+1
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439768#comment-13439768 ]
Ahmed Radwan commented on MAPREDUCE-4470:
-----------------------------------------
Seems the new changes pushed by HDFS-3672 exposed more errors. I see that the test was succeeding before this commit and failing afterwards.
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Fix For: 2.1.0-alpha, 3.0.0
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4470) Fix
TestCombineFileInputFormat.testForEmptyFile
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440103#comment-13440103 ]
Hudson commented on MAPREDUCE-4470:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2653 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2653/])
MAPREDUCE-4470. Fix TestCombineFileInputFormat.testForEmptyFile (ikatsov via tucu) (Revision 1376325)
Result = FAILURE
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1376325
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/CombineFileInputFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFileInputFormat.java
> Fix TestCombineFileInputFormat.testForEmptyFile
> -----------------------------------------------
>
> Key: MAPREDUCE-4470
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4470
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Kihwal Lee
> Assignee: Ilya Katsov
> Fix For: 2.2.0-alpha
>
> Attachments: MAPREDUCE-4470-2.patch, MAPREDUCE-4470.patch, TestFileInputFormat.java, TestFileInputFormat.java
>
>
> TestCombineFileInputFormat.testForEmptyFile started failing after HADOOP-8599.
> It expects one split on an empty input file, but with HADOOP-8599 it gets zero. The new behavior seems correct, but is it breaking anything else?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira