You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Kay Kay (JIRA)" <ji...@apache.org> on 2009/12/25 11:17:29 UTC
[jira] Created: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
-----------------------------------------------------------------------------------------------------
Key: MAPREDUCE-1334
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Reporter: Kay Kay
Fix For: 0.21.0
$ cd src/contrib/index
$ ant clean test
This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
(ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated MAPREDUCE-1334:
----------------------------------------
Status: Open (was: Patch Available)
Going over the patch queue. This patch is still fine and fixes the test. There was one unnecessary import of FileOutputCommitter in the patch which I removed. I shall upload a patch with this change.
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/index
> Reporter: Kay Kay
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated MAPREDUCE-1334:
-------------------------------
Attachment: MAPREDUCE-1334.patch
Set the configuration parameter to not create _SUCCESS on the completion of the job.
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Kay Kay
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1334.patch, MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala reassigned MAPREDUCE-1334:
-------------------------------------------
Assignee: Kay Kay
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/index
> Reporter: Kay Kay
> Assignee: Kay Kay
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1334.patch, MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated MAPREDUCE-1334:
-------------------------------
Status: Patch Available (was: Open)
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/index
> Reporter: Kay Kay
> Priority: Critical
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794560#action_12794560 ]
Hadoop QA commented on MAPREDUCE-1334:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12428949/MAPREDUCE-1334.patch
against trunk revision 893828.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/343/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/343/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/343/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/343/console
This message is automatically generated.
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Kay Kay
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated MAPREDUCE-1334:
-------------------------------
Component/s: (was: test)
contrib/index
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/index
> Reporter: Kay Kay
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794553#action_12794553 ]
Kay Kay commented on MAPREDUCE-1334:
------------------------------------
This depends on MAPREDUCE-1328 (build/ivy.xml patch ) to run successfully .
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Kay Kay
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12795105#action_12795105 ]
Kay Kay commented on MAPREDUCE-1334:
------------------------------------
This bug ( due to the presence of an extra file , _SUCCESS ) seems to be an artifact of MAPREDUCE-947 .
May be - trying with FileOutputCommitter.SUCCESSFUL_JOB_OUTPUT_DIR_MARKER to false in Configuration might help.
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Kay Kay
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated MAPREDUCE-1334:
----------------------------------------
Status: Patch Available (was: Open)
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/index
> Reporter: Kay Kay
> Assignee: Kay Kay
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1334.patch, MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated MAPREDUCE-1334:
-------------------------------
Attachment: MAPREDUCE-1334.patch
TestIndexUpdater changed to make the test case pass ( Array Index Error due to the presence of a file called as _SUCCESS ).
(hdfs://localhost:36021/myoutput/_SUCCESS) .
Longer term - need to revisit test fixtures before we get here.
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Kay Kay
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated MAPREDUCE-1334:
-------------------------------
Status: Patch Available (was: Open)
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Kay Kay
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated MAPREDUCE-1334:
-------------------------------
Status: Open (was: Patch Available)
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/index
> Reporter: Kay Kay
> Priority: Critical
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated MAPREDUCE-1334:
-------------------------------
Attachment: (was: MAPREDUCE-1334.patch)
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Reporter: Kay Kay
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated MAPREDUCE-1334:
----------------------------------------
Attachment: MAPREDUCE-1334.patch
Removed an unnecessary import.
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/index
> Reporter: Kay Kay
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1334.patch, MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated MAPREDUCE-1334:
-------------------------------
Priority: Critical (was: Major)
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/index
> Reporter: Kay Kay
> Priority: Critical
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated MAPREDUCE-1334:
-------------------------------
Fix Version/s: 0.22.0
(was: 0.21.0)
Priority: Major (was: Critical)
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/index
> Reporter: Kay Kay
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1334) contrib/index - test -
TestIndexUpdater fails due to an additional presence of file _SUCCESS in
hdfs
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832709#action_12832709 ]
Hadoop QA commented on MAPREDUCE-1334:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12429081/MAPREDUCE-1334.patch
against trunk revision 908321.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/444/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/444/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/444/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/444/console
This message is automatically generated.
> contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/index
> Reporter: Kay Kay
> Priority: Critical
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1334.patch
>
>
> $ cd src/contrib/index
> $ ant clean test
> This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.
> (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).
> Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.
> Attaching a patch that would circumvent this by increasing the array length of shards by 1 .
> But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.
> Any comments / suggestions on the same welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.