You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Sameer Paranjpye (JIRA)" <ji...@apache.org> on 2006/11/08 23:18:52 UTC
[jira] Created: (HADOOP-696) TestTextInputFormat fails on some
platforms due to non-determinism in format.getSplits()
TestTextInputFormat fails on some platforms due to non-determinism in format.getSplits()
----------------------------------------------------------------------------------------
Key: HADOOP-696
URL: http://issues.apache.org/jira/browse/HADOOP-696
Project: Hadoop
Issue Type: Bug
Components: test
Affects Versions: 0.8.0
Reporter: Sameer Paranjpye
Assigned To: Sameer Paranjpye
Priority: Minor
Fix For: 0.9.0
TestTextInputFormat depends on format.getSplits() returning splits in the order they were created. On a local filesystem getSplits() depends on File.list(), the order produced by which is non-deterministic. This causes the test to fail on some platforms.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Work started: (HADOOP-696) TestTextInputFormat fails on some
platforms due to non-determinism in format.getSplits()
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-696?page=all ]
Work on HADOOP-696 started by Sameer Paranjpye.
> TestTextInputFormat fails on some platforms due to non-determinism in format.getSplits()
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-696
> URL: http://issues.apache.org/jira/browse/HADOOP-696
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.8.0
> Reporter: Sameer Paranjpye
> Assigned To: Sameer Paranjpye
> Priority: Minor
> Fix For: 0.9.0
>
>
> TestTextInputFormat depends on format.getSplits() returning splits in the order they were created. On a local filesystem getSplits() depends on File.list(), the order produced by which is non-deterministic. This causes the test to fail on some platforms.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-696) TestTextInputFormat fails on some
platforms due to non-determinism in format.getSplits()
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-696?page=all ]
Sameer Paranjpye updated HADOOP-696:
------------------------------------
Status: Patch Available (was: In Progress)
This patch fixes the issue by ensuring that the test always processes splits in the same order.
> TestTextInputFormat fails on some platforms due to non-determinism in format.getSplits()
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-696
> URL: http://issues.apache.org/jira/browse/HADOOP-696
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.8.0
> Reporter: Sameer Paranjpye
> Assigned To: Sameer Paranjpye
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: reorder-splits.txt
>
>
> TestTextInputFormat depends on format.getSplits() returning splits in the order they were created. On a local filesystem getSplits() depends on File.list(), the order produced by which is non-deterministic. This causes the test to fail on some platforms.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-696) TestTextInputFormat fails on some
platforms due to non-determinism in format.getSplits()
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-696?page=all ]
Doug Cutting updated HADOOP-696:
--------------------------------
Status: Resolved (was: Patch Available)
Resolution: Fixed
I just committed this. Thanks, Sameer!
> TestTextInputFormat fails on some platforms due to non-determinism in format.getSplits()
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-696
> URL: http://issues.apache.org/jira/browse/HADOOP-696
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.8.0
> Reporter: Sameer Paranjpye
> Assigned To: Sameer Paranjpye
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: reorder-splits.txt
>
>
> TestTextInputFormat depends on format.getSplits() returning splits in the order they were created. On a local filesystem getSplits() depends on File.list(), the order produced by which is non-deterministic. This causes the test to fail on some platforms.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-696) TestTextInputFormat fails on some
platforms due to non-determinism in format.getSplits()
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-696?page=all ]
Sameer Paranjpye updated HADOOP-696:
------------------------------------
Attachment: reorder-splits.txt
> TestTextInputFormat fails on some platforms due to non-determinism in format.getSplits()
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-696
> URL: http://issues.apache.org/jira/browse/HADOOP-696
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.8.0
> Reporter: Sameer Paranjpye
> Assigned To: Sameer Paranjpye
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: reorder-splits.txt
>
>
> TestTextInputFormat depends on format.getSplits() returning splits in the order they were created. On a local filesystem getSplits() depends on File.list(), the order produced by which is non-deterministic. This causes the test to fail on some platforms.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira