You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hong Tang (JIRA)" <ji...@apache.org> on 2008/11/18 12:23:44 UTC
[jira] Created: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
-----------------------------------------------------------------------------------------------
Key: HADOOP-4677
URL: https://issues.apache.org/jira/browse/HADOOP-4677
Project: Hadoop Core
Issue Type: Bug
Reporter: Hong Tang
The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Attachment: hadoop-4677-trunk.patch
Added boundary checking for input parameters. Updated unit test accordingly.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Fix Version/s: 0.20.0
Affects Version/s: 0.20.0
0.19.1
0.18.2
Status: Patch Available (was: Open)
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.2, 0.18.1, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649701#action_12649701 ]
Hadoop QA commented on HADOOP-4677:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12394287/hadoop-4677-trunk.patch
against trunk revision 719431.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3620/console
This message is automatically generated.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651296#action_12651296 ]
Hadoop QA commented on HADOOP-4677:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12394698/hadoop-4677-trunk.patch
against trunk revision 720930.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3658/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3658/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3658/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3658/console
This message is automatically generated.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Assignee: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Attachment: hadoop-4677-trunk.patch
Added guard against negative start or len.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Status: Open (was: Patch Available)
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1
> Reporter: Hong Tang
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Affects Version/s: 0.19.0
Status: Patch Available (was: Open)
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.19.0, 0.18.2, 0.18.1, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Status: Patch Available (was: Open)
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1
> Reporter: Hong Tang
> Attachments: hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-4677:
----------------------------------
Assignee: Hong Tang
Hadoop Flags: [Reviewed]
+1
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Assignee: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Attachment: (was: hadoop-4677-trunk.patch)
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12652121#action_12652121 ]
Hudson commented on HADOOP-4677:
--------------------------------
Integrated in Hadoop-trunk #675 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/675/])
. Fix semantics of FileSystem::getBlockLocations to return
meaningful values. Contributed by Hong Tang.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Assignee: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Attachment: hadoop-4677-trunk.patch
Fixed the bug by returning one block covering the whole local file.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1
> Reporter: Hong Tang
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-4677:
----------------------------------
Status: Open (was: Patch Available)
A couple nits on the test case:
* This should use
{code}
new Path(System.getProperty("test.build.data", "/tmp"), "work-dir")
{code}
for its temporary space, rather than hard-coding {{build/test/data/work-dir}}.
* TestCase is a subclass of Assert, isn't it? Is there a particular reason to use the static methods?
I also agree with Nicholas: if this is going to be corrected, it should handle its boundary conditions coherently.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.19.0, 0.18.2, 0.18.1, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650761#action_12650761 ]
Chris Douglas commented on HADOOP-4677:
---------------------------------------
bq. Is it a common practice to store data under /tmp for unit tests? Might lead to problems if different developers share the same physical machine for testing.
No; /tmp is used if test.build.data is undefined.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12648578#action_12648578 ]
Hong Tang commented on HADOOP-4677:
-----------------------------------
{code:title=FileSystem.java|borderStyle=solid}
public BlockLocation[] getFileBlockLocations(FileStatus file,
long start, long len) throws IOException {
if (file == null) {
return null;
}
String[] name = { "localhost:50010" };
String[] host = { "localhost" };
return new BlockLocation[] { new BlockLocation(name, host, 0, len) };
}
{code}
The last statement return statement should either be
"return new BlockLocation[] { new BlockLocation(name, host, 0, file.getLen()) };"
or
"return new BlockLocation[] { new BlockLocation(name, host, start, len) };"
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Hong Tang
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Attachment: hadoop-4677-trunk.patch
Changed the default path to a sub directory under /tmp for unit test.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650159#action_12650159 ]
Hadoop QA commented on HADOOP-4677:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12394519/hadoop-4677-trunk.patch
against trunk revision 719787.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3643/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3643/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3643/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3643/console
This message is automatically generated.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Component/s: fs
Affects Version/s: 0.18.1
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1
> Reporter: Hong Tang
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Status: Open (was: Patch Available)
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.2, 0.18.1, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Attachment: hadoop-4677.patch
Changed the call to return a block that contains the whole file.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1
> Reporter: Hong Tang
> Attachments: hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-4677:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Hong
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Assignee: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649704#action_12649704 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4677:
------------------------------------------------
If start >= file.getLen(), should it return an empty array? Also, should we check for invalid arguments, e.g. len < 0, start < 0, etc.?
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Attachment: hadoop-4677-for-trunk.patch
Same patch, for hadoop-trunk
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1
> Reporter: Hong Tang
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-4677)
FileSystem.getFileBlockLocations() (aka default implementation for Local
FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649197#action_12649197 ]
hong.tang edited comment on HADOOP-4677 at 11/19/08 2:48 PM:
-------------------------------------------------------------
Fixed the bug by returning one block covering the whole local file.
Added a unit test that verifies that the range returned by getFileBlockLocations cover the input (offset, length) region.
was (Author: hong.tang):
Fixed the bug by returning one block covering the whole local file.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Status: Patch Available (was: Open)
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.19.0, 0.18.2, 0.18.1, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-4677:
------------------------------
Attachment: hadoop-4677-trunk.patch
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4677) FileSystem.getFileBlockLocations()
(aka default implementation for Local FileSystem) incorrect.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650745#action_12650745 ]
Hong Tang commented on HADOOP-4677:
-----------------------------------
bq.
* This should use
{code}
new Path(System.getProperty("test.build.data", "/tmp"), "work-dir")
{code}
Is it a common practice to store data under /tmp for unit tests? Might lead to problems if different developers share the same physical machine for testing.
> FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect.
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-4677
> URL: https://issues.apache.org/jira/browse/HADOOP-4677
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.20.0
> Reporter: Hong Tang
> Fix For: 0.20.0
>
> Attachments: hadoop-4677-for-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677-trunk.patch, hadoop-4677.patch
>
>
> The default implementation of FileSystem.getFileBlockLocations(FileStatus file, long start, long len) seems to be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.