[jira] [Commented] (MAPREDUCE-6155) MapFiles are not always correctly detected by SequenceFileInputFormat

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258853#comment-14258853 ] 

Hadoop QA commented on MAPREDUCE-6155:
--------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12689130/MAPREDUCE-6155.001.patch
  against trunk revision a164ce2.

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 1 new or modified test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:red}-1 findbugs{color}.  The patch appears to introduce 13 new Findbugs (version 2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.

    {color:green}+1 core tests{color}.  The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.

Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5093//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5093//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5093//console

This message is automatically generated.

> MapFiles are not always correctly detected by SequenceFileInputFormat
> ---------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6155
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6155
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: trunk
>            Reporter: Jens Rabe
>             Fix For: trunk
>
>         Attachments: MAPREDUCE-6155.001.patch
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> MapFiles are not correctly detected by SequenceFileInputFormat.
> This is because the listStatus method only detects a MapFile correctly if the path it checks is a directory - it then replaces it by the path of the data file.
> This is likely to fail if the data file does not exist, i.e., if the input path is a directory, but does not belong to a MapFile, or if recursion is turned on and the input format comes across a file (not a directory) which is indeed part of a MapFile.
> The listStatus method should be changed to detect these cases correctly:
> * if the current candidate is a file and its name is "index" or "data", check if its corresponding other file exists, and if the key types of both files match and if the value type of the index file is LongWritable
> * If the current candidate is a directory, it is only a MapFile if (and only if) an index and a data file exist, they are both SequenceFiles and their key types match (and the index value type is LongWritable)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)