You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org> on 2008/10/23 02:04:44 UTC

[jira] Created: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

multifilesplit is using job default filesystem incorrectly
----------------------------------------------------------

                 Key: HADOOP-4500
                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
             Project: Hadoop Core
          Issue Type: Bug
          Components: mapred
            Reporter: Joydeep Sen Sarma


getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joydeep Sen Sarma updated HADOOP-4500:
--------------------------------------

    Attachment: 4500.patch

> multifilesplit is using job default filesystem incorrectly
> ----------------------------------------------------------
>
>                 Key: HADOOP-4500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Joydeep Sen Sarma
>         Attachments: 4500.patch
>
>
> getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-4500:
----------------------------------

       Resolution: Fixed
    Fix Version/s:     (was: 0.19.1)
                   0.20.0
     Hadoop Flags: [Reviewed]
           Status: Resolved  (was: Patch Available)

I just committed this. Thanks, Joydeep

This isn't a regression, so I don't think it belongs in 0.19.

> multifilesplit is using job default filesystem incorrectly
> ----------------------------------------------------------
>
>                 Key: HADOOP-4500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Joydeep Sen Sarma
>            Assignee: Joydeep Sen Sarma
>             Fix For: 0.20.0
>
>         Attachments: 4500.patch
>
>
> getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647163#action_12647163 ] 

Hudson commented on HADOOP-4500:
--------------------------------

Integrated in Hadoop-trunk #659 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/659/])
    . Fix MultiFileSplit to get the FileSystem from the relevant
path rather than the JobClient. Contributed by Joydeep Sen Sarma.


> multifilesplit is using job default filesystem incorrectly
> ----------------------------------------------------------
>
>                 Key: HADOOP-4500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Joydeep Sen Sarma
>            Assignee: Joydeep Sen Sarma
>             Fix For: 0.20.0
>
>         Attachments: 4500.patch
>
>
> getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Resolved: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhruba borthakur resolved HADOOP-4500.
--------------------------------------

       Resolution: Fixed
    Fix Version/s:     (was: 0.20.0)
                   0.19.0

I committed this to 0.19. Thanks Joydeep!

> multifilesplit is using job default filesystem incorrectly
> ----------------------------------------------------------
>
>                 Key: HADOOP-4500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Joydeep Sen Sarma
>            Assignee: Joydeep Sen Sarma
>             Fix For: 0.19.0
>
>         Attachments: 4500.patch
>
>
> getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642095#action_12642095 ] 

Enis Soztutar commented on HADOOP-4500:
---------------------------------------

+1 for the patch, and +1 for 0.19.1. 

> multifilesplit is using job default filesystem incorrectly
> ----------------------------------------------------------
>
>                 Key: HADOOP-4500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Joydeep Sen Sarma
>             Fix For: 0.19.1
>
>         Attachments: 4500.patch
>
>
> getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12643151#action_12643151 ] 

dhruba borthakur commented on HADOOP-4500:
------------------------------------------

 +1 @author. The patch does not contain any @author tags.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs warnings.

+1 core tests. The patch passed core unit tests.

+1 contrib tests. The patch passed contrib unit tests.



I would like to commit this to trunk and the 0.19 branch. Thanks Enis for reviewing this one.

> multifilesplit is using job default filesystem incorrectly
> ----------------------------------------------------------
>
>                 Key: HADOOP-4500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Joydeep Sen Sarma
>            Assignee: Joydeep Sen Sarma
>             Fix For: 0.19.1
>
>         Attachments: 4500.patch
>
>
> getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647690#action_12647690 ] 

Hudson commented on HADOOP-4500:
--------------------------------

Integrated in Hadoop-trunk #661 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/661/])
    Move  into the 0.19 release.


> multifilesplit is using job default filesystem incorrectly
> ----------------------------------------------------------
>
>                 Key: HADOOP-4500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Joydeep Sen Sarma
>            Assignee: Joydeep Sen Sarma
>             Fix For: 0.19.0
>
>         Attachments: 4500.patch
>
>
> getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Reopened: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhruba borthakur reopened HADOOP-4500:
--------------------------------------


I am going to commit this one for 0.19 branch too.

> multifilesplit is using job default filesystem incorrectly
> ----------------------------------------------------------
>
>                 Key: HADOOP-4500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Joydeep Sen Sarma
>            Assignee: Joydeep Sen Sarma
>             Fix For: 0.20.0
>
>         Attachments: 4500.patch
>
>
> getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647133#action_12647133 ] 

dhruba borthakur commented on HADOOP-4500:
------------------------------------------

I agree that this is not a regression, but it is a major bug that basically renders MultiFileSplit useless to process files from different file systems. This is a major use case for us. Also, the fix is very low-risk. I would really like this to go into 0.19

> multifilesplit is using job default filesystem incorrectly
> ----------------------------------------------------------
>
>                 Key: HADOOP-4500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Joydeep Sen Sarma
>            Assignee: Joydeep Sen Sarma
>             Fix For: 0.20.0
>
>         Attachments: 4500.patch
>
>
> getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhruba borthakur updated HADOOP-4500:
-------------------------------------

    Fix Version/s: 0.19.1

This looks like an issue that could merit being in 0.19.1?

> multifilesplit is using job default filesystem incorrectly
> ----------------------------------------------------------
>
>                 Key: HADOOP-4500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Joydeep Sen Sarma
>             Fix For: 0.19.1
>
>         Attachments: 4500.patch
>
>
> getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Assigned: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Enis Soztutar reassigned HADOOP-4500:
-------------------------------------

    Assignee: Joydeep Sen Sarma

> multifilesplit is using job default filesystem incorrectly
> ----------------------------------------------------------
>
>                 Key: HADOOP-4500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Joydeep Sen Sarma
>            Assignee: Joydeep Sen Sarma
>             Fix For: 0.19.1
>
>         Attachments: 4500.patch
>
>
> getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HADOOP-4500) multifilesplit is using job default filesystem incorrectly

Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joydeep Sen Sarma updated HADOOP-4500:
--------------------------------------

    Status: Patch Available  (was: Open)

> multifilesplit is using job default filesystem incorrectly
> ----------------------------------------------------------
>
>                 Key: HADOOP-4500
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4500
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Joydeep Sen Sarma
>         Attachments: 4500.patch
>
>
> getLocations() is using default filesystem based on jobconf instead of obtaining the right filesystem from the path

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.