You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Jothi Padmanabhan (JIRA)" <ji...@apache.org> on 2008/06/19 13:17:44 UTC
[jira] Updated: (HADOOP-3597) SortValidator always uses the default
file system irrespective of the actual input
[ https://issues.apache.org/jira/browse/HADOOP-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jothi Padmanabhan updated HADOOP-3597:
--------------------------------------
Attachment: hadoop-3597.patch
Here is a patch that fixes the above issue.
Note that, even with this patch, it is not possible to use different hars for input and output for the SortValidator as the Cache that stores the different file systems is based only on the scheme, authority and user information and does not use the entire har name. So,
bin/hadoop jar hadoop-0.19.0-dev-test.jar testmapredsort -sortInput har:///user/jothipn/foo.har/user/jothipn/input -sortOutput ouput -->works
bin/hadoop jar hadoop-0.19.0-dev-test.jar testmapredsort -sortInput input -sortOutput har:///user/jothipn/foo1.har/user/jothipn/output -->works
bin/hadoop jar hadoop-0.19.0-dev-test.jar testmapredsort -sortInput har:///user/jothipn/foo.har/user/jothipn/input -sortOutput har:///user/jothipn/foo1.har/user/jothipn/output -->does not work.
> SortValidator always uses the default file system irrespective of the actual input
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-3597
> URL: https://issues.apache.org/jira/browse/HADOOP-3597
> Project: Hadoop Core
> Issue Type: Bug
> Components: test
> Reporter: Jothi Padmanabhan
> Attachments: hadoop-3597.patch
>
>
> In SortValidator, the underlying file system should be obtained from the given inputs (sortInput and sortOutput). It is currently assumed to be the default (HDFS) always.
> So, the following usage does not work
> bin/hadoop jar hadoop-0.19.0-dev-test.jar testmapredsort -sortInput har:///user/jothipn/foo.har/user/jothipn/input -sortOutput output
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.