You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2006/03/09 01:18:39 UTC
[jira] Created: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
-------------------------------------------------------------------------------------
Key: HADOOP-71
URL: http://issues.apache.org/jira/browse/HADOOP-71
Project: Hadoop
Type: Bug
Components: mapred
Versions: 0.1
Reporter: Owen O'Malley
Assigned to: Owen O'Malley
The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12519081 ]
Arun C Murthy commented on HADOOP-71:
-------------------------------------
+1
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: non-default-fs-3.patch, non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-71:
--------------------------------
Status: Patch Available (was: Open)
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12505686 ]
Owen O'Malley commented on HADOOP-71:
-------------------------------------
*sigh*
I guess it is a bit too aggressive. The point of doing that is that any OutputFormats who use that FileSystem are broken in exactly the way fixed by this patch. Furthermore, the parameter has been named 'ignored' for many releases. (Only OutputFormats were given nulls, because InputFormats don't take a FileSystem. I think I removed the InputFormat FileSystem parameter a long time ago.)
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502510 ]
Hadoop QA commented on HADOOP-71:
---------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12359029/non-default-fs-input.patch applied and successfully tested against trunk revision r544740.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/257/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/257/console
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-71?page=all ]
Owen O'Malley updated HADOOP-71:
--------------------------------
Status: In Progress (was: Patch Available)
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: http://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assigned To: Owen O'Malley
> Fix For: 0.6.0
>
> Attachments: explicit-fs.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Work started: (HADOOP-71) The SequenceFileRecordReader uses
the default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HADOOP-71 started by Owen O'Malley.
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-71:
--------------------------------
Attachment: non-default-fs-2.patch
Updated to trunk and continued passing in the default fs.
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: non-default-fs-2.patch, non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-71:
--------------------------------
Attachment: (was: non-default-fs-2.patch)
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: non-default-fs-3.patch, non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work stopped: (HADOOP-71) The SequenceFileRecordReader uses
the default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HADOOP-71 stopped by Owen O'Malley.
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assigned To: Owen O'Malley
> Attachments: explicit-fs.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-71:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this.
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: non-default-fs-3.patch, non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-71?page=all ]
Owen O'Malley updated HADOOP-71:
--------------------------------
Attachment: explicit-fs.patch
Here is a patch that adds the FileSystem as an argument to the constructor.
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: http://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Type: Bug
> Components: mapred
> Versions: 0.1
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: explicit-fs.patch
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-71?page=all ]
Owen O'Malley updated HADOOP-71:
--------------------------------
Status: Patch Available (was: Open)
Fix Version/s: 0.6.0
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: http://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assigned To: Owen O'Malley
> Fix For: 0.6.0
>
> Attachments: explicit-fs.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-71:
--------------------------------
Attachment: (was: explicit-fs.patch)
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-71:
--------------------------------
Attachment: non-default-fs-3.patch
Realized that one of the changes wasn't needed.
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: non-default-fs-3.patch, non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-71?page=comments#action_12371453 ]
Doug Cutting commented on HADOOP-71:
------------------------------------
This looks good, although it makes an incompatible changes to a public APIs used by Nutch. So we should probably continue to support the old method signatures too, or else make a co-ordinated change to Nutch.
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: http://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Type: Bug
> Components: mapred
> Versions: 0.1
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: explicit-fs.patch
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-71:
--------------------------------
Attachment: non-default-fs-input.patch
This patch also fixes the TextInputFormat and the framework's handling of the output path too. I will file a follow-up but to get deprecate the current FileSystem.get(Configuration) method and replace it like FileSystem.getDefault(Configuration). There are many place still left in the code where the default file system is gotten. However, with this patch, I was able to run map/reduce jobs (including a unit test) with the input and output going to non-default file systems.
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley reassigned HADOOP-71:
-----------------------------------
Assignee: Owen O'Malley (was: Wendy Chien)
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assigned To: Owen O'Malley
> Attachments: explicit-fs.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-71?page=all ]
Sameer Paranjpye reassigned HADOOP-71:
--------------------------------------
Assignee: Wendy Chien (was: Owen O'Malley)
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: http://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assigned To: Wendy Chien
> Fix For: 0.6.0
>
> Attachments: explicit-fs.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-71?page=comments#action_12371434 ]
Owen O'Malley commented on HADOOP-71:
-------------------------------------
Doug,
This one seems to have dropped off your radar. The fix isn't mandatory any more, because we created a custom config for the junit tests. However, it still seems cleaner to use the provided FileSystem rather than fetching the default one.
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: http://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Type: Bug
> Components: mapred
> Versions: 0.1
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: explicit-fs.patch
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12504940 ]
Doug Cutting commented on HADOOP-71:
------------------------------------
Owen, this will break InputFormats and OutputFormats that use the FileSystem passed to getRecordReader() and getRecordWriter(), since null is now passed. Is that intended? Is it now an error to touch that parameter? If so, we should at least provide some warning in the release notes, no? It'd be nicer to have a new API and then deprecate this one...
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-71:
-------------------------------
Fix Version/s: 0.14.0
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: explicit-fs.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-71:
-------------------------------
Status: Open (was: Patch Available)
This patch no longer applies to trunk, and I still think we need to make it more back-compatible.
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-71) The SequenceFileRecordReader uses the
default FileSystem rather than the supplied one
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-71:
--------------------------------
Fix Version/s: 0.14.0
Status: Patch Available (was: In Progress)
Sorry for the lateness on this one. It had fallen off my radar as being bounced. We need this one.
> The SequenceFileRecordReader uses the default FileSystem rather than the supplied one
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-71
> URL: https://issues.apache.org/jira/browse/HADOOP-71
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.1.0
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.14.0
>
> Attachments: non-default-fs-3.patch, non-default-fs-input.patch
>
>
> The mapred.TestSequenceFileInputFormat test was failing when run with a conf directory that pointed to a dfs cluster. The reason was that SequenceFileRecordReader was using the default FileSystem from the config, while the test program was assuming the "local" file system was being used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.