You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2011/01/06 02:12:45 UTC
[jira] Created: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
SequenceFile.createWriter ignores FileSystem parameter
------------------------------------------------------
Key: HADOOP-7087
URL: https://issues.apache.org/jira/browse/HADOOP-7087
Project: Hadoop Common
Issue Type: Bug
Components: io
Affects Versions: 0.22.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979032#action_12979032 ]
Owen O'Malley commented on HADOOP-7087:
---------------------------------------
Why not just do:
{code}
return createWriter(conf, Writer.file(name.makeQualified(fs)), Writer.keyClass(keyClass),
{code}
then all of the code will be removed when the deprecated constructors are removed. We won't need to worry about the private FileSystem option being forgotten.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-7087:
--------------------------------
Priority: Blocker (was: Major)
Given it looks like HADOOP-6856 changed the semantics of a public API (no matter how insignificant you think that incompatibility might be) then either that incompatibility should be reverted or this should be a blocker. Marking it as such for now.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980638#action_12980638 ]
Hudson commented on HADOOP-7087:
--------------------------------
Integrated in Hadoop-Common-22-branch #13 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/13/])
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: hadoop-7087.txt, hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980190#action_12980190 ]
Owen O'Malley commented on HADOOP-7087:
---------------------------------------
+1, although I don't see what the javadoc warning is coming from.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: hadoop-7087.txt, hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980221#action_12980221 ]
Todd Lipcon commented on HADOOP-7087:
-------------------------------------
I think the new 1 warning is the test case calling the deprecated createWriter API. Will commit shortly.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: hadoop-7087.txt, hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979650#action_12979650 ]
Owen O'Malley commented on HADOOP-7087:
---------------------------------------
I've never seen user code actually write a SequenceFile to the RawLocalFileSystem, so the performance loss would only effect unit tests. *smile*
It seems much better to leave the compatibility code in the deprecated methods rather than pushing it into mainline code. To make it worse, it would be good to remove the internal use of FileSystem all together to ensure that FileContext is working well. I think it would be wise to minimize the amount of new code using it.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978940#action_12978940 ]
Todd Lipcon commented on HADOOP-7087:
-------------------------------------
So would you be OK with this patch if I change the filesystem Option to be private, marked for removal when the createWriter methods are gone?
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978210#action_12978210 ]
Todd Lipcon commented on HADOOP-7087:
-------------------------------------
Hi Owen. That's the original direction I went with this patch, but then I thought more about it, and decided that it is not quite correct - for example, RawLocalFileSystem and LocalFileSystem (checksummed) both use file:/// URLs. But {{path.getFileSystem(conf)}} will always return the checksummed version. So using {{fs.makeQualified(path)}} in these wrapper functions fails to preserve the user's choice of file system when they explicitly want the non-checksummed version.
What do you mean by "an option that isn't forward compatible"? Are you referring to the FileSystem API going out of style in favor of FileContext?
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-7087:
--------------------------------
Fix Version/s: 0.22.0
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HADOOP-7087:
--------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: hadoop-7087.txt, hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983573#action_12983573 ]
Hudson commented on HADOOP-7087:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #479 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/479/])
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: hadoop-7087.txt, hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HADOOP-7087:
--------------------------------
Status: Patch Available (was: Open)
Attached patch fixes MR trunk build
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979895#action_12979895 ]
Hadoop QA commented on HADOOP-7087:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12467945/hadoop-7087.txt
against trunk revision 1056006.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The applied patch generated 1049 javac compiler warnings (more than the trunk's current 1048 warnings).
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/167//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/167//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/167//console
This message is automatically generated.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: hadoop-7087.txt, hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979798#action_12979798 ]
Owen O'Malley commented on HADOOP-7087:
---------------------------------------
{quote}
What about also marking the FileSystem Option both private and @deprecated then?
{quote}
Please also include documentation in the Javadoc about it only being used for backwards compatibility. I still think the other solution is better, less code, and introduces less noise, but it does introduce a slight incompatibility.
{quote}
I'm 100% in agreement we should move our internal usage to FileContext, but that's way outside the scope of this bug.
{quote}
I wasn't asking you too, just pointing out that eventual migration away from FileSystem means introducing new APIs with FileSystem parameters is increasing the technical debt.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979664#action_12979664 ]
Todd Lipcon commented on HADOOP-7087:
-------------------------------------
What about also marking the FileSystem Option both private and @deprecated then?
I'm 100% in agreement we should move our internal usage to FileContext, but that's way outside the scope of this bug.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978919#action_12978919 ]
Owen O'Malley commented on HADOOP-7087:
---------------------------------------
But the point is that we are moving away from FileSystem and it particular are pulling it out of interfaces. As we move to FileContext, we don't want to have to change all of the public interfaces. Even before FileContext, we have been ignoring FileSystem parameters in most of the MapReduce APIs.
Moving forward, we should just use paths. Using FileSystem and path is a very confusing pattern.
You do have a valid point that we can't distinguish between checksum and raw filesystems. I suspect the fix for that is to define a new prefix for one of them.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978177#action_12978177 ]
Owen O'Malley commented on HADOOP-7087:
---------------------------------------
Rather than add a FileSystem option and then using it, create a method that makes a fully qualified path from a filesystem and a path. It seems like there is already one out there and that won't introduce an option that isn't forward compatible.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980664#action_12980664 ]
Hudson commented on HADOOP-7087:
--------------------------------
Integrated in Hadoop-Common-trunk #574 (See [https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/574/])
HADOOP-7087. SequenceFile.createWriter ignores FileSystem parameter. Contributed by Todd Lipcon
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: hadoop-7087.txt, hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978202#action_12978202 ]
Hadoop QA commented on HADOOP-7087:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12467603/hadoop-7087.txt
against trunk revision 1055206.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The applied patch generated 1049 javac compiler warnings (more than the trunk's current 1048 warnings).
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/160//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/160//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/160//console
This message is automatically generated.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HADOOP-7087:
--------------------------------
Attachment: hadoop-7087.txt
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979049#action_12979049 ]
Todd Lipcon commented on HADOOP-7087:
-------------------------------------
Like I said, this breaks the RawLocalFileSystem vs ChecksumLocalFileSystem distinction which is a very big performance difference. Sure, some day, we want to fix that, but this patch is currently a regression from previous behavior and also breaking the MR trunk build, so I don't think we should block it on fixing the URI schemes of Raw vs Checksum FS.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-7087) SequenceFile.createWriter ignores
FileSystem parameter
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HADOOP-7087:
--------------------------------
Attachment: hadoop-7087.txt
New version of the patch makes the filesystem Option private, removes it from the public options in the util package, and marks it deprecated both by javadoc and annotations.
> SequenceFile.createWriter ignores FileSystem parameter
> ------------------------------------------------------
>
> Key: HADOOP-7087
> URL: https://issues.apache.org/jira/browse/HADOOP-7087
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: hadoop-7087.txt, hadoop-7087.txt
>
>
> The SequenceFile.createWriter methods that take a FileSystem ignore this parameter after HADOOP-6856. This is causing some MR tests to fail and is a breaking change when users pass unqualified paths to these calls.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.