You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org> on 2010/06/26 00:06:50 UTC
[jira] Created: (HADOOP-6840) Support non-recursive create() in
SequenceFile.Writer
Support non-recursive create() in SequenceFile.Writer
-----------------------------------------------------
Key: HADOOP-6840
URL: https://issues.apache.org/jira/browse/HADOOP-6840
Project: Hadoop Common
Issue Type: Improvement
Components: fs, io
Affects Versions: 0.21.0, 0.20-append
Reporter: Nicolas Spiegelberg
Priority: Minor
The proposed solution for HBASE-2312 requires the sequence file to handle a non-recursive create. This is already supported by HDFS, but needs to have an equivalent SequenceFile.Writer API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6840) Support non-recursive create() in
FileSystem & SequenceFile.Writer
Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicolas Spiegelberg updated HADOOP-6840:
----------------------------------------
Attachment: HADOOP-6840_0.21-2.patch
fixed bug in original patch. constructors with the FSDataOutputStream param will not confer ownership of the stream by default, so SequenceFile.close() would not call os.close(). Added ownsStream() package-private API to set ownership after construction for the new API changes. again, this does not touch existing APIs
> Support non-recursive create() in FileSystem & SequenceFile.Writer
> ------------------------------------------------------------------
>
> Key: HADOOP-6840
> URL: https://issues.apache.org/jira/browse/HADOOP-6840
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, io
> Affects Versions: 0.21.0, 0.20-append
> Reporter: Nicolas Spiegelberg
> Priority: Minor
> Attachments: HADOOP-6840_0.21-2.patch, HADOOP-6840_0.21.patch
>
>
> The proposed solution for HBASE-2312 requires the sequence file to handle a non-recursive create. This is already supported by HDFS, but needs to have an equivalent FileSystem & SequenceFile.Writer API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6840) Support non-recursive create() in
FileSystem & SequenceFile.Writer
Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicolas Spiegelberg updated HADOOP-6840:
----------------------------------------
Summary: Support non-recursive create() in FileSystem & SequenceFile.Writer (was: Support non-recursive create() in SequenceFile.Writer)
Description: The proposed solution for HBASE-2312 requires the sequence file to handle a non-recursive create. This is already supported by HDFS, but needs to have an equivalent FileSystem & SequenceFile.Writer API. (was: The proposed solution for HBASE-2312 requires the sequence file to handle a non-recursive create. This is already supported by HDFS, but needs to have an equivalent SequenceFile.Writer API.)
> Support non-recursive create() in FileSystem & SequenceFile.Writer
> ------------------------------------------------------------------
>
> Key: HADOOP-6840
> URL: https://issues.apache.org/jira/browse/HADOOP-6840
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, io
> Affects Versions: 0.21.0, 0.20-append
> Reporter: Nicolas Spiegelberg
> Priority: Minor
>
> The proposed solution for HBASE-2312 requires the sequence file to handle a non-recursive create. This is already supported by HDFS, but needs to have an equivalent FileSystem & SequenceFile.Writer API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6840) Support non-recursive create() in
FileSystem & SequenceFile.Writer
Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicolas Spiegelberg updated HADOOP-6840:
----------------------------------------
Attachment: HADOOP-6840_0.21.patch
0.21 patch, also applies on 0.22 cleanly.
Kept this patch simple. Adds 2 static create() API calls to SequenceFile. The first one adds a 'boolean createParent' and is marked as deprecated. I would like to add this functionality to 0.20-append, which will use this API call instead of having to import the FileContext code. The 2nd API call uses the new FileContext class and should be used for clients that don't need 0.20-append backward compatibility. Please sanity check my usage of the new API.
No existing APIs were touched to make this change. Did fix a minor bug where BlockCompressWriter() was not properly looking at io.seqfile.compress.blocksize.
> Support non-recursive create() in FileSystem & SequenceFile.Writer
> ------------------------------------------------------------------
>
> Key: HADOOP-6840
> URL: https://issues.apache.org/jira/browse/HADOOP-6840
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, io
> Affects Versions: 0.21.0, 0.20-append
> Reporter: Nicolas Spiegelberg
> Priority: Minor
> Attachments: HADOOP-6840_0.21.patch
>
>
> The proposed solution for HBASE-2312 requires the sequence file to handle a non-recursive create. This is already supported by HDFS, but needs to have an equivalent FileSystem & SequenceFile.Writer API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6840) Support non-recursive create() in
FileSystem & SequenceFile.Writer
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893874#action_12893874 ]
Todd Lipcon commented on HADOOP-6840:
-------------------------------------
20-append patch looks good.
> Support non-recursive create() in FileSystem & SequenceFile.Writer
> ------------------------------------------------------------------
>
> Key: HADOOP-6840
> URL: https://issues.apache.org/jira/browse/HADOOP-6840
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, io
> Affects Versions: 0.21.0, 0.20-append
> Reporter: Nicolas Spiegelberg
> Priority: Minor
> Attachments: HADOOP-6840_0.20-append.patch, HADOOP-6840_0.21-2.patch, HADOOP-6840_0.21.patch
>
>
> The proposed solution for HBASE-2312 requires the sequence file to handle a non-recursive create. This is already supported by HDFS, but needs to have an equivalent FileSystem & SequenceFile.Writer API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6840) Support non-recursive create() in
FileSystem & SequenceFile.Writer
Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicolas Spiegelberg updated HADOOP-6840:
----------------------------------------
Attachment: HADOOP-6840_0.20-append.patch
version for 0.20-append. requires HDFS-617 as a prerequisite
> Support non-recursive create() in FileSystem & SequenceFile.Writer
> ------------------------------------------------------------------
>
> Key: HADOOP-6840
> URL: https://issues.apache.org/jira/browse/HADOOP-6840
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, io
> Affects Versions: 0.21.0, 0.20-append
> Reporter: Nicolas Spiegelberg
> Priority: Minor
> Attachments: HADOOP-6840_0.20-append.patch, HADOOP-6840_0.21-2.patch, HADOOP-6840_0.21.patch
>
>
> The proposed solution for HBASE-2312 requires the sequence file to handle a non-recursive create. This is already supported by HDFS, but needs to have an equivalent FileSystem & SequenceFile.Writer API.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.