You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2017/09/21 13:55:02 UTC
[jira] [Commented] (HADOOP-12981) Remove/deprecate s3native
properties from S3NativeFileSystemConfigKeys and core-default.xml
[ https://issues.apache.org/jira/browse/HADOOP-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174788#comment-16174788 ]
Steve Loughran commented on HADOOP-12981:
-----------------------------------------
with HADOOP-14738 taking the knife to all of s3n, we avoid this problem altogether, so closing at duplicate for now. Thanks for the effort though: there's lots of bits the codebase which need cleaning up...
> Remove/deprecate s3native properties from S3NativeFileSystemConfigKeys and core-default.xml
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-12981
> URL: https://issues.apache.org/jira/browse/HADOOP-12981
> Project: Hadoop Common
> Issue Type: Improvement
> Components: documentation, tools
> Affects Versions: 3.0.0-alpha1
> Reporter: Wei-Chiu Chuang
> Assignee: Wei-Chiu Chuang
> Priority: Minor
> Labels: aws
> Attachments: HADOOP-12981.001.patch, HADOOP-12981.002.patch
>
>
> It seems all properties defined in {{S3NativeFileSystemConfigKeys}} are not used. Those properties are prefixed by {{s3native}}, and the current s3native properties are all prefixed by {{fs.s3n}}, so this is likely not used currently. Additionally, core-default.xml has the description of these unused properties:
> {noformat}
> <!-- s3native File System -->
> <property>
> <name>s3native.stream-buffer-size</name>
> <value>4096</value>
> <description>The size of buffer to stream files.
> The size of this buffer should probably be a multiple of hardware
> page size (4096 on Intel x86), and it determines how much data is
> buffered during read and write operations.</description>
> </property>
> <property>
> <name>s3native.bytes-per-checksum</name>
> <value>512</value>
> <description>The number of bytes per checksum. Must not be larger than
> s3native.stream-buffer-size</description>
> </property>
> <property>
> <name>s3native.client-write-packet-size</name>
> <value>65536</value>
> <description>Packet size for clients to write</description>
> </property>
> <property>
> <name>s3native.blocksize</name>
> <value>67108864</value>
> <description>Block size</description>
> </property>
> <property>
> <name>s3native.replication</name>
> <value>3</value>
> <description>Replication factor</description>
> </property>
> {noformat}
> I think they should be removed (or deprecated) to avoid confusion if these properties are defunct.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org