You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Wei-Chiu Chuang (Jira)" <ji...@apache.org> on 2019/11/22 00:10:00 UTC
[jira] [Comment Edited] (HDFS-14820) The default 8KB buffer of
BlockReaderRemote#newBlockReader#BufferedOutputStream is too big
[ https://issues.apache.org/jira/browse/HDFS-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16979714#comment-16979714 ]
Wei-Chiu Chuang edited comment on HDFS-14820 at 11/22/19 12:09 AM:
-------------------------------------------------------------------
In this case, the buffer size is used for client-to-DataNode request. It's not used to read the data from DataNode. So IMO, this is a strictly better change and it doesn't really change the behavior. There's no good reason to have a 8KB buffer.
was (Author: jojochuang):
In this case, the buffer size is used for client-to-DataNode request. It's not used to read the data from DataNode. So IMO, this is a strictly better change. There's no good reason to have a 8KB buffer.
> The default 8KB buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream is too big
> -------------------------------------------------------------------------------------------
>
> Key: HDFS-14820
> URL: https://issues.apache.org/jira/browse/HDFS-14820
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Lisheng Sun
> Assignee: Lisheng Sun
> Priority: Major
> Attachments: HDFS-14820.001.patch, HDFS-14820.002.patch
>
>
> this issue is similar to HDFS-14535.
> {code:java}
> public static BlockReader newBlockReader(String file,
> ExtendedBlock block,
> Token<BlockTokenIdentifier> blockToken,
> long startOffset, long len,
> boolean verifyChecksum,
> String clientName,
> Peer peer, DatanodeID datanodeID,
> PeerCache peerCache,
> CachingStrategy cachingStrategy,
> int networkDistance) throws IOException {
> // in and out will be closed when sock is closed (by the caller)
> final DataOutputStream out = new DataOutputStream(new BufferedOutputStream(
> peer.getOutputStream()));
> new Sender(out).readBlock(block, blockToken, clientName, startOffset, len,
> verifyChecksum, cachingStrategy);
> }
> public BufferedOutputStream(OutputStream out) {
> this(out, 8192);
> }
> {code}
> Sender#readBlock parameter( block,blockToken, clientName, startOffset, len, verifyChecksum, cachingStrategy) could not use such a big buffer.
> So i think it should reduce BufferedOutputStream buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org