You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hairong Kuang (JIRA)" <ji...@apache.org> on 2007/06/08 23:41:26 UTC
[jira] Created: (HADOOP-1478) The blockStream of
DFSClient.FSInputStream should not be buffered
The blockStream of DFSClient.FSInputStream should not be buffered
-----------------------------------------------------------------
Key: HADOOP-1478
URL: https://issues.apache.org/jira/browse/HADOOP-1478
Project: Hadoop
Issue Type: Improvement
Components: dfs
Affects Versions: 0.13.0
Reporter: Hairong Kuang
Assignee: Hairong Kuang
Fix For: 0.14.0
Because DFSDataInputStream is buffered, the containing FSInputStream does not need to be buffered to avoid memory overhead and an extra copy.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1478) The blockStream of
DFSClient.FSInputStream should not be buffered
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502953 ]
Raghu Angadi commented on HADOOP-1478:
--------------------------------------
This will be handled in HADOOP-1134. We could mark it as 'sub' issue. Though 1134 will remove buffering in DFSDataInputStream as opposed to removing buffering inside DFSInputSream, in order to make checksum validation closest to user (same approach as in HADOOP-1450).
> The blockStream of DFSClient.FSInputStream should not be buffered
> -----------------------------------------------------------------
>
> Key: HADOOP-1478
> URL: https://issues.apache.org/jira/browse/HADOOP-1478
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.14.0
>
>
> Because DFSDataInputStream is buffered, the containing FSInputStream does not need to be buffered to avoid memory overhead and an extra copy.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1478) The blockStream of
DFSClient.FSInputStream should not be buffered
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502949 ]
Hairong Kuang commented on HADOOP-1478:
---------------------------------------
We should also remove buffering in DFSClient.FSOutputStream for the same reason.
> The blockStream of DFSClient.FSInputStream should not be buffered
> -----------------------------------------------------------------
>
> Key: HADOOP-1478
> URL: https://issues.apache.org/jira/browse/HADOOP-1478
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.14.0
>
>
> Because DFSDataInputStream is buffered, the containing FSInputStream does not need to be buffered to avoid memory overhead and an extra copy.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-1478) The blockStream of
DFSClient.FSInputStream should not be buffered
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi resolved HADOOP-1478.
----------------------------------
Resolution: Fixed
Buffering has changed as lot with HADOOP-1470 and HADOOP-1134. Now, DFSInputStreams keeps the buffer just above the input socket.
> The blockStream of DFSClient.FSInputStream should not be buffered
> -----------------------------------------------------------------
>
> Key: HADOOP-1478
> URL: https://issues.apache.org/jira/browse/HADOOP-1478
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
>
> Because DFSDataInputStream is buffered, the containing FSInputStream does not need to be buffered to avoid memory overhead and an extra copy.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1478) The blockStream of
DFSClient.FSInputStream should not be buffered
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502967 ]
Hairong Kuang commented on HADOOP-1478:
---------------------------------------
HADOOP-1134 does not remove buffering. It moves buffer from FSDataInputStream to DFSDataInputStream (datas) in FSInputChecker. Shall we remove buffering completely from FSDataInputStream?
> The blockStream of DFSClient.FSInputStream should not be buffered
> -----------------------------------------------------------------
>
> Key: HADOOP-1478
> URL: https://issues.apache.org/jira/browse/HADOOP-1478
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.14.0
>
>
> Because DFSDataInputStream is buffered, the containing FSInputStream does not need to be buffered to avoid memory overhead and an extra copy.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1478) The blockStream of
DFSClient.FSInputStream should not be buffered
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502972 ]
Raghu Angadi commented on HADOOP-1478:
--------------------------------------
bq. HADOOP-1134 does not remove buffering. It moves buffer from FSDataInputStream to DFSDataInputStream (datas) in FSInputChecker. Shall we remove buffering completely from FSDataInputStream?
I don't think there is any {{datas}} in 1134. Is it the {{datas}} in ChecksumFS? Then it is unrelated.
> The blockStream of DFSClient.FSInputStream should not be buffered
> -----------------------------------------------------------------
>
> Key: HADOOP-1478
> URL: https://issues.apache.org/jira/browse/HADOOP-1478
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.14.0
>
>
> Because DFSDataInputStream is buffered, the containing FSInputStream does not need to be buffered to avoid memory overhead and an extra copy.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.