You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hairong Kuang (JIRA)" <ji...@apache.org> on 2007/06/08 23:41:26 UTC

[jira] Created: (HADOOP-1478) The blockStream of DFSClient.FSInputStream should not be buffered

The blockStream of DFSClient.FSInputStream should not be buffered
-----------------------------------------------------------------

                 Key: HADOOP-1478
                 URL: https://issues.apache.org/jira/browse/HADOOP-1478
             Project: Hadoop
          Issue Type: Improvement
          Components: dfs
    Affects Versions: 0.13.0
            Reporter: Hairong Kuang
            Assignee: Hairong Kuang
             Fix For: 0.14.0


Because DFSDataInputStream is buffered, the containing FSInputStream does not need to be buffered to avoid memory overhead and an extra copy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1478) The blockStream of DFSClient.FSInputStream should not be buffered

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502953 ] 

Raghu Angadi commented on HADOOP-1478:
--------------------------------------


This will be handled in HADOOP-1134. We could mark it as 'sub' issue. Though 1134 will remove buffering in DFSDataInputStream as opposed to removing buffering inside DFSInputSream, in order to make checksum validation closest to user (same approach as in HADOOP-1450).

> The blockStream of DFSClient.FSInputStream should not be buffered
> -----------------------------------------------------------------
>
>                 Key: HADOOP-1478
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1478
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.13.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.14.0
>
>
> Because DFSDataInputStream is buffered, the containing FSInputStream does not need to be buffered to avoid memory overhead and an extra copy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1478) The blockStream of DFSClient.FSInputStream should not be buffered

Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502949 ] 

Hairong Kuang commented on HADOOP-1478:
---------------------------------------

We should also remove buffering in DFSClient.FSOutputStream for the same reason.

> The blockStream of DFSClient.FSInputStream should not be buffered
> -----------------------------------------------------------------
>
>                 Key: HADOOP-1478
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1478
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.13.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.14.0
>
>
> Because DFSDataInputStream is buffered, the containing FSInputStream does not need to be buffered to avoid memory overhead and an extra copy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (HADOOP-1478) The blockStream of DFSClient.FSInputStream should not be buffered

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi resolved HADOOP-1478.
----------------------------------

    Resolution: Fixed


Buffering has changed as lot with HADOOP-1470 and HADOOP-1134. Now, DFSInputStreams keeps the buffer just above the input socket.


> The blockStream of DFSClient.FSInputStream should not be buffered
> -----------------------------------------------------------------
>
>                 Key: HADOOP-1478
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1478
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.13.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>
> Because DFSDataInputStream is buffered, the containing FSInputStream does not need to be buffered to avoid memory overhead and an extra copy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1478) The blockStream of DFSClient.FSInputStream should not be buffered

Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502967 ] 

Hairong Kuang commented on HADOOP-1478:
---------------------------------------

HADOOP-1134 does not remove buffering. It moves buffer from FSDataInputStream to DFSDataInputStream (datas) in FSInputChecker. Shall we remove buffering completely from FSDataInputStream?

> The blockStream of DFSClient.FSInputStream should not be buffered
> -----------------------------------------------------------------
>
>                 Key: HADOOP-1478
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1478
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.13.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.14.0
>
>
> Because DFSDataInputStream is buffered, the containing FSInputStream does not need to be buffered to avoid memory overhead and an extra copy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1478) The blockStream of DFSClient.FSInputStream should not be buffered

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502972 ] 

Raghu Angadi commented on HADOOP-1478:
--------------------------------------

bq. HADOOP-1134 does not remove buffering. It moves buffer from FSDataInputStream to DFSDataInputStream (datas) in FSInputChecker. Shall we remove buffering completely from FSDataInputStream?

I don't think there is any {{datas}} in 1134. Is it the {{datas}} in ChecksumFS? Then it is unrelated.


> The blockStream of DFSClient.FSInputStream should not be buffered
> -----------------------------------------------------------------
>
>                 Key: HADOOP-1478
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1478
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.13.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.14.0
>
>
> Because DFSDataInputStream is buffered, the containing FSInputStream does not need to be buffered to avoid memory overhead and an extra copy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.