You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/10/26 17:03:47 UTC
[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #1523: HDDS-4320. Let Ozone input streams implement CanUnbuffer
adoroszlai opened a new pull request #1523:
URL: https://github.com/apache/hadoop-ozone/pull/1523
## What changes were proposed in this pull request?
Implement [`CanUnbuffer` interface](https://github.com/apache/hadoop/blob/b32926f1108bef1f1e506de684c021203b2432f1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CanUnbuffer.java#L32-L35) in Ozone input streams (`ChunkInputStream`, `BlockInputStream`): release buffers and disconnect the client when `unbuffer()` is called.
https://issues.apache.org/jira/browse/HDDS-4320
## How was this patch tested?
Added contract test (base test copied from Hadoop 3.3) for both OFS and O3FS:
https://github.com/adoroszlai/hadoop-ozone/runs/1309754540#step:4:3093
https://github.com/adoroszlai/hadoop-ozone/runs/1309754540#step:4:3107
also added unit test:
https://github.com/adoroszlai/hadoop-ozone/runs/1309754295#step:3:3540
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bshashikant commented on pull request #1523: HDDS-4320. Let Ozone input streams implement CanUnbuffer
Posted by GitBox <gi...@apache.org>.
bshashikant commented on pull request #1523:
URL: https://github.com/apache/hadoop-ozone/pull/1523#issuecomment-717257127
Thanks @adoroszlai . I am still reviewing this, however, i have couple of questions:
1) In unbuffer, do we need to remove the corresponding blockInputStreams and chunkInputStreams which are already read?
2) While the connection to datanode is closed, connection to OM still is kept open, do we need to close this as well?
3) As i see the last position is cached, so after unbuffer is called, it will go into corresponding blockInputStream and chunkInputStream and starts reading again. What if the pipelne is not valid anymore i.e, the datanodes containing the blocks is replicated to a different set of datanodes? Do we need to handle this?
As far as i remember , we only refresh the pipeline during initialisation of blockInputStream only.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org