You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/04/20 04:34:48 UTC

[GitHub] [incubator-doris] platoneko opened a new issue, #9122: [Refactor] Refactor IO stack

platoneko opened a new issue, #9122:
URL: https://github.com/apache/incubator-doris/issues/9122

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Description
   
   For more details see https://cwiki.apache.org/confluence/display/DORIS/DSIP-006%3A+Refactor+IO+stack
   
   ### Use case
   
   _No response_
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] platoneko commented on issue #9122: [Refactor] Refactor IO stack

Posted by GitBox <gi...@apache.org>.
platoneko commented on issue #9122:
URL: https://github.com/apache/incubator-doris/issues/9122#issuecomment-1103470937

   Currently, Doris IO related code dependencies:
   Rowset -> BlockManager -> ReadableBlock/WritableBlock -> Env
   Scanner -> FileReader -> Env/S3Client/HDFSFS
   FileResultWriter -> FileWriter -> Env/S3Client/HDFSFS
   
   Expected dependencies:
   Rowset -> FileSystem -> ReadStream/WriteStream (different Rowset may use different FileSystem backend)
   Scanner -> ReadStream
   FileResultWriter -> WriteStream
   
   FileSystem provides APIs for directory and file management, and manage the file cache in the future.
   ReadStream/WriteStream may contain buffer and prefetch data in parallel.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org