[GitHub] [pulsar] congbobo184 commented on issue #4403: (WIP)File system offload

[GitBox <gi...@apache.org>]

congbobo184 commented on issue #4403: (WIP)File system offload
URL: https://github.com/apache/pulsar/pull/4403#issuecomment-498087155
 
 
   > @congbobo184
   > 
   > I think the approach in this pull request follows what we have done for cloud storage. It is an okay approach.
   > 
   > However I would actually suggest using what hadoop ecosystem already provides. There are many file formats in hadoop ecosystem.
   > 
   > I would actually suggest using SequenceFile which is the most common file format in hadoop ecosystem. SequenceFile already supports multiple types of compression, which we don't need to reinvent again.
   > 
   > https://hadoop.apache.org/docs/r2.7.5/api/org/apache/hadoop/io/SequenceFile.html
   > 
   > So you can have two SequenceFile per offloaded ledger, one for data, the other for index.
   
   OK, I'm going to look at the API and change my PR

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services