[GitHub] Tielem commented on issue #1376: Introduce Offset Index

[GitBox <gi...@apache.org>]

Tielem commented on issue #1376: Introduce Offset Index
URL: https://github.com/apache/bookkeeper/issues/1376#issuecomment-463231137
 
 
   **FEATURE REQUEST**
   
   1. Please describe the feature you are requesting.
   
   We create continuous streams of growing data, eg ledgers with entities.
   However, we also require random access in the underlying data stream.
   
   Proposed API's:
   
   * getLastAddConfirmedByteOffset(): long
   
   Returns the byte offset of the last byte of the last entry committed to the ledger.
   
   * readBytes(long startOffset, long endOffset): Enumerator<Byte[]>
   
   Reads all bytes between startOffset and endOffset (inclusive), returned per stored entry.
   In case the endOffset is beyond the end of the ledger, the behavior should be the same as readEntries.
   
   * readBytes(long startOffset): Enumerator<Byte[]>
   
   Reads all bytes from startOffset to current confirmed end of ledger, returned per stored entry.
   
   2. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
   
   Currently using a different solution to tackle this use case, with less durability/scalability/etc.
   It would make our future architecture simpler and better overall.
   
   3. Provide any additional detail on your proposed use case for this feature.
   
   * Uncommitted API's
   
   While not needed for our use-case, to be API complete, uncommitted API's might be good to have.
   
   * Polling API's
   
   While not immediately needed for our use-case and we can tackle this with other polling mechanisms, it would be useful if we can open read binary from a ledger.
   Starting at a given startOffset, keep receiving byte[] until either the handler is closed or endOffset is met.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services