[GitHub] [druid] dat-vikash commented on issue #7030: Add apache pulsar based indexing service

[GitBox <gi...@apache.org>]

dat-vikash commented on issue #7030:
URL: https://github.com/apache/druid/issues/7030#issuecomment-1126194538

   @devinbost The topic pesistence is set at creation 
   
   Persistent Topic: `persistent://property/cluster/namespace/topic`  
     All messages will be stored on disk
   
   Non persistent: `non-persistent://property/cluster/namespace/topic`  
       Messages will not be persisted to Disk
   
   The durability refers to the cursor persistence. `Durable` subscriptions have the cursor persisted 
   >If a broker restarts from a failure, it can recover the cursor from the persistent storage (BookKeeper), so that messages can continue to be consumed from the last consumed position.
   
   While `non-durable` the cursor is lost on bookie restart. 
   >Once a broker stops, the cursor is lost and can never be recovered, so that messages can not continue to be consumed from the last consumed position.
   
   Since druid's supervisor creates short lived tasks and we want to support resumption from any point in the stream, we require a `non-durable` subscription mode. `durable` subscription cursor position is enforced by the bookie and cannot be overridden, meaning we could not support resumption from any point in the stream.  So to properly support resiliency and idempotent resumption, we need to use the `non-durable` subscription. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org