[GitHub] [pulsar] rdhabalia commented on pull request #6787: Using Readers still causes backlog quota to be observed

[GitBox <gi...@apache.org>]

rdhabalia commented on pull request #6787:
URL: https://github.com/apache/pulsar/pull/6787#issuecomment-634895127


   Pulsar reader is a non-durable cursor which means
   1. Broker should not persist data because of non-durable cursor's existence but broker should allow non-durable cursor to read data which has been persisted due to durable-cursors or retention policy. 
   2. Data should be persisted only if a durable-cursor is backlogged or retention policy is configured.
   3. If durable-cursor is not backlogged and retention duration is already passed then broker should delete the data in a such a way that non-durable cursor's offset should skip already deleted data and offset should be advanced to ledger which is not deleted yet.
   
   So, I would agree that data should not be retained because of non-durable cursor and broker should not consider non-durable cursor's state while deleting the data.
   
   > The behavior of a non-durable cursor reading a managed ledger should be close to the behavior of a file reader reading a file in the filesystem. If a reader is actively reading a file, the file is still accessible even it is deleted. 
   
   Yes, that is correct. But I think file-reader is one of the usecase of non-durable readers and there should be a way to do it with non-durable cursors and for that, we can configure retention period to retain data for the expected duration. that also guarantees data availability for non-durable curosr in case of reconnection.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org