[GitHub] yingfeng commented on issue #3214: [Feature Request]Tiered storage like Pravega

[GitBox <gi...@apache.org>]

yingfeng commented on issue #3214: [Feature Request]Tiered storage like Pravega
URL: https://github.com/apache/pulsar/issues/3214#issuecomment-448491885
 
 
   Thank you. There are some other design features for auto tiering:
   1. Retention policy, describes a policy about how much data to retain within a stream. This is similar with what Pulsar has done to let customer configure how data was migrated to low tier storage.
   2. Scaling policy, describes if and under what circumstances a stream should `automatically` scale its number of segments. This is a bit more complicated: the controller monitors traffic of stream segment writing requests, if some thresholds have been reached, it will decide how to scale. There are two types of traffic reports that can be received for segments. First type identifies if a segment should be scaled up (split) and second type identifies if a segment should be scaled down. For segments eligible for scale up, controller immediately posts request for segment scale up in the request stream for request event processor to process. However, for scale down, Controller needs to wait for at least two neighboring segments to become eligible for scale down. This auto scaling could be seen [here](https://github.com/pravega/pravega/blob/master/documentation/src/docs/pravega-concepts.md#elastic-streams-auto-scaling) with more detail.
   ![Uploading 图片.png…](https://raw.githubusercontent.com/pravega/pravega/master/documentation/src/docs/img/segment.split.merge.overtime.new.png)
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services