[GitHub] [incubator-druid] surekhasaharan opened a new issue #7233: Set "is_published" to false for overshadowed segments in sys.segments table

[GitBox <gi...@apache.org>]

surekhasaharan opened a new issue #7233: Set "is_published" to false for overshadowed segments in sys.segments table
URL: https://github.com/apache/incubator-druid/issues/7233
 
 
   ### Description
   
   `sys.segments` table retrieves published segments from coordinator, and marks all those segments with `is_published=true`. The published segments can include overshadowed segments if compaction is underway. Currently, `sys.segments` table gets all the segments from coordinator where `used=1`, there might be some segments which are overshadowed and still have `used=1` in coordinator as there can be a lag before `used=0` is set for overshadowed segments, at which point they will not be returned from coordinator. But, if those overshadowed segments are not handled properly, the `sys.segments` table may show dangling segments for a while.
   
   ### Motivation
   
   Showing overshadowed segments as part of published segments with `is_published=true` may not be correct behavior, and `sys.segments` queries would be showing _phantom_ segments for sometime. A better behavior for `sys.segments` is to mark the overshadowed segments with `is_published=false` in the query results, until they are removed completely.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org