[GitHub] kaijianding edited a comment on issue #6988: [Improvement] historical fast restart by lazy load columns metadata(20X faster)

[GitBox <gi...@apache.org>]

kaijianding edited a comment on issue #6988: [Improvement] historical fast restart by lazy load columns metadata(20X faster)
URL: https://github.com/apache/incubator-druid/pull/6988#issuecomment-463528940
 
 
   @pzhdfy I did similar lazy load thing in my code base, but I found there is a consequence. If historical node is killed when it is asked to download new segment, it is very likely unzip failure and left a corrupted segment folder. When lazy load=false, this segment can be ignored during historical startup(the load will fail), but when lazy load=true, this corruption is only known when a query comes in. Currently there is no interface to tell SegmentLoaderLocalCacheManager to unload this segment or reload this segment, thus queries always fail. Currently the only solution is to delete this segment folder and restart historical node to ignore this segment(because folder is gone).
   
   Better introduce unload/reload interface together in this PR.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org