[GitHub] [incubator-druid] viongpanzi opened a new issue #7865: SQLMetadataSegmentManager should exec the first poll method after a poll duration if incremental poll is enabled

[GitBox <gi...@apache.org>]

viongpanzi opened a new issue #7865: SQLMetadataSegmentManager should exec the first poll method after a poll duration if incremental poll is enabled
URL: https://github.com/apache/incubator-druid/issues/7865
 
 
   If incremental poll is enabled, the poll and incrementalPoll methods will be executed one by one in the start method of SQLMetadataSegmentManager. But the first execution of the incrementalPoll is different than the subsequent ones, lastModifyDateRef is inited to 0. So the result of incrementalPoll() contains those returned by poll().
   
   In our prod env, poll and incrementalPoll methods need to poll nearly ten million data segments. And the replaceWithExistingSegmentIfPresent method in those two methods will replace the polled segments with already existing segments which produce too many objects to collect for garbage collector. The worst case in which frequent master-slave switching happens between coordiantors when time used to fgc is longer than the timeout of zk.
   
   During the startup of SQLMetadataSegmentManager(when start coordinator or coordinator becomes leader), we can save at most 50% cost if we postpone the poll execution until next execution like this:
   ```
       exec.scheduleWithFixedDelay(
             new Runnable()
             {
               @Override
               public void run()
               {
                 // poll() is synchronized together with start(), stop() and isStarted() to ensure that when stop() exists,
                 // poll() won't actually run anymore after that (it could only enter the syncrhonized section and exit
                 // immediately because the localStartedOrder doesn't match the new currentStartOrder). It's needed
                 // to avoid flakiness in SQLMetadataSegmentManagerTest.
                 // See https://github.com/apache/incubator-druid/issues/6028
                 readLock.lock();
                 try {
                   if (localStartOrder == currentStartOrder) {
                     poll();
                   }
                 }
                 catch (Exception e) {
                   log.makeAlert(e, "uncaught exception in segment manager polling thread").emit();
   
                 }
                 finally {
                   readLock.unlock();
                 }
               }
             },
             config.get().getIncrementalPollDuration().equals(Period.ZERO) ? 1000 : delay.getMillis(),
             delay.getMillis(),
             TimeUnit.MILLISECONDS
         );
   ```
   
   Please correct me if I'm wrong.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org