[GitHub] jihoonson opened a new issue #6816: Race when killing segments

[GitBox <gi...@apache.org>]

jihoonson opened a new issue #6816: Race when killing segments
URL: https://github.com/apache/incubator-druid/issues/6816
 
 
   In Druid, Killing segments means removing the segment files from the deep storage and deletes the entries from the metadata store. This is currently done using KillTask. Since it's a task, it first gets taskLocks and do the below:
   
   ```java
       final List<DataSegment> unusedSegments = toolbox
           .getTaskActionClient()
           .submit(new SegmentListUnusedAction(getDataSource(), getInterval()));
   
       if (!TaskActionPreconditions.isLockCoversSegments(taskLockMap, unusedSegments)) {
         throw new ISE(
             "Locks[%s] for task[%s] can't cover segments[%s]",
             taskLockMap.values().stream().flatMap(List::stream).collect(Collectors.toList()),
             getId(),
             unusedSegments
         );
       }
   
       // Kill segments
       for (DataSegment segment : unusedSegments) {
         toolbox.getDataSegmentKiller().kill(segment);
         toolbox.getTaskActionClient().submit(new SegmentNukeAction(ImmutableSet.of(segment)));
       }
   ```
   
   However, the coordinator provides an HTTP endpoint to enable a segment (setting `used` = true for a segment) which can happen between `SegmentListUnusedAction` and `SegmentNukeAction` without getting a taskLock.
   
   Also, the order of killing segments should be changed: the metadata store must be updated first before removing the segment file.
   
   I think we may change the way of killing segments to not use the task. Instead, the coordinator can kill segments directly because enabling/disabling existing segments happens only in the coordinator.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org