[GitHub] [incubator-druid] gianm commented on a change in pull request #7648: Handle exceptions while fetching offsets

[GitBox <gi...@apache.org>]

gianm commented on a change in pull request #7648: Handle exceptions while fetching offsets 
URL: https://github.com/apache/incubator-druid/pull/7648#discussion_r291380710
 
 

 ##########
 File path: indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java
 ##########
 @@ -2438,9 +2438,19 @@ public int getNoticesQueueSize()
           builder.put(partition, makeSequenceNumber(sequence, useExclusiveStartSequenceNumberForNonFirstSequence()));
         }
       } else {
-        // if we don't have a startingOffset (first run or we had some previous failures and reset the sequences) then
-        // get the sequence from metadata storage (if available) or Kafka/Kinesis (otherwise)
-        OrderedSequenceNumber<SequenceOffsetType> offsetFromStorage = getOffsetFromStorageForPartition(partition);
+        OrderedSequenceNumber<SequenceOffsetType> offsetFromStorage;
+        try {
+          // if we don't have a startingOffset (first run or we had some previous failures and reset
+          // the sequences) then get the sequence from metadata storage (if available) or
+          // Kafka/Kinesis (otherwise)
+          offsetFromStorage = getOffsetFromStorageForPartition(partition);
+        }
+        catch (ISE iseException) {
+          log.warn(
+              "Could not fetch offset for partition [%s], sequence [%s] - skipping this partition",
+              partition, sequence);
+          continue;
 
 Review comment:
   This means that if there is any error in getting the offset for a partition, we'll act like it doesn't exist. It works well in empty-partition case but would be confusing in any other case (or even wrong -- a transient error would cause us to skip a partition). Is there any way we can differentiate an empty partition from some sort of other error?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org