[GitHub] [incubator-druid] SEKIRO-J edited a comment on issue #7776: kafka indexing service should always use earliest offset for newly discovered topic-partitions instead of useEarliestOffset config

[GitBox <gi...@apache.org>]

SEKIRO-J edited a comment on issue #7776: kafka indexing service should always use earliest offset for newly discovered topic-partitions instead of useEarliestOffset config
URL: https://github.com/apache/incubator-druid/issues/7776#issuecomment-535747035
 
 
   Seems like this feature already exists based on code:
   https://github.com/apache/incubator-druid/blob/7f2b6577ef19f18523e8353336ad496e8dc4a270/indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java#L2550
   And a simple sharding test on Kinesis that generates new partitions tells this code works functionally. (always starts from earliest offset for new discovered partitions, otherwise, it depends on the `useEarliestOffset ` config)
   ```2019-09-27T00:43:13,934 INFO [KinesisSupervisor-sharding-test] org.apache.druid.indexing.seekablestream.supervisor.SeekableStreamSupervisor - Overriding useEarliestSequenceNumber and starting from beginning of newly discovered partition [shardId-000000000002] (which is probably from a split or merge)
   2019-09-27T00:43:14,852 INFO [KinesisSupervisor-sharding-test] org.apache.druid.indexing.seekablestream.supervisor.SeekableStreamSupervisor - Overriding useEarliestSequenceNumber and starting from beginning of newly discovered partition [shardId-000000000001] (which is probably from a split or merge)```
   
   Let me know if there is anything I missed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org