[GitHub] [pinot] sajjad-moradi opened a new issue, #11165: Consuming segment should not go to ERROR state in External View because of stream connection errors

["sajjad-moradi (via GitHub)" <gi...@apache.org>]

sajjad-moradi opened a new issue, #11165:
URL: https://github.com/apache/pinot/issues/11165

   ## Background
   In the event of consumption issues, a consuming segment sends a "stopped consumption" message to the controller, which then marks that consuming segment as OFFLINE in the Ideal State. This approach allows us to recover from transient stream errors through the RealtimeSegmentValidationManager periodic job. This job creates a new consuming segment in the Ideal State if the last segment of a stream partition is in the OFFLINE state.
   
   ## Problem Statement
   The scenario described in the previous section comes into play after consumption has started, which is when the OFFLINE to CONSUMING transition is completed. During this transition, stream consumers are set up to collect some metadata from the stream. However, if there is any exception thrown while connecting to the stream, the segment goes into the ERROR state in the External View. Unfortunately, this doesn't align with the mentioned automatic recovery process using the RealtimeSegmentValidationManager job.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org