You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 17:21:15 UTC

[GitHub] [beam] damccorm opened a new issue, #20440: Unbounded sources unable to recover from checkpointMark when withMaxReadTime() is used

damccorm opened a new issue, #20440:
URL: https://github.com/apache/beam/issues/20440

   Observed on v2.22.0 
   
   When withMaxReadTime() is used, Beam creates an BoundedReadFromUnboundedSource [1].  The ReadFn<T\> class in BoundedReadFromUnboundedSource which is responsible for reading records from source. You can see this class doesnt verify if there is a recoverable checkpoint exist. Instead it always creates Reader with checkpointMark set as null [2]. 
   
    
   
   Reference:
   
   [1] [https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/Read.java#L205](https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/Read.java#L205)
   
   [2][https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedReadFromUnboundedSource.java#L193](https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedReadFromUnboundedSource.java#L193)
   
    
   
   !image-2020-07-15-15-15-12-937.png|width=836,height=346!
   
   Imported from Jira [BEAM-10493](https://issues.apache.org/jira/browse/BEAM-10493). Original Jira may contain additional context.
   Reported by: manikolbe.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org