[I] [Bug]: SpannerIO Change Streams may skip ChildPartitionsRecord when advancing tracker to artificial query end timestamp [beam]

["jiangzzhu (via GitHub)" <gi...@apache.org>]

jiangzzhu opened a new issue, #40232:
URL: https://github.com/apache/beam/issues/40232

   ### What happened?
   
   When reading from an unbounded Spanner Change Stream (or a Mutable Change Stream with a capped query end timestamp), `QueryChangeStreamAction` issues change stream queries with an artificial `changeStreamQueryEndTimestamp` (`now + 2 minutes`).
   
   Previously, when the query completed and needed to resume (`!stopAfterQuerySucceeds`), `QueryChangeStreamAction` called `tracker.tryClaim(changeStreamQueryEndTimestamp)` before returning `ProcessContinuation.resume()`:
   
   https://github.com/apache/beam/blob/ac4acbb6282/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/action/QueryChangeStreamAction.java#L360-L365
   
   If a change stream query finishes early without returning the `ChildPartitionsRecord` at the end of a partition's range, claiming `changeStreamQueryEndTimestamp` advances the restriction tracker past the partition's actual end timestamp. On the next continuation, the query resumes from `changeStreamQueryEndTimestamp + 1ns`, which falls outside the partition's valid timestamp range and returns an out-of-range `start_timestamp` error. `QueryChangeStreamAction` treats this out-of-range error as the end of the partition and marks the partition `FINISHED` without scheduling its child partitions, resulting in unscheduled child partitions and missing data from those child partitions.
   
   ### Expected Behavior
   
   When `!stopAfterQuerySucceeds`, `QueryChangeStreamAction` should leave the restriction tracker at the last claimed position (from the last processed data or heartbeat record) instead of advancing to `changeStreamQueryEndTimestamp`, so the subsequent query resumes from `lastClaimedTimestamp + 1ns` and reads any remaining records (including `ChildPartitionsRecord`) before the partition ends.
   
   ### Issue Priority
   
   Priority: 1 (data loss / correctness bug)
   
   ### Issue Components
   
   - [x] Component: Java SDK
   - [x] Component: IO Connectors


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug]: SpannerIO Change Streams may skip ChildPartitionsRecord when advancing tracker to artificial query end timestamp [beam]

["damccorm (via GitHub) via github" <gi...@beam.apache.org>]

damccorm commented on issue #40232:
URL: https://github.com/apache/beam/issues/40232#issuecomment-6745925823

   Fixed on `master` in #40209 (d1ebefb6f6301f6b99358b63dafa1c206cce5849) and cherry-picked to `release-2.77` in #40247 (b692dd1d50). Closing as resolved.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug]: SpannerIO Change Streams may skip ChildPartitionsRecord when advancing tracker to artificial query end timestamp [beam]

["damccorm (via GitHub) via github" <gi...@beam.apache.org>]

damccorm closed issue #40232: [Bug]: SpannerIO Change Streams may skip ChildPartitionsRecord when advancing tracker to artificial query end timestamp
URL: https://github.com/apache/beam/issues/40232


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org