You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "fhueske (via GitHub)" <gi...@apache.org> on 2026/05/22 11:32:50 UTC

[PR] [FLINK-39719][table] Drop late probe records from TemporalRowTimeJoinOperator [flink]

fhueske opened a new pull request, #28229:
URL: https://github.com/apache/flink/pull/28229

   ## What is the purpose of the change
   
   TemporalRowTimeJoinOperator cleans up build-side state when the watermark advances. Hence, late arriving probe-side records might not see the correct build-side state and join with no or incorrect data. Especially for probe-side inputs with retractions, this is causing problems for downstream operators that cannot match retraction because insertions and retractions differ. This change drops late arriving probe-side records and tracks the number of dropped records in a metric. The new behavior is aligned with other operators that operate in event-time and clean up state on watermark progress such as windowed or OVER aggregates and interval joins.
   
   ## Brief change log
   
   * drop late arriving probe-side records
   * track dropped record count as a metric
   * adjust existing tests
   * add new unit tests to ensure correct dropping behavior
   * add warning to docs about late data handling
   
   ## Verifying this change
   
   * existing and new unit tests
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): **yes**
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
     - If yes, how is the feature documented? yes
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes (please specify the tool below)
   
   Generated-by: Claude Code 2.1.148 (Opus 4.7)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-39719][table] Drop late probe records from TemporalRowTimeJoinOperator [flink]

Posted by "fhueske (via GitHub)" <gi...@apache.org>.
fhueske commented on PR #28229:
URL: https://github.com/apache/flink/pull/28229#issuecomment-4563572417

   This change caused `TemporalJoinITCase` to become flaky.
   Working on a fix...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-39719][table] Drop late probe records from TemporalRowTimeJoinOperator [flink]

Posted by "fhueske (via GitHub)" <gi...@apache.org>.
fhueske merged PR #28229:
URL: https://github.com/apache/flink/pull/28229


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-39719][table] Drop late probe records from TemporalRowTimeJoinOperator [flink]

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #28229:
URL: https://github.com/apache/flink/pull/28229#issuecomment-4518338775

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "9522e99da603eb98d7134f0ae5ddf54d14fb2bd3",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "9522e99da603eb98d7134f0ae5ddf54d14fb2bd3",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 9522e99da603eb98d7134f0ae5ddf54d14fb2bd3 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-39719][table] Drop late probe records from TemporalRowTimeJoinOperator [flink]

Posted by "fhueske (via GitHub)" <gi...@apache.org>.
fhueske commented on PR #28229:
URL: https://github.com/apache/flink/pull/28229#issuecomment-4566738034

   Fix was merged https://github.com/apache/flink/commit/09547cea2e250e147b126cf3e7e225cb8230f99f


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org