You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Jim Hughes (Jira)" <ji...@apache.org> on 2023/12/05 19:07:00 UTC
[jira] [Updated] (FLINK-33756) Missing record with CUMULATE/HOP windows using an optimization
[ https://issues.apache.org/jira/browse/FLINK-33756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Hughes updated FLINK-33756:
-------------------------------
Description:
I have seen an optimization cause a window fail to emit a record.
With the optimization `TABLE_OPTIMIZER_DISTINCT_AGG_SPLIT_ENABLED` set to true,
the configuration AggregatePhaseStrategy.TWO_PHASE set, using a HOP or CUMULATE window with an offset, a record can be sent which causes one of the multiple active windows to fail to emit a record.
The linked code (https://github.com/jnh5y/flink/commit/ec90aa501d86f95559f8b22b0610e9fb786f05d4) modifies the `WindowAggregateJsonITCase` to demonstrate the case.
The test `testDistinctSplitDisabled` shows the expected behavior. The test `testDistinctSplitEnabled` tests the above configurations and shows that one record is missing from the output.
was:
I have seen an optimization cause a window fail to emit a record.
With the optimization `TABLE_OPTIMIZER_DISTINCT_AGG_SPLIT_ENABLED` set to true,
the configuration AggregatePhaseStrategy.TWO_PHASE set, using a HOP or CUMULATE window with an offset, a record can be sent which causes one of the multiple active windows to fail to emit a record.
The link code modifies the `WindowAggregateJsonITCase` to demonstrate the case.
The test `testDistinctSplitDisabled` shows the expected behavior. The test `testDistinctSplitEnabled` tests the above configurations and shows that one record is missing from the output.
> Missing record with CUMULATE/HOP windows using an optimization
> --------------------------------------------------------------
>
> Key: FLINK-33756
> URL: https://issues.apache.org/jira/browse/FLINK-33756
> Project: Flink
> Issue Type: Bug
> Reporter: Jim Hughes
> Priority: Major
>
> I have seen an optimization cause a window fail to emit a record.
> With the optimization `TABLE_OPTIMIZER_DISTINCT_AGG_SPLIT_ENABLED` set to true,
> the configuration AggregatePhaseStrategy.TWO_PHASE set, using a HOP or CUMULATE window with an offset, a record can be sent which causes one of the multiple active windows to fail to emit a record.
> The linked code (https://github.com/jnh5y/flink/commit/ec90aa501d86f95559f8b22b0610e9fb786f05d4) modifies the `WindowAggregateJsonITCase` to demonstrate the case.
>
> The test `testDistinctSplitDisabled` shows the expected behavior. The test `testDistinctSplitEnabled` tests the above configurations and shows that one record is missing from the output.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)