You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "Ramin Gharib (Jira)" <ji...@apache.org> on 2025/10/19 16:00:00 UTC
[jira] [Created] (FLINK-38532) FLIP-551: Make FRESHNESS Optional for Materialized Tables
Ramin Gharib created FLINK-38532:
------------------------------------
Summary: FLIP-551: Make FRESHNESS Optional for Materialized Tables
Key: FLINK-38532
URL: https://issues.apache.org/jira/browse/FLINK-38532
Project: Flink
Issue Type: New Feature
Components: Table SQL / API
Reporter: Ramin Gharib
The {{FRESHNESS}} clause, as introduced in [FLIP-435|https://cwiki.apache.org/confluence/display/FLINK/FLIP-435%3A+Introduce+a+New+Materialized+Table+for+Simplifying+Data+Pipelines], is a mandatory part of the {{CREATE MATERIALIZED TABLE}} syntax. While this forces users to be explicit about their data recency requirements, it introduces friction, especially for new users and everyday use cases.
The primary motivations for making this clause optional are:
# *Reduce Boilerplate:* For many users, the goal is to create a continuously updating materialized table with a low-latency, "near real-time" refresh. Requiring them to specify {{FRESHNESS = INTERVAL '...' SECOND/MINUTE}} in every single statement is redundant for this common pattern.
# *Lower Barrier to Entry:* The {{FRESHNESS}} concept, while powerful, currently forces new users to immediately understand the distinction between {{CONTINUOUS}} and {{FULL}} refresh modes. More importantly, to choose a sensible {{{}FRESHNESS{}}}, they implicitly need to understand that this value becomes the job's _checkpoint interval_ in the default streaming mode.
By providing a sensible default, we remove this requirement. A new user can get a working, continuous pipeline without needing to know about the underlying checkpointing mechanism upfront. They can start with a simple, functional table and learn about performance tuning concepts like checkpointing later, as their needs become more advanced.
# *Enable Platform-Level Intelligence:* A more powerful architectural approach is to allow the underlying {{Catalog}} to determine the default freshness, enabling "smart" catalogs that can implement context-aware logic.
More information to be found on the FLIP: https://cwiki.apache.org/confluence/display/FLINK/FLIP-551%3A+Make+FRESHNESS+Optional+for+Materialized+Tables
--
This message was sent by Atlassian Jira
(v8.20.10#820010)