You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "alpinegizmo (via GitHub)" <gi...@apache.org> on 2026/05/27 18:52:51 UTC
[PR] Add Flink 2.3.0 release [flink-web]
alpinegizmo opened a new pull request, #853:
URL: https://github.com/apache/flink-web/pull/853
This is the release blog post for Flink 2.3.0.
After the release is finalized, this PR will be updated with the other updates needed for the website and then published.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "1996fanrui (via GitHub)" <gi...@apache.org>.
1996fanrui commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3344925843
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: Cleaner deployment with smaller footprint
+
+The plugin registers the standard `s3://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+**More Information**
+* [FLIP-555](https://cwiki.apache.org/confluence/x/uYqmFw)
+* [Native S3 FileSystem Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/filesystems/s3/)
+
+# Runtime Improvements
+
+## Adaptive Partition Selection for RPC-Heavy Workloads (FLIP-339)
+
+When upstream and downstream parallelism differ, Flink uses `RebalancePartitioner`, which selects
+target channels round-robin. For jobs that interact with external RPC services (Redis, HBase,
+LLM serving, etc.) round-robin selection causes severe backpressure as soon as a single
+downstream subtask slows down — the partitioner keeps feeding it new data even though it is
+already overloaded.
+
+Flink 2.3 adds an adaptive, load-aware partition-selection mode for
+`StreamPartitioner` that routes records to the least-loaded downstream channel instead. In
+benchmarks, this delivers up to ~3x throughput improvement under skewed downstream processing.
+
+The feature is opt-in via two new options:
+
+- `taskmanager.network.adaptive-partitioner.enabled` (default: `false`)
+- `taskmanager.network.adaptive-partitioner.max-traverse-size` (default: `4`) — number of
+ channels examined when selecting the idlest target.
+
+**More Information**
+* [FLIP-339](https://cwiki.apache.org/confluence/x/nYyzDw)
+
+## AdaptiveScheduler Rescale History and Web UI (FLIP-487 and FLIP-495)
+
+Streaming jobs using the adaptive scheduler now record a complete history of rescale events, including parallelism changes, slot allocations, scheduler state transitions, and termination reasons. This data is available through new REST endpoints and visualized in a dedicated "Rescales" tab in the Flink Web UI.
+
+The UI exposes:
+- Rescale event counts and timeline
+- Duration statistics with percentiles
+- Per-event details (parallelism before/after, triggered-by reason)
+- Adaptive scheduler configuration
+
+```yaml
+# Enable rescale history (disabled by default)
+web.adaptive-scheduler.rescale-history.size: 100
+```
+
+Setting the history size to a positive integer activates the feature.
+
+**More Information**
+* [FLIP-487](https://cwiki.apache.org/confluence/x/vZCMEw)
+* [FLIP-495](https://cwiki.apache.org/confluence/x/TQr0Ew)
+* [Elastic Scaling Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/elastic_scaling/)
+
+## Watermark Alignment for Fast Backlog Processing (FLINK-37399)
+
+Prior to Flink 2.3, watermark alignment due to the announcement delays was inadvertently
+limiting how quickly a job could process a backlog. For example with max allowed drift
+configured to 30s and watermark alignment updated every ~1s, prior to Flink 2.3 watermark
+alignment was de facto capping the backlog processing speed to:
+
+> 30 "event time" seconds per each 1 "real world" second
+
+In Flink 2.3 the watermark alignment was redesigned to solve those announcement delays by the
+introduction of a watermark alignment buffer. By default this buffer has a size of 3 and it
+delays the application of the watermark alignment algorithm by 3 update intervals. This means
+in Flink 2.3+, by default watermark alignment will be pausing sources a couple of seconds later
+than it used to, potentially increasing state size of windowed and temporal operators.
+However this should be negligible for all practical use cases. Nevertheless, the size of this
+buffer can be configured:
+
+```yaml
+# Control alignment buffer size (3 by default)
+pipeline.watermark-alignment.buffer-size: 3
+
+# Set to 0 to restore Flink 2.2 behavior
+pipeline.watermark-alignment.buffer-size: 0
+```
+
+**More Information**
+* [FLINK-37399](https://issues.apache.org/jira/browse/FLINK-37399)
+
+## Checkpointing During Recovery (FLIP-547)
Review Comment:
Thanks for the ping, it is fine for me.
It is same with the release note from https://issues.apache.org/jira/browse/FLINK-35761
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "alpinegizmo (via GitHub)" <gi...@apache.org>.
alpinegizmo commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3333225388
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
Review Comment:
Thanks; I've added this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "alpinegizmo (via GitHub)" <gi...@apache.org>.
alpinegizmo commented on PR #853:
URL: https://github.com/apache/flink-web/pull/853#issuecomment-4802387880
@RocMarshal conflict resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "alpinegizmo (via GitHub)" <gi...@apache.org>.
alpinegizmo merged PR #853:
URL: https://github.com/apache/flink-web/pull/853
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "alpinegizmo (via GitHub)" <gi...@apache.org>.
alpinegizmo commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3333227726
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "alpinegizmo (via GitHub)" <gi...@apache.org>.
alpinegizmo commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3334441330
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers application management. Flink 2.3.0 brings together contributors from around the globe, implements full or core functionalities of 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+Flink 2.3 includes only these enhancements that enable more sophisticated temporal processing patterns and late data handling directly in SQL. The other features described in FLIP-565, around improved state access and broadcast state, are expected to follow in a future release.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up S3 Filesystem with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Better performance**: See [Benchmarking Native S3 FileSystem (flink-s3-fs-native) vs Presto S3 (flink-s3-fs-presto)](https://cwiki.apache.org/confluence/x/7Ig8G) for details
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: No dependency mess with smaller footprint
+
+The plugin registers the standard `s3://` and `s3a://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+Note: The Native S3 FileSystem is experimental in Flink 2.3. It is functionally complete and has demonstrated strong performance in benchmarks, but more experience is needed before it can be recommended for use in production.
Review Comment:
@Samrat002 Does this sound right to you? Also see my update above, at line 132, about better performance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "alpinegizmo (via GitHub)" <gi...@apache.org>.
alpinegizmo commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3334441330
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers application management. Flink 2.3.0 brings together contributors from around the globe, implements full or core functionalities of 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+Flink 2.3 includes only these enhancements that enable more sophisticated temporal processing patterns and late data handling directly in SQL. The other features described in FLIP-565, around improved state access and broadcast state, are expected to follow in a future release.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up S3 Filesystem with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Better performance**: See [Benchmarking Native S3 FileSystem (flink-s3-fs-native) vs Presto S3 (flink-s3-fs-presto)](https://cwiki.apache.org/confluence/x/7Ig8G) for details
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: No dependency mess with smaller footprint
+
+The plugin registers the standard `s3://` and `s3a://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+Note: The Native S3 FileSystem is experimental in Flink 2.3. It is functionally complete and has demonstrated strong performance in benchmarks, but more experience is needed before it can be recommended for use in production.
Review Comment:
@Samrat002 Does this sound right to you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3324718470
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: Cleaner deployment with smaller footprint
+
+The plugin registers the standard `s3://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+**More Information**
+* [FLIP-555](https://cwiki.apache.org/confluence/x/uYqmFw)
+* [Native S3 FileSystem Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/filesystems/s3/)
Review Comment:
Hi, @Samrat002
Could you help check ? thx
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: Cleaner deployment with smaller footprint
+
+The plugin registers the standard `s3://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+**More Information**
+* [FLIP-555](https://cwiki.apache.org/confluence/x/uYqmFw)
+* [Native S3 FileSystem Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/filesystems/s3/)
+
+# Runtime Improvements
+
+## Adaptive Partition Selection for RPC-Heavy Workloads (FLIP-339)
+
+When upstream and downstream parallelism differ, Flink uses `RebalancePartitioner`, which selects
+target channels round-robin. For jobs that interact with external RPC services (Redis, HBase,
+LLM serving, etc.) round-robin selection causes severe backpressure as soon as a single
+downstream subtask slows down — the partitioner keeps feeding it new data even though it is
+already overloaded.
+
+Flink 2.3 adds an adaptive, load-aware partition-selection mode for
+`StreamPartitioner` that routes records to the least-loaded downstream channel instead. In
+benchmarks, this delivers up to ~3x throughput improvement under skewed downstream processing.
+
+The feature is opt-in via two new options:
+
+- `taskmanager.network.adaptive-partitioner.enabled` (default: `false`)
+- `taskmanager.network.adaptive-partitioner.max-traverse-size` (default: `4`) — number of
+ channels examined when selecting the idlest target.
+
+**More Information**
+* [FLIP-339](https://cwiki.apache.org/confluence/x/nYyzDw)
+
+## AdaptiveScheduler Rescale History and Web UI (FLIP-487 and FLIP-495)
+
+Streaming jobs using the adaptive scheduler now record a complete history of rescale events, including parallelism changes, slot allocations, scheduler state transitions, and termination reasons. This data is available through new REST endpoints and visualized in a dedicated "Rescales" tab in the Flink Web UI.
+
+The UI exposes:
+- Rescale event counts and timeline
+- Duration statistics with percentiles
+- Per-event details (parallelism before/after, triggered-by reason)
+- Adaptive scheduler configuration
+
+```yaml
+# Enable rescale history (disabled by default)
+web.adaptive-scheduler.rescale-history.size: 100
+```
+
+Setting the history size to a positive integer activates the feature.
+
+**More Information**
+* [FLIP-487](https://cwiki.apache.org/confluence/x/vZCMEw)
+* [FLIP-495](https://cwiki.apache.org/confluence/x/TQr0Ew)
+* [Elastic Scaling Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/elastic_scaling/)
+
+## Watermark Alignment for Fast Backlog Processing (FLINK-37399)
+
+Prior to Flink 2.3, watermark alignment due to the announcement delays was inadvertently
+limiting how quickly a job could process a backlog. For example with max allowed drift
+configured to 30s and watermark alignment updated every ~1s, prior to Flink 2.3 watermark
+alignment was de facto capping the backlog processing speed to:
+
+> 30 "event time" seconds per each 1 "real world" second
+
+In Flink 2.3 the watermark alignment was redesigned to solve those announcement delays by the
+introduction of a watermark alignment buffer. By default this buffer has a size of 3 and it
+delays the application of the watermark alignment algorithm by 3 update intervals. This means
+in Flink 2.3+, by default watermark alignment will be pausing sources a couple of seconds later
+than it used to, potentially increasing state size of windowed and temporal operators.
+However this should be negligible for all practical use cases. Nevertheless, the size of this
+buffer can be configured:
+
+```yaml
+# Control alignment buffer size (3 by default)
+pipeline.watermark-alignment.buffer-size: 3
+
+# Set to 0 to restore Flink 2.2 behavior
+pipeline.watermark-alignment.buffer-size: 0
+```
+
+**More Information**
+* [FLINK-37399](https://issues.apache.org/jira/browse/FLINK-37399)
+
+## Checkpointing During Recovery (FLIP-547)
Review Comment:
Hi, @1996fanrui Could you help take a look if you had the free time ? many thanks
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: Cleaner deployment with smaller footprint
+
+The plugin registers the standard `s3://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+**More Information**
+* [FLIP-555](https://cwiki.apache.org/confluence/x/uYqmFw)
+* [Native S3 FileSystem Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/filesystems/s3/)
+
+# Runtime Improvements
+
+## Adaptive Partition Selection for RPC-Heavy Workloads (FLIP-339)
+
+When upstream and downstream parallelism differ, Flink uses `RebalancePartitioner`, which selects
+target channels round-robin. For jobs that interact with external RPC services (Redis, HBase,
+LLM serving, etc.) round-robin selection causes severe backpressure as soon as a single
+downstream subtask slows down — the partitioner keeps feeding it new data even though it is
+already overloaded.
+
+Flink 2.3 adds an adaptive, load-aware partition-selection mode for
+`StreamPartitioner` that routes records to the least-loaded downstream channel instead. In
+benchmarks, this delivers up to ~3x throughput improvement under skewed downstream processing.
+
+The feature is opt-in via two new options:
+
+- `taskmanager.network.adaptive-partitioner.enabled` (default: `false`)
+- `taskmanager.network.adaptive-partitioner.max-traverse-size` (default: `4`) — number of
+ channels examined when selecting the idlest target.
+
+**More Information**
+* [FLIP-339](https://cwiki.apache.org/confluence/x/nYyzDw)
+
+## AdaptiveScheduler Rescale History and Web UI (FLIP-487 and FLIP-495)
+
+Streaming jobs using the adaptive scheduler now record a complete history of rescale events, including parallelism changes, slot allocations, scheduler state transitions, and termination reasons. This data is available through new REST endpoints and visualized in a dedicated "Rescales" tab in the Flink Web UI.
+
+The UI exposes:
+- Rescale event counts and timeline
+- Duration statistics with percentiles
+- Per-event details (parallelism before/after, triggered-by reason)
+- Adaptive scheduler configuration
+
+```yaml
+# Enable rescale history (disabled by default)
+web.adaptive-scheduler.rescale-history.size: 100
+```
+
+Setting the history size to a positive integer activates the feature.
+
+**More Information**
+* [FLIP-487](https://cwiki.apache.org/confluence/x/vZCMEw)
+* [FLIP-495](https://cwiki.apache.org/confluence/x/TQr0Ew)
+* [Elastic Scaling Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/elastic_scaling/)
+
+## Watermark Alignment for Fast Backlog Processing (FLINK-37399)
+
+Prior to Flink 2.3, watermark alignment due to the announcement delays was inadvertently
+limiting how quickly a job could process a backlog. For example with max allowed drift
+configured to 30s and watermark alignment updated every ~1s, prior to Flink 2.3 watermark
+alignment was de facto capping the backlog processing speed to:
+
+> 30 "event time" seconds per each 1 "real world" second
+
+In Flink 2.3 the watermark alignment was redesigned to solve those announcement delays by the
+introduction of a watermark alignment buffer. By default this buffer has a size of 3 and it
+delays the application of the watermark alignment algorithm by 3 update intervals. This means
+in Flink 2.3+, by default watermark alignment will be pausing sources a couple of seconds later
+than it used to, potentially increasing state size of windowed and temporal operators.
+However this should be negligible for all practical use cases. Nevertheless, the size of this
+buffer can be configured:
+
+```yaml
+# Control alignment buffer size (3 by default)
+pipeline.watermark-alignment.buffer-size: 3
+
+# Set to 0 to restore Flink 2.2 behavior
+pipeline.watermark-alignment.buffer-size: 0
+```
+
+**More Information**
+* [FLINK-37399](https://issues.apache.org/jira/browse/FLINK-37399)
+
+## Checkpointing During Recovery (FLIP-547)
+
+Flink now supports triggering checkpoints while a job is still recovering from unaligned
+checkpoints. Previously, a checkpoint could only be triggered after all restored channel state
+had been fully consumed; when the in-flight state was large, this meant no new checkpoint could
+complete for hours, and any restart or rescaling during that window forced the job to redo the
+entire recovery from the original checkpoint.
+
+With this feature enabled, jobs can checkpoint early during recovery, preserving work across
+subsequent restarts and scaling events. Exactly-once semantics are unchanged.
+
+The feature is disabled by default and can be enabled via two new options:
+
+- `execution.checkpointing.unaligned.recover-output-on-downstream.enabled` (default: `false`)
+- `execution.checkpointing.unaligned.during-recovery.enabled` (default: `false`, requires the
+ option above to be enabled)
+
+Enabling both is recommended for jobs with large unaligned checkpoint state or frequent
+rescaling.
+
+**More Information**
+* [FLIP-547](https://cwiki.apache.org/confluence/x/UQrxFg)
+
+## Application-Level Lifecycle Management (FLIP-549 and FLIP-560)
Review Comment:
Hi, @eemario
Could you help take a look if you had the free time ? many thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "Samrat002 (via GitHub)" <gi...@apache.org>.
Samrat002 commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3332040851
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
Review Comment:
```suggestion
Flink 2.3 introduces a ground-up S3 Filesystem with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
```
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: Cleaner deployment with smaller footprint
+
+The plugin registers the standard `s3://` scheme and uses a new `s3.*` configuration namespace:
Review Comment:
```suggestion
The plugin registers the standard `s3://` and `s3a://` scheme and uses a new `s3.*` configuration namespace:
```
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
Review Comment:
This plugin has shown better performance compared to existing `flink-s3-fs-hadoop` and `flink-s3-fs-presto` . Reference : https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=406620396&src=contextnavpagetreemode
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
Review Comment:
```suggestion
title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Filesystem Support, Improved Performance, and Application Management"
```
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
Review Comment:
This plugin is drop in replacement for Hadoop and Presto-based connectors
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: Cleaner deployment with smaller footprint
Review Comment:
```suggestion
- **Zero Hadoop dependencies**: No dependency mess with smaller footprint
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "Samrat002 (via GitHub)" <gi...@apache.org>.
Samrat002 commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3343367900
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers application management. Flink 2.3.0 brings together contributors from around the globe, implements full or core functionalities of 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+Flink 2.3 includes only these enhancements that enable more sophisticated temporal processing patterns and late data handling directly in SQL. The other features described in FLIP-565, around improved state access and broadcast state, are expected to follow in a future release.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up S3 Filesystem with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Better performance**: See [Benchmarking Native S3 FileSystem (flink-s3-fs-native) vs Presto S3 (flink-s3-fs-presto)](https://cwiki.apache.org/confluence/x/7Ig8G) for details
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: No dependency mess with smaller footprint
+
+The plugin registers the standard `s3://` and `s3a://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+Note: The Native S3 FileSystem is experimental in Flink 2.3. It is functionally complete and has demonstrated strong performance in benchmarks, but more experience is needed before it can be recommended for use in production.
Review Comment:
I think this is partially correct. We can remove the part below
> but more experience is needed before it can be recommended for use in production.
@alpinegizmo
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers application management. Flink 2.3.0 brings together contributors from around the globe, implements full or core functionalities of 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+Flink 2.3 includes only these enhancements that enable more sophisticated temporal processing patterns and late data handling directly in SQL. The other features described in FLIP-565, around improved state access and broadcast state, are expected to follow in a future release.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up S3 Filesystem with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Better performance**: See [Benchmarking Native S3 FileSystem (flink-s3-fs-native) vs Presto S3 (flink-s3-fs-presto)](https://cwiki.apache.org/confluence/x/7Ig8G) for details
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: No dependency mess with smaller footprint
+
+The plugin registers the standard `s3://` and `s3a://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+Note: The Native S3 FileSystem is experimental in Flink 2.3. It is functionally complete and has demonstrated strong performance in benchmarks, but more experience is needed before it can be recommended for use in production.
Review Comment:
I think this is partially correct. We can remove the part below
> but more experience is needed before it can be recommended for use in production.
@alpinegizmo WDUT ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3324474067
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
Review Comment:
```suggestion
title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Application Management"
```
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
Review Comment:
```suggestion
This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers application management. Flink 2.3.0 brings together contributors from around the globe, implements full or core functionalities of 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
```
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: Cleaner deployment with smaller footprint
+
+The plugin registers the standard `s3://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+**More Information**
+* [FLIP-555](https://cwiki.apache.org/confluence/x/uYqmFw)
+* [Native S3 FileSystem Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/filesystems/s3/)
+
+# Runtime Improvements
+
+## Adaptive Partition Selection for RPC-Heavy Workloads (FLIP-339)
+
+When upstream and downstream parallelism differ, Flink uses `RebalancePartitioner`, which selects
+target channels round-robin. For jobs that interact with external RPC services (Redis, HBase,
+LLM serving, etc.) round-robin selection causes severe backpressure as soon as a single
+downstream subtask slows down — the partitioner keeps feeding it new data even though it is
+already overloaded.
+
+Flink 2.3 adds an adaptive, load-aware partition-selection mode for
+`StreamPartitioner` that routes records to the least-loaded downstream channel instead. In
+benchmarks, this delivers up to ~3x throughput improvement under skewed downstream processing.
Review Comment:
```suggestion
## Support Adaptive Partition Selection (FLIP-339)
Uneven load across tasks in a Flink job may cause backpressure and latency.
For the `RebalancePartitioner` and `RescalePartitioner` data partitioning modes,
Flink 2.3 introduces an adaptive data partitioning feature based on downstream task load.
It distributes data dynamically according to downstream workloads to balance traffic and improve job throughput.
```
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: Cleaner deployment with smaller footprint
+
+The plugin registers the standard `s3://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+**More Information**
+* [FLIP-555](https://cwiki.apache.org/confluence/x/uYqmFw)
+* [Native S3 FileSystem Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/filesystems/s3/)
+
+# Runtime Improvements
+
+## Adaptive Partition Selection for RPC-Heavy Workloads (FLIP-339)
+
+When upstream and downstream parallelism differ, Flink uses `RebalancePartitioner`, which selects
+target channels round-robin. For jobs that interact with external RPC services (Redis, HBase,
+LLM serving, etc.) round-robin selection causes severe backpressure as soon as a single
+downstream subtask slows down — the partitioner keeps feeding it new data even though it is
+already overloaded.
+
+Flink 2.3 adds an adaptive, load-aware partition-selection mode for
+`StreamPartitioner` that routes records to the least-loaded downstream channel instead. In
+benchmarks, this delivers up to ~3x throughput improvement under skewed downstream processing.
+
+The feature is opt-in via two new options:
+
+- `taskmanager.network.adaptive-partitioner.enabled` (default: `false`)
+- `taskmanager.network.adaptive-partitioner.max-traverse-size` (default: `4`) — number of
+ channels examined when selecting the idlest target.
+
+**More Information**
+* [FLIP-339](https://cwiki.apache.org/confluence/x/nYyzDw)
+
+## AdaptiveScheduler Rescale History and Web UI (FLIP-487 and FLIP-495)
+
+Streaming jobs using the adaptive scheduler now record a complete history of rescale events, including parallelism changes, slot allocations, scheduler state transitions, and termination reasons. This data is available through new REST endpoints and visualized in a dedicated "Rescales" tab in the Flink Web UI.
Review Comment:
```suggestion
Streaming jobs using the adaptive scheduler now record a complete history of rescale events,
including parallelism changes, slot allocations, scheduler state transitions,
and termination reasons. This data is available through new REST endpoints and visualized in a dedicated "Rescales" tab in the Flink Web UI.
```
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: Cleaner deployment with smaller footprint
+
+The plugin registers the standard `s3://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+**More Information**
+* [FLIP-555](https://cwiki.apache.org/confluence/x/uYqmFw)
+* [Native S3 FileSystem Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/filesystems/s3/)
+
+# Runtime Improvements
+
+## Adaptive Partition Selection for RPC-Heavy Workloads (FLIP-339)
+
+When upstream and downstream parallelism differ, Flink uses `RebalancePartitioner`, which selects
+target channels round-robin. For jobs that interact with external RPC services (Redis, HBase,
+LLM serving, etc.) round-robin selection causes severe backpressure as soon as a single
+downstream subtask slows down — the partitioner keeps feeding it new data even though it is
+already overloaded.
+
+Flink 2.3 adds an adaptive, load-aware partition-selection mode for
+`StreamPartitioner` that routes records to the least-loaded downstream channel instead. In
+benchmarks, this delivers up to ~3x throughput improvement under skewed downstream processing.
+
+The feature is opt-in via two new options:
+
+- `taskmanager.network.adaptive-partitioner.enabled` (default: `false`)
+- `taskmanager.network.adaptive-partitioner.max-traverse-size` (default: `4`) — number of
+ channels examined when selecting the idlest target.
+
+**More Information**
+* [FLIP-339](https://cwiki.apache.org/confluence/x/nYyzDw)
+
+## AdaptiveScheduler Rescale History and Web UI (FLIP-487 and FLIP-495)
+
+Streaming jobs using the adaptive scheduler now record a complete history of rescale events, including parallelism changes, slot allocations, scheduler state transitions, and termination reasons. This data is available through new REST endpoints and visualized in a dedicated "Rescales" tab in the Flink Web UI.
+
+The UI exposes:
+- Rescale event counts and timeline
+- Duration statistics with percentiles
+- Per-event details (parallelism before/after, triggered-by reason)
+- Adaptive scheduler configuration
+
+```yaml
+# Enable rescale history (disabled by default)
+web.adaptive-scheduler.rescale-history.size: 100
+```
+
+Setting the history size to a positive integer activates the feature.
+
+**More Information**
+* [FLIP-487](https://cwiki.apache.org/confluence/x/vZCMEw)
+* [FLIP-495](https://cwiki.apache.org/confluence/x/TQr0Ew)
+* [Elastic Scaling Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/elastic_scaling/)
Review Comment:
```suggestion
* [Elastic Scaling / Rescale History](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/elastic_scaling/#rescale-history)
```
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
Review Comment:
```suggestion
## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables
```
It may be unnecessary to put the FLIP ID in the title, since related information is already included in each More Information section. Pls let me know your opinions.
If so, Could you please go through all the titles and remove the FLIP IDs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "alpinegizmo (via GitHub)" <gi...@apache.org>.
alpinegizmo commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3346991814
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers application management. Flink 2.3.0 brings together contributors from around the globe, implements full or core functionalities of 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+Flink 2.3 includes only these enhancements that enable more sophisticated temporal processing patterns and late data handling directly in SQL. The other features described in FLIP-565, around improved state access and broadcast state, are expected to follow in a future release.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up S3 Filesystem with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Better performance**: See [Benchmarking Native S3 FileSystem (flink-s3-fs-native) vs Presto S3 (flink-s3-fs-presto)](https://cwiki.apache.org/confluence/x/7Ig8G) for details
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: No dependency mess with smaller footprint
+
+The plugin registers the standard `s3://` and `s3a://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+Note: The Native S3 FileSystem is experimental in Flink 2.3. It is functionally complete and has demonstrated strong performance in benchmarks, but more experience is needed before it can be recommended for use in production.
Review Comment:
I've reworked this. See what you think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "Samrat002 (via GitHub)" <gi...@apache.org>.
Samrat002 commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3343367900
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers application management. Flink 2.3.0 brings together contributors from around the globe, implements full or core functionalities of 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+Flink 2.3 includes only these enhancements that enable more sophisticated temporal processing patterns and late data handling directly in SQL. The other features described in FLIP-565, around improved state access and broadcast state, are expected to follow in a future release.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up S3 Filesystem with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Better performance**: See [Benchmarking Native S3 FileSystem (flink-s3-fs-native) vs Presto S3 (flink-s3-fs-presto)](https://cwiki.apache.org/confluence/x/7Ig8G) for details
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: No dependency mess with smaller footprint
+
+The plugin registers the standard `s3://` and `s3a://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+Note: The Native S3 FileSystem is experimental in Flink 2.3. It is functionally complete and has demonstrated strong performance in benchmarks, but more experience is needed before it can be recommended for use in production.
Review Comment:
I think this is partially correct. We can remove the part below
> but more experience is needed before it can be recommended for use in production.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "Samrat002 (via GitHub)" <gi...@apache.org>.
Samrat002 commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3355922332
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers application management. Flink 2.3.0 brings together contributors from around the globe, implements full or core functionalities of 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+Flink 2.3 includes only these enhancements that enable more sophisticated temporal processing patterns and late data handling directly in SQL. The other features described in FLIP-565, around improved state access and broadcast state, are expected to follow in a future release.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up S3 Filesystem with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Better performance**: See [Benchmarking Native S3 FileSystem (flink-s3-fs-native) vs Presto S3 (flink-s3-fs-presto)](https://cwiki.apache.org/confluence/x/7Ig8G) for details
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: No dependency mess with smaller footprint
+
+The plugin registers the standard `s3://` and `s3a://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+Note: The Native S3 FileSystem is experimental in Flink 2.3. It is functionally complete and has demonstrated strong performance in benchmarks, but more experience is needed before it can be recommended for use in production.
Review Comment:
new wording looks good 🚀
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3324702012
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
Review Comment:
```suggestion
## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables
```
It may be unnecessary to put the FLIP ID in the title, since related information is already included in each More Information section. Pls let me know your opinions.
If so, Could you please go through all the titles and remove the FLIP IDs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "eemario (via GitHub)" <gi...@apache.org>.
eemario commented on code in PR #853:
URL: https://github.com/apache/flink-web/pull/853#discussion_r3332637452
##########
docs/content/posts/2026-06-10-release-2.3.0.md:
##########
@@ -0,0 +1,308 @@
+---
+title: "Apache Flink 2.3.0: Enhanced SQL Capabilities, Native S3 Support, Improved Performance, and Enterprise-Grade Application Management"
+date: "2026-06-10T00:00:00.000Z"
+aliases:
+- /news/2026/06/10/release-2.3.0.html
+authors:
+- flink:
+ name: "Apache Flink PMC"
+
+---
+
+The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.
+
+This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers enterprise-grade application management. Flink 2.3.0 brings together contributors from around the globe, implements 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.
+
+Key improvements in this release include new SQL operators for changelog manipulation (`FROM_CHANGELOG` and `TO_CHANGELOG`), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.
+
+We extend our heartfelt thanks to all contributors for making this release possible!
+
+Let's dive into the details.
+
+# Flink SQL Improvements
+
+## FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables (FLIP-564)
+
+The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working
+with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in
+Process Table Functions:
+
+- `FROM_CHANGELOG` converts an append-only stream that carries an operation column into a dynamic
+ table. A configurable `op_mapping` makes it straightforward to plug in custom CDC formats and
+ controls how rows with unmapped operation codes are treated.
+- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only
+ changelog stream. This is the first SQL-level operator that lets users convert retract or
+ upsert streams into append form — useful for archival, audit, writing to append-only sinks,
+ and working around pipelines that require an append-only table.
+
+The 2.3 release covers limited basic use cases for both. Future versions will extend both
+functions with `PARTITION BY`, `invalid_op_handling`, `produces_full_deletes` and more to make
+both features powerful and extensive.
+
+**More Information**
+* [FLIP-564](https://cwiki.apache.org/confluence/x/34k8G)
+
+## Materialized Table Evolution: DDL Parity and Refresh Control (FLIP-550 and FLIP-557)
+
+Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.
+
+First, `CREATE MATERIALIZED TABLE` now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. `ALTER MATERIALIZED TABLE` gains full DDL capabilities—`ADD`, `MODIFY`, and `DROP` operations for metadata and computed columns, plus `RENAME TO`, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.
+
+Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table's query changes. The new `START_MODE` clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.
+
+These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.
+
+**More Information**
+* [FLIP-550](https://cwiki.apache.org/confluence/x/XwobFw)
+* [FLIP-557](https://cwiki.apache.org/confluence/x/9oPMFw)
+
+## SinkUpsertMaterializer: Explicit Conflict Handling (FLIP-558)
+
+The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.
+
+Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.
+
+By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new `ON CONFLICT` clause that makes the behavior explicit. You choose how to handle conflicts: `DO NOTHING` (silent skip), `DO ERROR` (fail the job), or `DO DEDUPLICATE` (materialize and deduplicate, similar to what Flink has done until now):
+
+```sql
+INSERT INTO target_table
+SELECT * FROM source
+ON CONFLICT DO DEDUPLICATE;
+```
+
+Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:
+
+ - `table.exec.sink.upserts.compaction-mode` (default: `WATERMARK`) — `WATERMARK` or
+ `CHECKPOINT`.
+ - `table.exec.sink.upserts.compaction-interval` — optional fallback interval for emitting
+ watermarks when none arrive naturally.
+
+**More Information**
+* [FLIP-558](https://cwiki.apache.org/confluence/x/NoTMFw)
+
+## Process Table Function Enhancements (FLIP-565)
+
+Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:
+
+- **Late data handling**: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
+- **Ordered table arguments**: The new `ORDER BY` clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
+
+```sql
+SELECT * FROM
+ MyTimestampedPtf(
+ input => TABLE events PARTITION BY user_id ORDER BY event_time
+ );
+```
+
+These enhancements enable more sophisticated temporal processing patterns directly in SQL.
+
+**More Information**
+* [FLIP-565](https://cwiki.apache.org/confluence/x/qIo8G)
+
+## ARTIFACT Keyword for User-Defined Functions (FLIP-559)
+
+The `USING` clause of `CREATE FUNCTION` now accepts an `ARTIFACT` keyword as a future-proof alternative to `JAR`. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:
+
+```sql
+-- New syntax
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING ARTIFACT 's3://bucket/path/my-udf.jar';
+
+-- Existing syntax still works
+CREATE FUNCTION my_func AS 'com.example.MyUdf'
+ USING JAR 's3://bucket/path/my-udf.jar';
+```
+
+**More Information**
+* [FLIP-559](https://cwiki.apache.org/confluence/x/64TMFw)
+
+## Critical Bug Fix: MiniBatch Aggregation Record Loss
+
+Flink 2.3 fixes a critical bug in `MiniBatchGroupAggFunction` that could silently drop records when mini-batch aggregation was enabled and the planner used a `ONE_PHASE` aggregation strategy. The bug occurred when a key's mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.
+
+**More Information**
+* [FLINK-35661](https://issues.apache.org/jira/browse/FLINK-35661)
+
+# Connectors
+
+## Native S3 FileSystem
+
+Flink 2.3 introduces a ground-up rewrite of S3 connectivity with `flink-s3-fs-native`, a new plugin built directly on AWS SDK v2. This replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:
+
+- **Native AWS integration**: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
+- **Non-blocking I/O**: Asynchronous operations for improved throughput
+- **Unified implementation**: Single plugin provides both `FileSystem` and `RecoverableWriter` (exactly-once streaming sinks)
+- **Zero Hadoop dependencies**: Cleaner deployment with smaller footprint
+
+The plugin registers the standard `s3://` scheme and uses a new `s3.*` configuration namespace:
+
+```yaml
+s3.region: us-west-2
+s3.endpoint: https://s3.us-west-2.amazonaws.com
+s3.path-style-access: false
+s3.upload.min.part.size: 5MB
+s3.upload.max.concurrent.uploads: 4
+s3.async.enabled: true
+s3.read.buffer.size: 64KB
+s3.bulk-copy.enabled: true
+```
+
+Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.
+
+**More Information**
+* [FLIP-555](https://cwiki.apache.org/confluence/x/uYqmFw)
+* [Native S3 FileSystem Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/filesystems/s3/)
+
+# Runtime Improvements
+
+## Adaptive Partition Selection for RPC-Heavy Workloads (FLIP-339)
+
+When upstream and downstream parallelism differ, Flink uses `RebalancePartitioner`, which selects
+target channels round-robin. For jobs that interact with external RPC services (Redis, HBase,
+LLM serving, etc.) round-robin selection causes severe backpressure as soon as a single
+downstream subtask slows down — the partitioner keeps feeding it new data even though it is
+already overloaded.
+
+Flink 2.3 adds an adaptive, load-aware partition-selection mode for
+`StreamPartitioner` that routes records to the least-loaded downstream channel instead. In
+benchmarks, this delivers up to ~3x throughput improvement under skewed downstream processing.
+
+The feature is opt-in via two new options:
+
+- `taskmanager.network.adaptive-partitioner.enabled` (default: `false`)
+- `taskmanager.network.adaptive-partitioner.max-traverse-size` (default: `4`) — number of
+ channels examined when selecting the idlest target.
+
+**More Information**
+* [FLIP-339](https://cwiki.apache.org/confluence/x/nYyzDw)
+
+## AdaptiveScheduler Rescale History and Web UI (FLIP-487 and FLIP-495)
+
+Streaming jobs using the adaptive scheduler now record a complete history of rescale events, including parallelism changes, slot allocations, scheduler state transitions, and termination reasons. This data is available through new REST endpoints and visualized in a dedicated "Rescales" tab in the Flink Web UI.
+
+The UI exposes:
+- Rescale event counts and timeline
+- Duration statistics with percentiles
+- Per-event details (parallelism before/after, triggered-by reason)
+- Adaptive scheduler configuration
+
+```yaml
+# Enable rescale history (disabled by default)
+web.adaptive-scheduler.rescale-history.size: 100
+```
+
+Setting the history size to a positive integer activates the feature.
+
+**More Information**
+* [FLIP-487](https://cwiki.apache.org/confluence/x/vZCMEw)
+* [FLIP-495](https://cwiki.apache.org/confluence/x/TQr0Ew)
+* [Elastic Scaling Documentation](https://nightlies.apache.org/flink/flink-docs-release-2.3/docs/deployment/elastic_scaling/)
+
+## Watermark Alignment for Fast Backlog Processing (FLINK-37399)
+
+Prior to Flink 2.3, watermark alignment due to the announcement delays was inadvertently
+limiting how quickly a job could process a backlog. For example with max allowed drift
+configured to 30s and watermark alignment updated every ~1s, prior to Flink 2.3 watermark
+alignment was de facto capping the backlog processing speed to:
+
+> 30 "event time" seconds per each 1 "real world" second
+
+In Flink 2.3 the watermark alignment was redesigned to solve those announcement delays by the
+introduction of a watermark alignment buffer. By default this buffer has a size of 3 and it
+delays the application of the watermark alignment algorithm by 3 update intervals. This means
+in Flink 2.3+, by default watermark alignment will be pausing sources a couple of seconds later
+than it used to, potentially increasing state size of windowed and temporal operators.
+However this should be negligible for all practical use cases. Nevertheless, the size of this
+buffer can be configured:
+
+```yaml
+# Control alignment buffer size (3 by default)
+pipeline.watermark-alignment.buffer-size: 3
+
+# Set to 0 to restore Flink 2.2 behavior
+pipeline.watermark-alignment.buffer-size: 0
+```
+
+**More Information**
+* [FLINK-37399](https://issues.apache.org/jira/browse/FLINK-37399)
+
+## Checkpointing During Recovery (FLIP-547)
+
+Flink now supports triggering checkpoints while a job is still recovering from unaligned
+checkpoints. Previously, a checkpoint could only be triggered after all restored channel state
+had been fully consumed; when the in-flight state was large, this meant no new checkpoint could
+complete for hours, and any restart or rescaling during that window forced the job to redo the
+entire recovery from the original checkpoint.
+
+With this feature enabled, jobs can checkpoint early during recovery, preserving work across
+subsequent restarts and scaling events. Exactly-once semantics are unchanged.
+
+The feature is disabled by default and can be enabled via two new options:
+
+- `execution.checkpointing.unaligned.recover-output-on-downstream.enabled` (default: `false`)
+- `execution.checkpointing.unaligned.during-recovery.enabled` (default: `false`, requires the
+ option above to be enabled)
+
+Enabling both is recommended for jobs with large unaligned checkpoint state or frequent
+rescaling.
+
+**More Information**
+* [FLIP-547](https://cwiki.apache.org/confluence/x/UQrxFg)
+
+## Application-Level Lifecycle Management (FLIP-549 and FLIP-560)
Review Comment:
This part LGTM. Thanks for your effort!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Flink 2.3.0 release [flink-web]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on PR #853:
URL: https://github.com/apache/flink-web/pull/853#issuecomment-4802345871
Hi @alpinegizmo
Would you mind helping resolve the conflicts before the next review? Thx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org