You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by mb...@apache.org on 2025/11/21 11:55:47 UTC
(datafusion-comet) branch main updated: chore: add 0.12.0 changelog (#2811)
This is an automated email from the ASF dual-hosted git repository.
mbutrovich pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new c2f2cc412 chore: add 0.12.0 changelog (#2811)
c2f2cc412 is described below
commit c2f2cc4122041a1be5321d8997a509e227888400
Author: Andy Grove <ag...@apache.org>
AuthorDate: Fri Nov 21 04:55:43 2025 -0700
chore: add 0.12.0 changelog (#2811)
---
dev/changelog/0.12.0.md | 161 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 161 insertions(+)
diff --git a/dev/changelog/0.12.0.md b/dev/changelog/0.12.0.md
new file mode 100644
index 000000000..5eeef1d10
--- /dev/null
+++ b/dev/changelog/0.12.0.md
@@ -0,0 +1,161 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# DataFusion Comet 0.12.0 Changelog
+
+This release consists of 105 commits from 13 contributors. See credits at the end of this changelog for more information.
+
+**Fixed bugs:**
+
+- fix: Fix `None.get` in `stringDecode` when `bin` child cannot be converted [#2606](https://github.com/apache/datafusion-comet/pull/2606) (cfmcgrady)
+- fix: Update FuzzDataGenerator to produce dictionary-encoded string arrays & fix bugs that this exposes [#2635](https://github.com/apache/datafusion-comet/pull/2635) (andygrove)
+- fix: Fallback to Spark for lpad/rpad for unsupported arguments & fix negative length handling [#2630](https://github.com/apache/datafusion-comet/pull/2630) (andygrove)
+- fix: Mark SortOrder with floating-point as incompatible [#2650](https://github.com/apache/datafusion-comet/pull/2650) (andygrove)
+- fix: Fall back to Spark for `trunc` / `date_trunc` functions when format string is unsupported, or is not a literal value [#2634](https://github.com/apache/datafusion-comet/pull/2634) (andygrove)
+- fix: [native_datafusion] only pass single partition of PartitionedFiles into DataSourceExec [#2675](https://github.com/apache/datafusion-comet/pull/2675) (mbutrovich)
+- fix: Fix subcommands options in fuzz-testing [#2684](https://github.com/apache/datafusion-comet/pull/2684) (manuzhang)
+- fix: Do not replace SMJ with HJ for `LeftSemi` [#2687](https://github.com/apache/datafusion-comet/pull/2687) (comphead)
+- fix: Apply spotless on Iceberg 1.8.1 diff [iceberg] [#2700](https://github.com/apache/datafusion-comet/pull/2700) (hsiang-c)
+- fix: Fix generate-user-guide-reference-docs failure when mvn command is not executed at root [#2691](https://github.com/apache/datafusion-comet/pull/2691) (manuzhang)
+- fix: Fix missing SortOrder fallback reason in range partitioning [#2716](https://github.com/apache/datafusion-comet/pull/2716) (andygrove)
+- fix: CometLiteral class cast exception with arrays [#2718](https://github.com/apache/datafusion-comet/pull/2718) (andygrove)
+- fix: NormalizeNaNAndZero::children() returns child's child [#2732](https://github.com/apache/datafusion-comet/pull/2732) (mbutrovich)
+- fix: checkSparkMaybeThrows should compare Spark and Comet results in success case [#2728](https://github.com/apache/datafusion-comet/pull/2728) (andygrove)
+- fix: Mark `WindowsExec` as incompatible [#2748](https://github.com/apache/datafusion-comet/pull/2748) (andygrove)
+- fix: Add strict floating point mode and fallback to Spark for min/max/sort on floating point inputs when enabled [#2747](https://github.com/apache/datafusion-comet/pull/2747) (andygrove)
+- fix: Implement producedAttributes for CometWindowExec [#2789](https://github.com/apache/datafusion-comet/pull/2789) (rahulbabarwal89)
+- fix: Pass all Comet configs to native plan [#2801](https://github.com/apache/datafusion-comet/pull/2801) (andygrove)
+
+**Implemented enhancements:**
+
+- feat: Add option to write benchmark results to file [#2640](https://github.com/apache/datafusion-comet/pull/2640) (andygrove)
+- feat: Implement metrics for iceberg compat [#2615](https://github.com/apache/datafusion-comet/pull/2615) (EmilyMatt)
+- feat: Define function signatures in CometFuzz [#2614](https://github.com/apache/datafusion-comet/pull/2614) (andygrove)
+- feat: cherry-pick UUID conversion logic from #2528 [#2648](https://github.com/apache/datafusion-comet/pull/2648) (mbutrovich)
+- feat: support `concat` for strings [#2604](https://github.com/apache/datafusion-comet/pull/2604) (comphead)
+- feat: Add support for `abs` [#2689](https://github.com/apache/datafusion-comet/pull/2689) (andygrove)
+- feat: Support variadic function in CometFuzz [#2682](https://github.com/apache/datafusion-comet/pull/2682) (manuzhang)
+- feat: CometExecRule refactor: Unify CometNativeExec creation with Serde in CometOperatorSerde trait [#2768](https://github.com/apache/datafusion-comet/pull/2768) (andygrove)
+- feat: support cot [#2755](https://github.com/apache/datafusion-comet/pull/2755) (psvri)
+- feat: Add bash script to build and run fuzz testing [#2686](https://github.com/apache/datafusion-comet/pull/2686) (manuzhang)
+- feat: Add getSupportLevel to CometAggregateExpressionSerde trait [#2777](https://github.com/apache/datafusion-comet/pull/2777) (andygrove)
+- feat: Add CI check to ensure generated docs are in sync with code [#2779](https://github.com/apache/datafusion-comet/pull/2779) (andygrove)
+- feat: Add prettier enforcement [#2783](https://github.com/apache/datafusion-comet/pull/2783) (andygrove)
+- feat: hyperbolic trig functions [#2784](https://github.com/apache/datafusion-comet/pull/2784) (psvri)
+- feat: [iceberg] Native scan by serializing FileScanTasks to iceberg-rust [#2528](https://github.com/apache/datafusion-comet/pull/2528) (mbutrovich)
+
+**Documentation updates:**
+
+- docs: Add changelog for 0.11.0 release [#2585](https://github.com/apache/datafusion-comet/pull/2585) (mbutrovich)
+- docs: Improve documentation layout [#2587](https://github.com/apache/datafusion-comet/pull/2587) (andygrove)
+- docs: Publish 0.11.0 user guide [#2589](https://github.com/apache/datafusion-comet/pull/2589) (andygrove)
+- docs: Put Comet logo in top nav bar, respect light/dark mode [#2591](https://github.com/apache/datafusion-comet/pull/2591) (andygrove)
+- docs: Improve main landing page [#2593](https://github.com/apache/datafusion-comet/pull/2593) (andygrove)
+- docs: Improve site navigation [#2597](https://github.com/apache/datafusion-comet/pull/2597) (andygrove)
+- docs: Update benchmark results [#2596](https://github.com/apache/datafusion-comet/pull/2596) (andygrove)
+- docs: Upgrade pydata-sphinx-theme to 0.16.1 [#2602](https://github.com/apache/datafusion-comet/pull/2602) (andygrove)
+- docs: Fix redirect [#2603](https://github.com/apache/datafusion-comet/pull/2603) (andygrove)
+- docs: Fix broken image link [#2613](https://github.com/apache/datafusion-comet/pull/2613) (andygrove)
+- docs: Add FFI docs to contributor guide [#2668](https://github.com/apache/datafusion-comet/pull/2668) (andygrove)
+- docs: Various documentation updates [#2674](https://github.com/apache/datafusion-comet/pull/2674) (andygrove)
+- docs: Add supported SortOrder expressions and fix a typo [#2694](https://github.com/apache/datafusion-comet/pull/2694) (andygrove)
+- docs: Minor docs update for running Spark SQL tests [#2712](https://github.com/apache/datafusion-comet/pull/2712) (andygrove)
+- docs: Update contributor guide for adding a new expression [#2704](https://github.com/apache/datafusion-comet/pull/2704) (andygrove)
+- docs: Documentation updates for `LocalTableScan` and `WindowExec` [#2742](https://github.com/apache/datafusion-comet/pull/2742) (andygrove)
+- docs: Typo fix [#2752](https://github.com/apache/datafusion-comet/pull/2752) (wForget)
+- docs: Categorize some configs as `testing` and add notes about known time zone issues [#2740](https://github.com/apache/datafusion-comet/pull/2740) (andygrove)
+- docs: Run prettier on all markdown files [#2782](https://github.com/apache/datafusion-comet/pull/2782) (andygrove)
+- docs: Ignore prettier formatting for generated tables [#2790](https://github.com/apache/datafusion-comet/pull/2790) (andygrove)
+- docs: Add new section to contributor guide, explaining how to add a new operator [#2758](https://github.com/apache/datafusion-comet/pull/2758) (andygrove)
+
+**Other:**
+
+- chore: Start 0.12.0 development [#2584](https://github.com/apache/datafusion-comet/pull/2584) (mbutrovich)
+- chore: Bump Spark from 3.5.6 to 3.5.7 [#2574](https://github.com/apache/datafusion-comet/pull/2574) (cfmcgrady)
+- chore(deps): bump parquet from 56.0.0 to 56.2.0 in /native [#2608](https://github.com/apache/datafusion-comet/pull/2608) (dependabot[bot])
+- chore(deps): bump tikv-jemallocator from 0.6.0 to 0.6.1 in /native [#2609](https://github.com/apache/datafusion-comet/pull/2609) (dependabot[bot])
+- chore(deps): bump tikv-jemalloc-ctl from 0.6.0 to 0.6.1 in /native [#2610](https://github.com/apache/datafusion-comet/pull/2610) (dependabot[bot])
+- tests: FuzzDataGenerator instead of Parquet-specific generator [#2616](https://github.com/apache/datafusion-comet/pull/2616) (mbutrovich)
+- chore: Simplify on-heap memory configuration [#2599](https://github.com/apache/datafusion-comet/pull/2599) (andygrove)
+- Feat: Add sha1 function impl [#2471](https://github.com/apache/datafusion-comet/pull/2471) (kazantsev-maksim)
+- chore: Refactor Parquet/DataFrame fuzz data generators [#2629](https://github.com/apache/datafusion-comet/pull/2629) (andygrove)
+- chore: Remove needless from_raw calls [#2638](https://github.com/apache/datafusion-comet/pull/2638) (EmilyMatt)
+- chore: support DataFusion 50.3.0 [#2605](https://github.com/apache/datafusion-comet/pull/2605) (comphead)
+- chore(deps): bump actions/upload-artifact from 4 to 5 [#2654](https://github.com/apache/datafusion-comet/pull/2654) (dependabot[bot])
+- chore(deps): bump cc from 1.2.42 to 1.2.43 in /native [#2653](https://github.com/apache/datafusion-comet/pull/2653) (dependabot[bot])
+- chore(deps): bump actions/download-artifact from 5 to 6 [#2652](https://github.com/apache/datafusion-comet/pull/2652) (dependabot[bot])
+- chore: extract comparison into separate tool [#2632](https://github.com/apache/datafusion-comet/pull/2632) (comphead)
+- chore: Various improvements to `checkSparkAnswer*` methods in `CometTestBase` [#2656](https://github.com/apache/datafusion-comet/pull/2656) (andygrove)
+- chore: Remove code for unpacking dictionaries prior to FilterExec [#2659](https://github.com/apache/datafusion-comet/pull/2659) (andygrove)
+- chore: display schema for datasets being compared [#2665](https://github.com/apache/datafusion-comet/pull/2665) (comphead)
+- chore: Remove `CopyExec` [#2663](https://github.com/apache/datafusion-comet/pull/2663) (andygrove)
+- chore: Add extended explain plans to stability suite [#2669](https://github.com/apache/datafusion-comet/pull/2669) (andygrove)
+- chore(deps): bump aws-config from 1.8.8 to 1.8.10 in /native [#2677](https://github.com/apache/datafusion-comet/pull/2677) (dependabot[bot])
+- chore(deps): bump cc from 1.2.43 to 1.2.44 in /native [#2678](https://github.com/apache/datafusion-comet/pull/2678) (dependabot[bot])
+- chore: `tpcbench` output `explain` just once and formatted [#2679](https://github.com/apache/datafusion-comet/pull/2679) (comphead)
+- chore: Add tolerance for `ComparisonTool` [#2699](https://github.com/apache/datafusion-comet/pull/2699) (comphead)
+- chore: Expand test coverage for `CometWindowsExec` [#2711](https://github.com/apache/datafusion-comet/pull/2711) (comphead)
+- chore: generate Float/Double NaN [#2695](https://github.com/apache/datafusion-comet/pull/2695) (hsiang-c)
+- minor: Combine two CI workflows for Spark SQL tests [#2727](https://github.com/apache/datafusion-comet/pull/2727) (andygrove)
+- chore: Improve framework for specifying that configs can be set with env vars [#2722](https://github.com/apache/datafusion-comet/pull/2722) (andygrove)
+- chore: Rename `COMET_EXPLAIN_VERBOSE_ENABLED` to `COMET_EXTENDED_EXPLAIN_FORMAT` and change default [#2644](https://github.com/apache/datafusion-comet/pull/2644) (andygrove)
+- chore: Fallback to Spark for windows functions [#2726](https://github.com/apache/datafusion-comet/pull/2726) (comphead)
+- chore: Refactor operator serde - part 1 [#2738](https://github.com/apache/datafusion-comet/pull/2738) (andygrove)
+- Feat: Add CometLocalTableScanExec operator [#2735](https://github.com/apache/datafusion-comet/pull/2735) (kazantsev-maksim)
+- chore(deps): bump cc from 1.2.44 to 1.2.45 in /native [#2750](https://github.com/apache/datafusion-comet/pull/2750) (dependabot[bot])
+- chore(deps): bump aws-credential-types from 1.2.8 to 1.2.9 in /native [#2751](https://github.com/apache/datafusion-comet/pull/2751) (dependabot[bot])
+- chore: Operator serde refactor part 2 [#2741](https://github.com/apache/datafusion-comet/pull/2741) (andygrove)
+- chore: Fallback to Spark for `array_reverse` for `array<binary>` [#2759](https://github.com/apache/datafusion-comet/pull/2759) (comphead)
+- chore: [iceberg] test iceberg 1.10.0 [#2709](https://github.com/apache/datafusion-comet/pull/2709) (manuzhang)
+- chore: Add `docs/comet-*` to rat exclude list [#2762](https://github.com/apache/datafusion-comet/pull/2762) (manuzhang)
+- Chore: Refactor static invoke exprs [#2671](https://github.com/apache/datafusion-comet/pull/2671) (kazantsev-maksim)
+- minor: Small refactor for consistent serde for hash aggregate [#2764](https://github.com/apache/datafusion-comet/pull/2764) (andygrove)
+- minor: Move `operator2Proto` to `CometExecRule` [#2767](https://github.com/apache/datafusion-comet/pull/2767) (andygrove)
+- chore: various refactoring changes for iceberg [iceberg] [#2680](https://github.com/apache/datafusion-comet/pull/2680) (parthchandra)
+- chore: Refactor CometExecRule handling of sink operators [#2771](https://github.com/apache/datafusion-comet/pull/2771) (andygrove)
+- minor: Refactor to move window-specific code from `QueryPlanSerde` to `CometWindowExec` [#2780](https://github.com/apache/datafusion-comet/pull/2780) (andygrove)
+- chore: Remove many references to `COMET_EXPR_ALLOW_INCOMPATIBLE` [#2775](https://github.com/apache/datafusion-comet/pull/2775) (andygrove)
+- chore: Remove COMET_EXPR_ALLOW_INCOMPATIBLE config [#2786](https://github.com/apache/datafusion-comet/pull/2786) (andygrove)
+- chore: check `missingInput` for Comet plan nodes [#2795](https://github.com/apache/datafusion-comet/pull/2795) (comphead)
+- chore: Finish refactoring expression serde out of `QueryPlanSerde` [#2791](https://github.com/apache/datafusion-comet/pull/2791) (andygrove)
+- chore: Update docs to fix CI after #2784 [#2799](https://github.com/apache/datafusion-comet/pull/2799) (mbutrovich)
+- chore: Update q79 golden plan for Spark 4.0 after #2795 [#2800](https://github.com/apache/datafusion-comet/pull/2800) (mbutrovich)
+- Fix: Fix null handling in CometVector implementations [#2643](https://github.com/apache/datafusion-comet/pull/2643) (cfmcgrady)
+
+## Credits
+
+Thank you to everyone who contributed to this release. Here is a breakdown of commits (PRs merged) per contributor.
+
+```
+ 54 Andy Grove
+ 11 Oleks V
+ 10 dependabot[bot]
+ 9 Matt Butrovich
+ 6 Manu Zhang
+ 3 Fu Chen
+ 3 Kazantsev Maksim
+ 2 Emily Matheys
+ 2 Vrishabh
+ 2 hsiang-c
+ 1 Parth Chandra
+ 1 Zhen Wang
+ 1 rahulbabarwal89
+```
+
+Thank you also to everyone who contributed in other ways such as filing issues, reviewing PRs, and providing feedback on this release.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org