You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/10/16 22:33:25 UTC
(datafusion-comet) branch main updated (3a4a159409 -> c4ca20e8b6)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
from 3a4a159409 fix: read fs.comet.libhdfs.schemes for the scan rule from the Hadoop conf (#6838)
add 1bad7c9e5f perf: make the native Iceberg writer's dictionary pre-pass 1.8x to 3.9x faster (#6937)
add e242944ec4 fix: keep unconverted DPP filters in a scan's canonical form (#6270)
add c4ca20e8b6 fix: fall back to Spark for NOT IN over an ordered streamed side (#6810)
No new revisions were added by this update.
Summary of changes:
.../spark/sql/comet/CometDeltaNativeScanExec.scala | 9 +-
.../contrib/delta/CometDeltaNativeScanSuite.scala | 108 ++-
.../contributor-guide/adding_a_new_operator.md | 13 +-
docs/source/contributor-guide/iceberg-writes.md | 1 +
.../user-guide/latest/compatibility/operators.md | 8 +
docs/source/user-guide/latest/operators.md | 5 +-
native/core/Cargo.toml | 4 +
native/core/benches/iceberg_dictionary_chooser.rs | 444 ++++++++++++
.../src/execution/operators/iceberg_dictionary.rs | 771 ++++++++++++++++-----
native/core/src/execution/operators/mod.rs | 3 +
.../main/scala/org/apache/comet/CometConf.scala | 3 +-
.../spark/sql/comet/CometNativeScanExec.scala | 14 +-
.../apache/spark/sql/comet/CometScanUtils.scala | 37 +-
.../org/apache/spark/sql/comet/operators.scala | 12 +
.../org/apache/comet/CometIcebergNativeSuite.scala | 81 +++
.../org/apache/comet/exec/CometExecSuite.scala | 123 ++++
.../org/apache/comet/exec/CometJoinSuite.scala | 178 ++++-
.../scala/org/apache/spark/sql/CometTestBase.scala | 38 +-
18 files changed, 1627 insertions(+), 225 deletions(-)
create mode 100644 native/core/benches/iceberg_dictionary_chooser.rs
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org