(datafusion-comet) branch main updated (7bd6e9b047 -> 1aa166fde8)

[gi...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git


    from 7bd6e9b047 fix: compact already-sliced batches before the native Iceberg writer counts NaNs (#6238)
     add 1aa166fde8 perf: fuse Comet cache vector reads into Spark codegen (#5859)

No new revisions were added by this update.

Summary of changes:
 .github/workflows/pr_build_linux.yml               |   1 +
 .github/workflows/pr_build_macos.yml               |   1 +
 docs/source/user-guide/latest/in-memory-cache.md   |  55 ++++--
 .../main/scala/org/apache/comet/CometConf.scala    |  43 +++--
 .../apache/comet/CometSparkSessionExtensions.scala |   4 +-
 .../comet/rules/CometCacheColumnarRule.scala       | 100 ++++++++++
 .../scala/org/apache/comet/rules/CometRule.scala   |  38 ++--
 .../arrow/ArrowCachedBatchSerializer.scala         |   8 +-
 .../execution/arrow/CachedBatchRowIterator.scala   | 186 ++++++++++++++++++
 .../comet/exec/CometInMemoryCacheSuite.scala       | 186 +++++++++++++++++-
 .../benchmark/CometInMemoryCacheBenchmark.scala    | 187 ++++++++++++++----
 .../arrow/CachedBatchRowIteratorSuite.scala        | 212 +++++++++++++++++++++
 12 files changed, 923 insertions(+), 98 deletions(-)
 create mode 100644 spark/src/main/scala/org/apache/comet/rules/CometCacheColumnarRule.scala
 create mode 100644 spark/src/main/scala/org/apache/spark/sql/comet/execution/arrow/CachedBatchRowIterator.scala
 create mode 100644 spark/src/test/scala/org/apache/spark/sql/comet/execution/arrow/CachedBatchRowIteratorSuite.scala


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org