You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/10/11 17:19:42 UTC
(datafusion-comet) branch main updated (6296625b90 -> d8d37fc527)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
from 6296625b90 perf: presize collections and reuse buffers in native loops (#6861)
add d8d37fc527 fix: resolve Azure ABFS authentication through Hadoop and keep ambient credentials out (#6059)
No new revisions were added by this update.
Summary of changes:
.github/workflows/pr_build_linux.yml | 3 +
.github/workflows/pr_build_macos.yml | 3 +
dev/release/rat_exclude_files.txt | 1 +
docs/source/contributor-guide/development.md | 32 +
docs/source/user-guide/latest/datasources.md | 105 +-
native/core/src/parquet/objectstore/azure.rs | 1167 +++++++++++++++-----
native/core/src/parquet/parquet_support.rs | 6 +-
pom.xml | 13 +
spark/pom.xml | 33 +
.../comet/objectstore/AbfsAuthResolver.scala | 517 +++++++++
.../apache/comet/objectstore/AbfsReflection.scala | 299 +++++
.../apache/comet/objectstore/NativeConfig.scala | 54 +-
.../org/apache/comet/rules/CometExecRule.scala | 2 +-
.../org/apache/comet/rules/CometScanRule.scala | 9 +-
.../comet/serde/operator/CometNativeScan.scala | 37 +-
.../spark/sql/comet/CometCsvNativeScanExec.scala | 22 +-
...hemeFileSystem.java => FakeAbfsFileSystem.java} | 27 +-
.../resources/abfs-auth-parity/expected-3.3.4.json | 876 +++++++++++++++
.../resources/abfs-auth-parity/expected-3.4.1.json | 936 ++++++++++++++++
.../resources/abfs-auth-parity/expected-3.4.2.json | 947 ++++++++++++++++
.../resources/abfs-auth-parity/expected-3.5.0.json | 943 ++++++++++++++++
.../comet/objectstore/AbfsAuthParityCases.scala | 1014 +++++++++++++++++
.../comet/objectstore/AbfsAuthParitySuite.scala | 422 +++++++
.../comet/objectstore/AbfsAuthResolverSuite.scala | 703 ++++++++++++
.../comet/objectstore/AbfsScanFallbackSuite.scala | 149 +++
.../comet/objectstore/NativeConfigSuite.scala | 179 ++-
26 files changed, 8084 insertions(+), 415 deletions(-)
create mode 100644 spark/src/main/scala/org/apache/comet/objectstore/AbfsAuthResolver.scala
create mode 100644 spark/src/main/scala/org/apache/comet/objectstore/AbfsReflection.scala
copy spark/src/test/java/org/apache/comet/hadoop/fs/{FakeHdfsSchemeFileSystem.java => FakeAbfsFileSystem.java} (58%)
create mode 100644 spark/src/test/resources/abfs-auth-parity/expected-3.3.4.json
create mode 100644 spark/src/test/resources/abfs-auth-parity/expected-3.4.1.json
create mode 100644 spark/src/test/resources/abfs-auth-parity/expected-3.4.2.json
create mode 100644 spark/src/test/resources/abfs-auth-parity/expected-3.5.0.json
create mode 100644 spark/src/test/scala/org/apache/comet/objectstore/AbfsAuthParityCases.scala
create mode 100644 spark/src/test/scala/org/apache/comet/objectstore/AbfsAuthParitySuite.scala
create mode 100644 spark/src/test/scala/org/apache/comet/objectstore/AbfsAuthResolverSuite.scala
create mode 100644 spark/src/test/scala/org/apache/comet/objectstore/AbfsScanFallbackSuite.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org