You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/10/16 19:43:52 UTC
(datafusion-comet) branch asf-site updated: Publish built docs triggered by 3a4a15940983e4bb9a9a74c816ee51e2cf6d9e17
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new c726a5e1ed Publish built docs triggered by 3a4a15940983e4bb9a9a74c816ee51e2cf6d9e17
c726a5e1ed is described below
commit c726a5e1ed9a9dd2e754019d62e4213076724eb1
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Fri Oct 16 19:43:46 2026 +0000
Publish built docs triggered by 3a4a15940983e4bb9a9a74c816ee51e2cf6d9e17
---
_images/mermaid-09341aca03757d9d3637a2eaff73472d472d0673.svg | 2 +-
_sources/user-guide/latest/configs.md.txt | 2 +-
searchindex.js | 2 +-
user-guide/latest/configs.html | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/_images/mermaid-09341aca03757d9d3637a2eaff73472d472d0673.svg b/_images/mermaid-09341aca03757d9d3637a2eaff73472d472d0673.svg
index acb884108e..6b9a1dda9f 100644
--- a/_images/mermaid-09341aca03757d9d3637a2eaff73472d472d0673.svg
+++ b/_images/mermaid-09341aca03757d9d3637a2eaff73472d472d0673.svg
@@ -1 +1 @@
-<svg id="my-svg" width="100%" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" class="flowchart" style="max-width: 511.73px; background-color: transparent;" viewBox="0 0 511.7299499511719 1014" role="graphics-document document" aria-roledescription="flowchart-v2"><style>#my-svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#my-sv [...]
\ No newline at end of file
+<svg id="my-svg" width="100%" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" class="flowchart" style="max-width: 511.73px; background-color: transparent;" viewBox="0 0 511.7299499511719 1014" role="graphics-document document" aria-roledescription="flowchart-v2"><style>#my-svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#my-sv [...]
\ No newline at end of file
diff --git a/_sources/user-guide/latest/configs.md.txt b/_sources/user-guide/latest/configs.md.txt
index fe8f9918b3..cf23f66dc7 100644
--- a/_sources/user-guide/latest/configs.md.txt
+++ b/_sources/user-guide/latest/configs.md.txt
@@ -33,7 +33,7 @@ Comet provides the following configuration settings.
| `spark.comet.scan.icebergNative.enabled` | Whether to enable native Iceberg table scan using iceberg-rust. When enabled, Iceberg tables are read directly through native execution, bypassing Spark's DataSource V2 API for better performance. | true |
| `spark.comet.scan.icebergNative.nestedSchemaPruning.enabled` | Whether the native Iceberg scan reads only the nested fields of a struct, list, or map column that Spark's nested schema pruning keeps. When false, it reads every nested field of a projected column and drops the unused ones after decoding. | true |
| `spark.comet.scan.unsignedSmallIntSafetyCheck` | Parquet files may contain unsigned 8-bit integers (UINT_8) which Spark maps to ShortType. When this config is true (default), Comet falls back to Spark for ShortType columns because we cannot distinguish signed INT16 (safe) from unsigned UINT_8 (may produce different results). Set to false to allow native execution of ShortType columns if you know your data does not contain unsigned UINT_8 columns from improperly encoded Parquet files. F [...]
-| `spark.hadoop.fs.comet.libhdfs.schemes` | Defines filesystem schemes (e.g., hdfs, webhdfs) that the native side accesses via libhdfs, separated by commas. Valid only when built with hdfs-opendal feature enabled. | |
+| `spark.hadoop.fs.comet.libhdfs.schemes` | Defines filesystem schemes (e.g., hdfs, webhdfs) that the native side accesses via libhdfs, separated by commas. Valid only when built with hdfs-opendal feature enabled. Read from the Hadoop configuration, so it must be set before the SparkSession is created. A scheme listed only through a reader option is not read natively. libhdfs reads the settings of a listed scheme, such as `fs.<scheme>.impl`, from the executor's `core-site.xml`. | |
| `spark.hadoop.fs.comet.s3Compliant.schemes` | Defines filesystem schemes (e.g., blob, minio, r2) that Comet treats as S3-compliant, separated by commas. Such schemes reuse the `fs.s3a.*` credential surface and accept vendor-style `fs.<scheme>.<authority>.*` keys. Empty by default, so no alias scheme is claimed unless opted in. Read from the Hadoop configuration, so it must be set before the SparkSession is created. | |
<!-- prettier-ignore-end -->
<!--END:CONFIG_TABLE-->
diff --git a/searchindex.js b/searchindex.js
index 31938acdce..4bda261d07 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"!": [[60, "id1"]], "%": [[58, "id1"]], "&": [[48, "id1"]], "*": [[58, "id2"]], "+": [[58, "id3"]], "-": [[58, "id4"]], "/": [[58, "id5"]], "1. Format Your Code": [[45, "format-your-code"]], "1. Install Comet": [[67, "install-comet"], [79, "install-comet"]], "1. Native Operators (nativeExecs map)": [[31, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[45, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[67, "clone-iceberg-and-apply- [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"!": [[60, "id1"]], "%": [[58, "id1"]], "&": [[48, "id1"]], "*": [[58, "id2"]], "+": [[58, "id3"]], "-": [[58, "id4"]], "/": [[58, "id5"]], "1. Format Your Code": [[45, "format-your-code"]], "1. Install Comet": [[67, "install-comet"], [79, "install-comet"]], "1. Native Operators (nativeExecs map)": [[31, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[45, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[67, "clone-iceberg-and-apply- [...]
\ No newline at end of file
diff --git a/user-guide/latest/configs.html b/user-guide/latest/configs.html
index d789f35510..3ee479baae 100644
--- a/user-guide/latest/configs.html
+++ b/user-guide/latest/configs.html
@@ -884,7 +884,7 @@ under the License.
<td><p>true</p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">spark.hadoop.fs.comet.libhdfs.schemes</span></code></p></td>
-<td><p>Defines filesystem schemes (e.g., hdfs, webhdfs) that the native side accesses via libhdfs, separated by commas. Valid only when built with hdfs-opendal feature enabled.</p></td>
+<td><p>Defines filesystem schemes (e.g., hdfs, webhdfs) that the native side accesses via libhdfs, separated by commas. Valid only when built with hdfs-opendal feature enabled. Read from the Hadoop configuration, so it must be set before the SparkSession is created. A scheme listed only through a reader option is not read natively. libhdfs reads the settings of a listed scheme, such as <code class="docutils literal notranslate"><span class="pre">fs.<scheme>.impl</span></code>, from [...]
<td><p></p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">spark.hadoop.fs.comet.s3Compliant.schemes</span></code></p></td>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org