You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by ag...@apache.org on 2025/09/09 20:09:13 UTC
(datafusion-comet) branch main updated: Update Iceberg docs for 0.10 release (#2355)
This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new 2056ace63 Update Iceberg docs for 0.10 release (#2355)
2056ace63 is described below
commit 2056ace6303c2d8f97b09b59c2ca2538a28edf46
Author: hsiang-c <13...@users.noreply.github.com>
AuthorDate: Tue Sep 9 13:09:08 2025 -0700
Update Iceberg docs for 0.10 release (#2355)
---
docs/source/user-guide/latest/iceberg.md | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/docs/source/user-guide/latest/iceberg.md b/docs/source/user-guide/latest/iceberg.md
index acae6ad27..ed7d49b4a 100644
--- a/docs/source/user-guide/latest/iceberg.md
+++ b/docs/source/user-guide/latest/iceberg.md
@@ -80,7 +80,7 @@ $SPARK_HOME/bin/spark-shell \
--conf spark.sql.catalog.spark_catalog.type=hadoop \
--conf spark.sql.catalog.spark_catalog.warehouse=/tmp/warehouse \
--conf spark.plugins=org.apache.spark.CometPlugin \
- --conf spark.comet.exec.shuffle.enabled=false \
+ --conf spark.shuffle.manager=org.apache.spark.sql.comet.execution.shuffle.CometShuffleManager \
--conf spark.sql.iceberg.parquet.reader-type=COMET \
--conf spark.comet.explainFallback.enabled=true \
--conf spark.memory.offHeap.enabled=true \
@@ -143,8 +143,5 @@ scala> spark.sql(s"SELECT * from t1").explain()
```
## Known issues
- - We temporarily disable Comet when there are delete files in Iceberg scan, see Iceberg [1.8.1 diff](https://github.com/apache/datafusion-comet/blob/main/dev/diffs/iceberg/1.8.1.diff) and this [PR](https://github.com/apache/iceberg/pull/13793)
- - Iceberg scan w/ delete files lead to [runtime exceptions](https://github.com/apache/datafusion-comet/issues/2117) and [incorrect results](https://github.com/apache/datafusion-comet/issues/2118)
- - Enabling `CometShuffleManager` leads to [runtime exceptions](https://github.com/apache/datafusion-comet/issues/2086)
- Spark Runtime Filtering isn't [working](https://github.com/apache/datafusion-comet/issues/2116)
- You can bypass the issue by either setting `spark.sql.adaptive.enabled=false` or `spark.comet.exec.broadcastExchange.enabled=false`
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org