(datafusion-comet) branch main updated: chore: deprecate native_comet scan in favor of native_iceberg_compat (#2949)

[ag...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

agrove pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git


The following commit(s) were added to refs/heads/main by this push:
     new 1009e9854 chore: deprecate native_comet scan in favor of native_iceberg_compat (#2949)
1009e9854 is described below

commit 1009e98547470fb52e30f731eb9825aa1f914bf3
Author: Shekhar Prasad Rajak <57...@users.noreply.github.com>
AuthorDate: Sat Jan 17 03:31:06 2026 +0530

    chore: deprecate native_comet scan in favor of native_iceberg_compat (#2949)
---
 common/src/main/scala/org/apache/comet/CometConf.scala | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/common/src/main/scala/org/apache/comet/CometConf.scala b/common/src/main/scala/org/apache/comet/CometConf.scala
index cccad53c5..89dbb6468 100644
--- a/common/src/main/scala/org/apache/comet/CometConf.scala
+++ b/common/src/main/scala/org/apache/comet/CometConf.scala
@@ -111,6 +111,10 @@ object CometConf extends ShimCometConf {
       .booleanConf
       .createWithDefault(false)
 
+  // Deprecated: native_comet uses mutable buffers incompatible with Arrow FFI best practices
+  // and does not support complex types. Use native_iceberg_compat or auto instead.
+  // See: https://github.com/apache/datafusion-comet/issues/2186
+  @deprecated("Use SCAN_AUTO instead", "0.9.0")
   val SCAN_NATIVE_COMET = "native_comet"
   val SCAN_NATIVE_DATAFUSION = "native_datafusion"
   val SCAN_NATIVE_ICEBERG_COMPAT = "native_iceberg_compat"
@@ -121,11 +125,14 @@ object CometConf extends ShimCometConf {
     .doc(
       s"The implementation of Comet Native Scan to use. Available modes are `$SCAN_NATIVE_COMET`," +
         s"`$SCAN_NATIVE_DATAFUSION`, and `$SCAN_NATIVE_ICEBERG_COMPAT`. " +
-        s"`$SCAN_NATIVE_COMET` is for the original Comet native scan which uses a jvm based " +
-        "parquet file reader and native column decoding. Supports simple types only " +
-        s"`$SCAN_NATIVE_DATAFUSION` is a fully native implementation of scan based on DataFusion" +
-        s"`$SCAN_NATIVE_ICEBERG_COMPAT` is a native implementation that exposes apis to read " +
-        s"parquet columns natively. `$SCAN_AUTO` chooses the best scan.")
+        s"`$SCAN_NATIVE_COMET` (DEPRECATED) is for the original Comet native scan which " +
+        "uses a jvm based parquet file reader and native column decoding. " +
+        "Supports simple types only. " +
+        s"`$SCAN_NATIVE_DATAFUSION` is a fully native implementation of scan based on " +
+        "DataFusion. " +
+        s"`$SCAN_NATIVE_ICEBERG_COMPAT` is the recommended native implementation that " +
+        "exposes apis to read parquet columns natively and supports complex types. " +
+        s"`$SCAN_AUTO` (default) chooses the best scan.")
     .internal()
     .stringConf
     .transform(_.toLowerCase(Locale.ROOT))


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org