You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2025/08/17 15:18:26 UTC

[I] `array_contains` causes panic if default scan is `auto` [datafusion-comet]

andygrove opened a new issue, #2173:
URL: https://github.com/apache/datafusion-comet/issues/2173

   ### Describe the bug
   
   While working on https://github.com/apache/datafusion-comet/issues/2172, I discovered a potential issue:
   
   ```
   - array_contains - test all types (native Parquet reader) *** FAILED *** (4 seconds, 152 milliseconds)
     org.apache.spark.SparkException: Job aborted due to stage failure: Task 3 in stage 387.0 failed 1 times, most recent failure: Lost task 3.0 in stage 387.0 (TID 1306) (woody.lan executor driver): org.apache.comet.CometNativeException: assertion `left == right` failed
     left: 2
    right: 1
           at comet::errors::init::{{closure}}(/home/andy/git/apache/datafusion-comet/native/core/src/errors.rs:151)
           at <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call(/rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/alloc/src/boxed.rs:1980)
           at std::panicking::rust_panic_with_hook(/rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/std/src/panicking.rs:841)
           at std::panicking::begin_panic_handler::{{closure}}(/rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/std/src/panicking.rs:706)
           at std::sys::backtrace::__rust_end_short_backtrace(/rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/std/src/sys/backtrace.rs:168)
           at __rustc::rust_begin_unwind(/rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/std/src/panicking.rs:697)
           at core::panicking::panic_fmt(/rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/core/src/panicking.rs:75)
           at core::panicking::assert_failed_inner(/rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/core/src/panicking.rs:432)
           at core::panicking::assert_failed(/rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/core/src/panicking.rs:387)
           at <&arrow_buffer::buffer::boolean::BooleanBuffer as core::ops::bit::BitAnd>::bitand(/home/andy/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-buffer-55.2.0/src/buffer/boolean.rs:233)
           at arrow_buffer::buffer::null::NullBuffer::union(/home/andy/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-buffer-55.2.0/src/buffer/null.rs:80)
           at datafusion_physical_expr_common::datum::compare_op_for_nested(/home/andy/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-expr-common-49.0.0/src/datum.rs:159)
           at datafusion_physical_expr_common::datum::compare_with_eq(/home/andy/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-expr-common-49.0.0/src/datum.rs:98)
           at datafusion_functions_nested::array_has::array_has_dispatch_for_array(/home/andy/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-functions-nested-49.0.0/src/array_has.rs:319)
           at datafusion_functions_nested::array_has::array_has_inner_for_array(/home/andy/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-functions-nested-49.0.0/src/array_has.rs:225)
           at <datafusion_functions_nested::array_has::ArrayHas as datafusion_expr::udf::ScalarUDFImpl>::invoke_with_args(/home/andy/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-functions-nested-49.0.0/src/array_has.rs:180)
           at datafusion_expr::udf::ScalarUDF::invoke_with_args(/home/andy/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-expr-49.0.0/src/udf.rs:209)
           at <datafusion_physical_expr::scalar_function::ScalarFunctionExpr as datafusion_physical_expr_common::physical_expr::PhysicalExpr>::evaluate(/home/andy/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-expr-49.0.0/src/scalar_function.rs:200)
           at datafusion_physical_plan::projection::ProjectionStream::batch_project::{{closure}}(/home/andy/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-plan-49.0.0/src/projection.rs:320)
   ```
   
   ### Steps to reproduce
   
   _No response_
   
   ### Expected behavior
   
   _No response_
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] `array_contains` test causes panic if default scan is `auto` [datafusion-comet]

Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead closed issue #2173: `array_contains` test causes panic if default scan is `auto`
URL: https://github.com/apache/datafusion-comet/issues/2173


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] `array_contains` test causes panic if default scan is `auto` [datafusion-comet]

Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #2173:
URL: https://github.com/apache/datafusion-comet/issues/2173#issuecomment-3206956921

   This looks like a Datafusion bug, I was able to reproduce it in DF and filed https://github.com/apache/datafusion/issues/17263 with details


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] `array_contains` test causes panic if default scan is `auto` [datafusion-comet]

Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #2173:
URL: https://github.com/apache/datafusion-comet/issues/2173#issuecomment-3222058168

   Closed through #2223 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] `array_contains` test causes panic if default scan is `auto` [datafusion-comet]

Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #2173:
URL: https://github.com/apache/datafusion-comet/issues/2173#issuecomment-3198153495

   I was able to localize tests 
   
   ```
     test("native reader - array_contains") {
       withSQLConf(
         CometConf.COMET_EXEC_ENABLED.key -> "true",
         SQLConf.USE_V1_SOURCE_LIST.key -> "parquet",
         CometConf.COMET_ENABLED.key -> "true",
         CometConf.COMET_EXPLAIN_FALLBACK_ENABLED.key -> "true",
         CometConf.COMET_NATIVE_SCAN_IMPL.key -> "native_iceberg_compat") {
         testSingleLineQuery(
           """
             |select array(array(false, true), array(false, true)) a, array(false, true) b union all select array(null, null), null
             |""".stripMargin,
           "select array_contains(a, b) from tbl")
       }
     }
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org