[I] data type Binary not supported in shuffle write [arrow-datafusion-comet]

["advancedxy (via GitHub)" <gi...@apache.org>]

advancedxy opened a new issue, #105:
URL: https://github.com/apache/arrow-datafusion-comet/issues/105

   ### Describe the bug
   
   Found the issue when `CometCollectExec` is added during #100.
   
   The error msg is: `data type Binary not supported in shuffle write`
   
   Reproduced in the local environment, the whole stack trace is :
   ```
   org.apache.comet.CometNativeException: not implemented: data type Binary not supported in shuffle write
           at std::backtrace_rs::backtrace::libunwind::trace(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93)
           at std::backtrace_rs::backtrace::trace_unsynchronized(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/std/src/../../backtrace/src/backtrace/mod.rs:66)
           at std::backtrace::Backtrace::create(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/std/src/backtrace.rs:331)
           at comet::errors::init::{{closure}}(./core/src/errors.rs:134)
           at  as core::ops::function::Fn>::call(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/alloc/src/boxed.rs:2021)
           at std::panicking::rust_panic_with_hook(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/std/src/panicking.rs:757)
           at std::panicking::begin_panic_handler::{{closure}}(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/std/src/panicking.rs:631)
           at std::sys_common::backtrace::__rust_end_short_backtrace(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/std/src/sys_common/backtrace.rs:170)
           at rust_begin_unwind(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/std/src/panicking.rs:619)
           at core::panicking::panic_fmt(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/core/src/panicking.rs:72)
           at comet::execution::datafusion::shuffle_writer::slot_size(./core/src/execution/datafusion/shuffle_writer.rs:319)
           at comet::execution::datafusion::shuffle_writer::PartitionBuffer::init_active_if_necessary::{{closure}}(./core/src/execution/datafusion/shuffle_writer.rs:225)
           at core::iter::adapters::map::map_fold::{{closure}}(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/core/src/iter/adapters/map.rs:84)
           at core::iter::traits::iterator::Iterator::fold(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/core/src/iter/traits/iterator.rs:2639)
           at  as core::iter::traits::iterator::Iterator>::fold(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/core/src/iter/adapters/map.rs:124)
           at ::sum(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/core/src/iter/traits/accum.rs:50)
           at core::iter::traits::iterator::Iterator::sum(/rustc/ec08a0337f3556212525dbf1d3b41e19bdf27621/library/core/src/iter/traits/iterator.rs:3634)
           at comet::execution::datafusion::shuffle_writer::PartitionBuffer::init_active_if_necessary(./core/src/execution/datafusion/shuffle_writer.rs:221)
           at comet::execution::datafusion::shuffle_writer::PartitionBuffer::append_rows(./core/src/execution/datafusion/shuffle_writer.rs:246)
           at comet::execution::datafusion::shuffle_writer::PartitionBuffer::append_batch(./core/src/execution/datafusion/shuffle_writer.rs:237)
           at comet::execution::datafusion::shuffle_writer::ShuffleRepartitioner::insert_batch::{{closure}}(./core/src/execution/datafusion/shuffle_writer.rs:730)
           at comet::execution::datafusion::shuffle_writer::external_shuffle::{{closure}}(./core/src/execution/datafusion/shuffle_writer.rs:930)
   ```
   
   It's clear that `slot_size` and `append_columns` doesn't include binary support.
   
   ### Steps to reproduce
   
   _No response_
   
   ### Expected behavior
   
   Binary data could be wrote by shuffle writer.
   
   ### Additional context
   
   I can submit a PR for this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] data type Binary not supported in shuffle write [arrow-datafusion-comet]

["sunchao (via GitHub)" <gi...@apache.org>]

sunchao closed issue #105: data type Binary not supported in shuffle write
URL: https://github.com/apache/arrow-datafusion-comet/issues/105


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org