[jira] [Updated] (ARROW-16503) [C++] Can't concatenate extension arrays

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-16503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joris Van den Bossche updated ARROW-16503:
------------------------------------------
    Fix Version/s: 11.0.0

> [C++] Can't concatenate extension arrays
> ----------------------------------------
>
>                 Key: ARROW-16503
>                 URL: https://issues.apache.org/jira/browse/ARROW-16503
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: C++
>            Reporter: Dewey Dunnington
>            Assignee: Joris Van den Bossche
>            Priority: Major
>              Labels: good-second-issue
>             Fix For: 11.0.0
>
>
> It looks like Arrays with an extension type can't be concatenated. From the R bindings:
> {code:R}
> library(arrow, warn.conflicts = FALSE)
> arr <- vctrs_extension_array(1:10)
> concat_arrays(arr, arr)
> #> Error: NotImplemented: concatenation of integer(0)
> #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/array/concatenate.cc:195  VisitTypeInline(*out_->type, this)
> #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/array/concatenate.cc:590  ConcatenateImpl(data, pool).Concatenate(&out_data)
> {code}
> This shows up more practically when using the query engine:
> {code:R}
> library(arrow, warn.conflicts = FALSE)
> table <- arrow_table(
>   group = rep(c("a", "b"), 5),
>   col1 = 1:10,
>   col2 = vctrs_extension_array(1:10)
> )
> tf <- tempfile()
> table |> dplyr::group_by(group) |> write_dataset(tf)
> open_dataset(tf) |>
>   dplyr::arrange(col1) |> 
>   dplyr::collect()
> #> Error in `dplyr::collect()`:
> #> ! NotImplemented: concatenation of extension<arrow.r.vctrs>
> #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/array/concatenate.cc:195  VisitTypeInline(*out_->type, this)
> #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/array/concatenate.cc:590  ConcatenateImpl(data, pool).Concatenate(&out_data)
> #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/compute/kernels/vector_selection.cc:2025  Concatenate(values.chunks(), ctx->memory_pool())
> #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/compute/kernels/vector_selection.cc:2084  TakeCA(*table.column(j), indices, options, ctx)
> #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/compute/exec/sink_node.cc:527  impl_->DoFinish()
> #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/compute/exec/exec_plan.cc:467  iterator_.Next()
> #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/record_batch.cc:337  ReadNext(&batch)
> #> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/record_batch.cc:351  ToRecordBatches()
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)