You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gluten.apache.org by Chang Chen <ba...@gmail.com> on 2024/10/28 03:54:27 UTC

Update in October Clickhouse backend.

Hi Guys，

Below is highlight for Octorber updates for clickhouse backend.
Arm

We are working with Huawei to prepare a build for ARM. Please see
https://github.com/apache/incubator-gluten/issues/7654 for details.
Performance

   1.

   [GLUTEN-7394][CH] Reduce the times of the calling listFiles when
   executing query from the parquet file format
   <https://github.com/apache/incubator-gluten/pull/7417>
   2.

   [GLUTEN-7482][CH] Remove redundant head object operation of s3
   <https://github.com/apache/incubator-gluten/pull/7565>

New Feature

   1.

   [GLUTEN-6856][CH]Support arrays_overlap and fix array_join diff
   <https://github.com/apache/incubator-gluten/pull/6857>
   2.

   [GLUTEN-7311][CH] Support grace aggregate algorithm in partial
   aggregating stages <https://github.com/apache/incubator-gluten/pull/7322>
   3.

   [GLUTEN-7455][CH] Add spark_modulo for compatibility
   <https://github.com/apache/incubator-gluten/pull/7619>
   4.

   Support Spark 353 <https://github.com/apache/incubator-gluten/pull/7537>
   5.

   [GLUTEN-7659][CH] Implement splittable bzip2 decompression
   <https://github.com/apache/incubator-gluten/pull/7638>

BUG Fix

   1.

   [CH] get_json_object fails when the json path contains array slice
   <https://github.com/apache/incubator-gluten/issues/7432>
   2.

   [GLUTEN-7389] [CH] fix cast map to string diff with spark
   <https://github.com/apache/incubator-gluten/pull/7393>
   3.

   [GLUTEN-7550][CH] Rewrite get_json_object in singular_or_list
   <https://github.com/apache/incubator-gluten/pull/7551>
   4.

   [GLUTEN-7542][CH] Fix cache not refresh
   <https://github.com/apache/incubator-gluten/pull/7547>
   5.

   [GLUTEN-7596][CH] Fix bnlj empty join error
   <https://github.com/apache/incubator-gluten/pull/7597>
   6.

   [GLUTEN-7545][CH] Fix regexp_replace group catching syntax diff
   <https://github.com/apache/incubator-gluten/pull/7603>
   7.

   [GLUTEN-7623][CH] Fix running cache command with error when executor add
   and removed. <https://github.com/apache/incubator-gluten/pull/7625>
   8.

   [GLUTEN-7673][CH] Fix substrait infinite loop
   <https://github.com/apache/incubator-gluten/pull/7674>

Re: Update in October Clickhouse backend.

Posted by zhang listar <zh...@gmail.com>.

Great! Looking forward to it!

felix f <bi...@gmail.com> 于2024年10月29日周二 06:01写道：

> Good summary, Thank you, Chang.
>
> On Sun, Oct 27, 2024 at 8:55 PM Chang Chen <ba...@gmail.com> wrote:
>
> > Hi Guys，
> >
> > Below is highlight for Octorber updates for clickhouse backend.
> > Arm
> >
> > We are working with Huawei to prepare a build for ARM. Please see
> > https://github.com/apache/incubator-gluten/issues/7654 for details.
> > Performance
> >
> >    1.
> >
> >    [GLUTEN-7394][CH] Reduce the times of the calling listFiles when
> >    executing query from the parquet file format
> >    <https://github.com/apache/incubator-gluten/pull/7417>
> >    2.
> >
> >    [GLUTEN-7482][CH] Remove redundant head object operation of s3
> >    <https://github.com/apache/incubator-gluten/pull/7565>
> >
> > New Feature
> >
> >    1.
> >
> >    [GLUTEN-6856][CH]Support arrays_overlap and fix array_join diff
> >    <https://github.com/apache/incubator-gluten/pull/6857>
> >    2.
> >
> >    [GLUTEN-7311][CH] Support grace aggregate algorithm in partial
> >    aggregating stages <
> > https://github.com/apache/incubator-gluten/pull/7322>
> >    3.
> >
> >    [GLUTEN-7455][CH] Add spark_modulo for compatibility
> >    <https://github.com/apache/incubator-gluten/pull/7619>
> >    4.
> >
> >    Support Spark 353 <
> https://github.com/apache/incubator-gluten/pull/7537
> > >
> >    5.
> >
> >    [GLUTEN-7659][CH] Implement splittable bzip2 decompression
> >    <https://github.com/apache/incubator-gluten/pull/7638>
> >
> > BUG Fix
> >
> >    1.
> >
> >    [CH] get_json_object fails when the json path contains array slice
> >    <https://github.com/apache/incubator-gluten/issues/7432>
> >    2.
> >
> >    [GLUTEN-7389] [CH] fix cast map to string diff with spark
> >    <https://github.com/apache/incubator-gluten/pull/7393>
> >    3.
> >
> >    [GLUTEN-7550][CH] Rewrite get_json_object in singular_or_list
> >    <https://github.com/apache/incubator-gluten/pull/7551>
> >    4.
> >
> >    [GLUTEN-7542][CH] Fix cache not refresh
> >    <https://github.com/apache/incubator-gluten/pull/7547>
> >    5.
> >
> >    [GLUTEN-7596][CH] Fix bnlj empty join error
> >    <https://github.com/apache/incubator-gluten/pull/7597>
> >    6.
> >
> >    [GLUTEN-7545][CH] Fix regexp_replace group catching syntax diff
> >    <https://github.com/apache/incubator-gluten/pull/7603>
> >    7.
> >
> >    [GLUTEN-7623][CH] Fix running cache command with error when executor
> add
> >    and removed. <https://github.com/apache/incubator-gluten/pull/7625>
> >    8.
> >
> >    [GLUTEN-7673][CH] Fix substrait infinite loop
> >    <https://github.com/apache/incubator-gluten/pull/7674>
> >
>

Re: Update in October Clickhouse backend.

Posted by felix f <bi...@gmail.com>.

Good summary, Thank you, Chang.

On Sun, Oct 27, 2024 at 8:55 PM Chang Chen <ba...@gmail.com> wrote:

> Hi Guys，
>
> Below is highlight for Octorber updates for clickhouse backend.
> Arm
>
> We are working with Huawei to prepare a build for ARM. Please see
> https://github.com/apache/incubator-gluten/issues/7654 for details.
> Performance
>
>    1.
>
>    [GLUTEN-7394][CH] Reduce the times of the calling listFiles when
>    executing query from the parquet file format
>    <https://github.com/apache/incubator-gluten/pull/7417>
>    2.
>
>    [GLUTEN-7482][CH] Remove redundant head object operation of s3
>    <https://github.com/apache/incubator-gluten/pull/7565>
>
> New Feature
>
>    1.
>
>    [GLUTEN-6856][CH]Support arrays_overlap and fix array_join diff
>    <https://github.com/apache/incubator-gluten/pull/6857>
>    2.
>
>    [GLUTEN-7311][CH] Support grace aggregate algorithm in partial
>    aggregating stages <
> https://github.com/apache/incubator-gluten/pull/7322>
>    3.
>
>    [GLUTEN-7455][CH] Add spark_modulo for compatibility
>    <https://github.com/apache/incubator-gluten/pull/7619>
>    4.
>
>    Support Spark 353 <https://github.com/apache/incubator-gluten/pull/7537
> >
>    5.
>
>    [GLUTEN-7659][CH] Implement splittable bzip2 decompression
>    <https://github.com/apache/incubator-gluten/pull/7638>
>
> BUG Fix
>
>    1.
>
>    [CH] get_json_object fails when the json path contains array slice
>    <https://github.com/apache/incubator-gluten/issues/7432>
>    2.
>
>    [GLUTEN-7389] [CH] fix cast map to string diff with spark
>    <https://github.com/apache/incubator-gluten/pull/7393>
>    3.
>
>    [GLUTEN-7550][CH] Rewrite get_json_object in singular_or_list
>    <https://github.com/apache/incubator-gluten/pull/7551>
>    4.
>
>    [GLUTEN-7542][CH] Fix cache not refresh
>    <https://github.com/apache/incubator-gluten/pull/7547>
>    5.
>
>    [GLUTEN-7596][CH] Fix bnlj empty join error
>    <https://github.com/apache/incubator-gluten/pull/7597>
>    6.
>
>    [GLUTEN-7545][CH] Fix regexp_replace group catching syntax diff
>    <https://github.com/apache/incubator-gluten/pull/7603>
>    7.
>
>    [GLUTEN-7623][CH] Fix running cache command with error when executor add
>    and removed. <https://github.com/apache/incubator-gluten/pull/7625>
>    8.
>
>    [GLUTEN-7673][CH] Fix substrait infinite loop
>    <https://github.com/apache/incubator-gluten/pull/7674>
>