You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Chak-Pong Chung <ch...@gmail.com> on 2022/06/23 19:35:43 UTC
vectorized processing for arrow::take()
I asked a question here about vectorized processing.
https://stackoverflow.com/questions/72735678/how-to-vectorize-arrowcomputetake
Any idea? I am also open to the approaches like Intel MKL, xsimd, clang and
so on.
--
Regards,
Chak-Pong
Re: vectorized processing for arrow::take()
Posted by Chak-Pong Chung <ch...@gmail.com>.
Hi Aldrin,
Use Case: I am taking a subset of a really large input_array. It is used in
places where OpenMP-like and MPI-like parallelism are used. So
vectorization seems to be the next low-hanging fruit.
I have added this to the original stackoverflow post.
On Thu, Jun 23, 2022 at 5:53 PM Aldrin <ak...@ucsc.edu.invalid> wrote:
> Without knowing implementation details of the Take function, I know that
> Arrow uses xsimd and does try to enable the compiler to vectorize code
> where it can. To clarify, are you asking how to improve the performance
> you're seeing, or are you asking how to check if the compiled code is using
> vector instructions? I think a little bit more context about what you know
> and what you're trying to do could also help others who know more about
> this function (and vectorization in Arrow in general) to chime in.
>
> Aldrin Montana
> Computer Science PhD Student
> UC Santa Cruz
>
>
> On Thu, Jun 23, 2022 at 12:41 PM Chak-Pong Chung <ch...@gmail.com>
> wrote:
>
> > correction: not clang, I meant the Vectorizers from LLVM
> >
> > https://llvm.org/docs/Vectorizers.html
> >
> > if we can use it with arrow array
> >
> > On Thu, Jun 23, 2022 at 3:35 PM Chak-Pong Chung <chakpongchung@gmail.com
> >
> > wrote:
> >
> > > I asked a question here about vectorized processing.
> > >
> > >
> > >
> >
> https://stackoverflow.com/questions/72735678/how-to-vectorize-arrowcomputetake
> > >
> > > Any idea? I am also open to the approaches like Intel MKL, xsimd, clang
> > > and so on.
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Chak-Pong
> > >
> >
> >
> > --
> > Regards,
> > Chak-Pong
> >
>
--
Regards,
Chak-Pong
Re: vectorized processing for arrow::take()
Posted by Aldrin <ak...@ucsc.edu.INVALID>.
Without knowing implementation details of the Take function, I know that
Arrow uses xsimd and does try to enable the compiler to vectorize code
where it can. To clarify, are you asking how to improve the performance
you're seeing, or are you asking how to check if the compiled code is using
vector instructions? I think a little bit more context about what you know
and what you're trying to do could also help others who know more about
this function (and vectorization in Arrow in general) to chime in.
Aldrin Montana
Computer Science PhD Student
UC Santa Cruz
On Thu, Jun 23, 2022 at 12:41 PM Chak-Pong Chung <ch...@gmail.com>
wrote:
> correction: not clang, I meant the Vectorizers from LLVM
>
> https://llvm.org/docs/Vectorizers.html
>
> if we can use it with arrow array
>
> On Thu, Jun 23, 2022 at 3:35 PM Chak-Pong Chung <ch...@gmail.com>
> wrote:
>
> > I asked a question here about vectorized processing.
> >
> >
> >
> https://stackoverflow.com/questions/72735678/how-to-vectorize-arrowcomputetake
> >
> > Any idea? I am also open to the approaches like Intel MKL, xsimd, clang
> > and so on.
> >
> >
> >
> > --
> > Regards,
> > Chak-Pong
> >
>
>
> --
> Regards,
> Chak-Pong
>
Re: vectorized processing for arrow::take()
Posted by Chak-Pong Chung <ch...@gmail.com>.
correction: not clang, I meant the Vectorizers from LLVM
https://llvm.org/docs/Vectorizers.html
if we can use it with arrow array
On Thu, Jun 23, 2022 at 3:35 PM Chak-Pong Chung <ch...@gmail.com>
wrote:
> I asked a question here about vectorized processing.
>
>
> https://stackoverflow.com/questions/72735678/how-to-vectorize-arrowcomputetake
>
> Any idea? I am also open to the approaches like Intel MKL, xsimd, clang
> and so on.
>
>
>
> --
> Regards,
> Chak-Pong
>
--
Regards,
Chak-Pong