Re: Gandiva and Threads

[Jacques Nadeau <ja...@apache.org>]

@ravindra@dremio.com <ra...@dremio.com> @praveen@dremio.com
<pr...@dremio.com> thoughts?

On Tue, Jul 28, 2020 at 3:39 PM Wes McKinney <we...@gmail.com> wrote:

> Perhaps Gandiva does not handle sliced arrays properly? This would be
> worth investigating
>
> On Mon, Jul 27, 2020 at 7:43 PM Matt Youill <ma...@burnian.com>
> wrote:
> >
> > Managed to track down the issue (sort of).
> >
> > I removed a call to set_chunksize on TableBatchReader where the chunk
> > size was less than the number of rows in a table being read. Runs fine
> > now (tested with 100s threads over mils of rows).
> >
> > Strangely, Gandiva fails if I don't call set_chunksize for a subsequent
> > filter evaluation. It's very strange.
> >
> >
> > On 28/7/20 7:58 am, Wes McKinney wrote:
> > > Crashing when running from multiple threads doesn't sound right,
> > > perhaps there are some missing synchronizations in internal data
> > > structures. Could you open a JIRA issue and show the backtraces of any
> > > crashes or other clues about how to reproduce the issues?
> > >
> > > On Sun, Jul 26, 2020 at 8:12 PM Matt Youill <ma...@burnian.com>
> wrote:
> > >> Hi,
> > >>
> > >> Are there any guidelines for using the Gandiva C++ lib from multiple
> > >> threads? As I increase the number of threads in an application Gandiva
> > >> begins to raise memory faults (e.g. double free). Sometimes it is on
> > >> projector creation, sometimes evaluation, seems to happen in multiple
> > >> unpredictable places inside Gandiva. Can it be used from multiple
> > >> threads, and if so which parts need to be guarded, if any?
> > >>
> > >> Thanks, Matt
> > >>
>