You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Vitalii Diravka <vi...@apache.org> on 2021/01/18 21:22:23 UTC
Re: Changing the scope of output_batch_size option
Hi Artem,
You can set both options. But it will be stored in the Persistence Store,
so the option will be preserved between Drill sessions.
Please check all info about difference system and session settings levels:
https://drill.apache.org/docs/alter-system/
Also please use user mailing list (user@drill.apache.org) for this type of
questions:
https://drill.apache.org/mailinglists/
Kind regards
Vitalii
On Wed, Nov 4, 2020 at 3:14 PM Artem Trush <ve...@gmail.com> wrote:
> Hi Drill Devs!
> I have a question about the "output_batch_size" option. Is it safe to
> change the scope of this parameter from system to both system and
> session level? If not, what can happen?
> The problem arose due to a performance issue in certain queries and
> changing this option helps. But due to the fact that it is setted at
> the system level, all executable queries began to absorb more memory
> and as a result - out of memory issue. So, the solution to the problem
> is to allow users to change this parameter at the session level to
> affect only certain queries, without affecting all others at the
> system.
>
Re: Changing the scope of output_batch_size option
Posted by Abhishek Girish <ag...@apache.org>.
To give more context, we had a customer request for being able to set this
property at session level. As having this option only at the system level
was too broad for them. Before making a change, I had requested Artem to
gather feedback from the community.
The change Artem made was to enable changing this at both Session and
System levels (not abandoning it).
It's tracked here: https://issues.apache.org/jira/browse/DRILL-7801,
https://github.com/apache/drill/pull/2114
Hope it's clear.
On Wed, Jan 20, 2021 at 6:46 AM Vitalii Diravka <vi...@apache.org> wrote:
> Oh, I see. This option is available only on system level. I don't see any
> reason to abandon it for users (not admins) and session level.
>
> @Artem Do you plan to do the change? Have you created the
> correspondent ticket?
>
> Kind regards
> Vitalii
>
>
> On Wed, Jan 20, 2021 at 8:36 AM Abhishek Girish <ag...@apache.org>
> wrote:
>
> > Hey Vitalii,
> >
> > I had requested Artem to use the Dev list as it relates to making changes
> > to the Drill code. He is a Developer and is planning on making / has made
> > changes to support updates at session level.
> >
> > Regards,
> > Abhishek
> >
> > On Mon, Jan 18, 2021 at 1:23 PM Vitalii Diravka <vi...@apache.org>
> > wrote:
> >
> > > Hi Artem,
> > >
> > > You can set both options. But it will be stored in the Persistence
> Store,
> > > so the option will be preserved between Drill sessions.
> > > Please check all info about difference system and session settings
> > levels:
> > > https://drill.apache.org/docs/alter-system/
> > >
> > > Also please use user mailing list (user@drill.apache.org) for this
> type
> > of
> > > questions:
> > > https://drill.apache.org/mailinglists/
> > >
> > >
> > > Kind regards
> > > Vitalii
> > >
> > >
> > > On Wed, Nov 4, 2020 at 3:14 PM Artem Trush <ve...@gmail.com> wrote:
> > >
> > > > Hi Drill Devs!
> > > > I have a question about the "output_batch_size" option. Is it safe to
> > > > change the scope of this parameter from system to both system and
> > > > session level? If not, what can happen?
> > > > The problem arose due to a performance issue in certain queries and
> > > > changing this option helps. But due to the fact that it is setted at
> > > > the system level, all executable queries began to absorb more memory
> > > > and as a result - out of memory issue. So, the solution to the
> problem
> > > > is to allow users to change this parameter at the session level to
> > > > affect only certain queries, without affecting all others at the
> > > > system.
> > > >
> > >
> >
>
Re: Changing the scope of output_batch_size option
Posted by Vitalii Diravka <vi...@apache.org>.
Oh, I see. This option is available only on system level. I don't see any
reason to abandon it for users (not admins) and session level.
@Artem Do you plan to do the change? Have you created the
correspondent ticket?
Kind regards
Vitalii
On Wed, Jan 20, 2021 at 8:36 AM Abhishek Girish <ag...@apache.org> wrote:
> Hey Vitalii,
>
> I had requested Artem to use the Dev list as it relates to making changes
> to the Drill code. He is a Developer and is planning on making / has made
> changes to support updates at session level.
>
> Regards,
> Abhishek
>
> On Mon, Jan 18, 2021 at 1:23 PM Vitalii Diravka <vi...@apache.org>
> wrote:
>
> > Hi Artem,
> >
> > You can set both options. But it will be stored in the Persistence Store,
> > so the option will be preserved between Drill sessions.
> > Please check all info about difference system and session settings
> levels:
> > https://drill.apache.org/docs/alter-system/
> >
> > Also please use user mailing list (user@drill.apache.org) for this type
> of
> > questions:
> > https://drill.apache.org/mailinglists/
> >
> >
> > Kind regards
> > Vitalii
> >
> >
> > On Wed, Nov 4, 2020 at 3:14 PM Artem Trush <ve...@gmail.com> wrote:
> >
> > > Hi Drill Devs!
> > > I have a question about the "output_batch_size" option. Is it safe to
> > > change the scope of this parameter from system to both system and
> > > session level? If not, what can happen?
> > > The problem arose due to a performance issue in certain queries and
> > > changing this option helps. But due to the fact that it is setted at
> > > the system level, all executable queries began to absorb more memory
> > > and as a result - out of memory issue. So, the solution to the problem
> > > is to allow users to change this parameter at the session level to
> > > affect only certain queries, without affecting all others at the
> > > system.
> > >
> >
>
Re: Changing the scope of output_batch_size option
Posted by Abhishek Girish <ag...@apache.org>.
Hey Vitalii,
I had requested Artem to use the Dev list as it relates to making changes
to the Drill code. He is a Developer and is planning on making / has made
changes to support updates at session level.
Regards,
Abhishek
On Mon, Jan 18, 2021 at 1:23 PM Vitalii Diravka <vi...@apache.org> wrote:
> Hi Artem,
>
> You can set both options. But it will be stored in the Persistence Store,
> so the option will be preserved between Drill sessions.
> Please check all info about difference system and session settings levels:
> https://drill.apache.org/docs/alter-system/
>
> Also please use user mailing list (user@drill.apache.org) for this type of
> questions:
> https://drill.apache.org/mailinglists/
>
>
> Kind regards
> Vitalii
>
>
> On Wed, Nov 4, 2020 at 3:14 PM Artem Trush <ve...@gmail.com> wrote:
>
> > Hi Drill Devs!
> > I have a question about the "output_batch_size" option. Is it safe to
> > change the scope of this parameter from system to both system and
> > session level? If not, what can happen?
> > The problem arose due to a performance issue in certain queries and
> > changing this option helps. But due to the fact that it is setted at
> > the system level, all executable queries began to absorb more memory
> > and as a result - out of memory issue. So, the solution to the problem
> > is to allow users to change this parameter at the session level to
> > affect only certain queries, without affecting all others at the
> > system.
> >
>