You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by PRAVEEN DEVERACHETTY <pr...@gmail.com> on 2019/03/11 18:09:36 UTC
Update planner/configuration keys
I am using apache drill on windows platform. My requirement is to udpate
the following parameters during apache drill startup. These parameter
vaules may differ in each apache drill nodes. can you share me an example
of how to update these in windows. I know another option is to run on webUI
or postman? i dont want to do it post installation of apache drill. i want
these changes to be reflected during apache drill startup.
*planner.cpu_load_average - 0.7*
*exec.queue.enable - true*
*exec.queue.large - 2*
*exec.queue.memory_ratio - 10.0*
*exec.queue.memory_reserve_ratio - 0.2*
*exec.queue.small - 4*
Thanks,
praveen
Re: Update planner/configuration keys
Posted by Paul Rogers <pa...@yahoo.com.INVALID>.
Hi Kunal and Praveen,
ALTER SYSTEM sets an option value persistently in Zookeeper, which makes the value permanent within that one cluster.
Sounds like you want to persist values in an embedded Drillbit. In this case, there is no Zookeeper. System options get written to the file system, but I don't know if they are persistent. Any embedded Drill users know how this works?
One solution is to start Drill as a server, even if it runs on only one node. I don't know, however, if the Drill server mode is supported on Windows. Any Windows users know this?
Finally, Kunal is correct. Since about a 18 months ago, the value for system/session options are defined in Drill's config system. See [1].
Although Drill does not encourage this usage, you can customize these values in your drill-override.conf as Kunal suggests. You can try this as a workaround for the fact that you are 1) using Embedded Drill, 2) on Windows.
But, Drill is really a distributed tool, so you'd really want to run Drill on Linux as a server, then allow the options to persist in Zookeeper.
Thanks,
- Paul
[1] https://github.com/apache/drill/blob/master/exec/java-exec/src/main/resources/drill-module.conf#L454
On Tuesday, March 12, 2019, 11:31:59 AM PDT, Kunal Khatua <ku...@apache.org> wrote:
That is correct... you'll need to run these on the command prompt for the
very first time. Like profiles, I believe, the ALTER SYSTEM commands also
have a permanent effect.
I'm not sure if there is a way for a user to provide a pre-configured set
of parameters to use during startup. You could see if setting these values
in DRILL_HOME/conf/drill-override.conf helps. A lot of the values you want
are originally set as defaults from an embedded file "drill-module.conf" ,
and drill-override.conf , well... overrides some of those values.
If this doesn't work, you can file an improvement/newFeature JIRA for this,
considering this is a nice feature to have.
~ Kunal
On Tue, Mar 12, 2019 at 3:18 AM PRAVEEN DEVERACHETTY <pr...@gmail.com>
wrote:
> Hi Kunal, But where should i keep those statements? is there any drill
> startup script that i can run these alter staements? i think these scripts
> only on the drill command prompt right?
>
> Thanks,
> Praveen
>
> On Tue, Mar 12, 2019 at 1:02 PM Kunal Khatua <ku...@apache.org> wrote:
>
> > Executing an "alter system set param=value" usually persists the value.
> > Not sure if that works for an embedded mode.
> > Could you try and let us know if that works?
> > On 3/11/2019 11:19:13 AM, PRAVEEN DEVERACHETTY <pr...@gmail.com>
> > wrote:
> > I am using apache drill on windows platform. My requirement is to udpate
> > the following parameters during apache drill startup. These parameter
> > vaules may differ in each apache drill nodes. can you share me an example
> > of how to update these in windows. I know another option is to run on
> webUI
> > or postman? i dont want to do it post installation of apache drill. i
> want
> > these changes to be reflected during apache drill startup.
> >
> > *planner.cpu_load_average - 0.7*
> > *exec.queue.enable - true*
> > *exec.queue.large - 2*
> > *exec.queue.memory_ratio - 10.0*
> > *exec.queue.memory_reserve_ratio - 0.2*
> > *exec.queue.small - 4*
> >
> > Thanks,
> > praveen
> >
>
Re: Update planner/configuration keys
Posted by Kunal Khatua <ku...@apache.org>.
That is correct... you'll need to run these on the command prompt for the
very first time. Like profiles, I believe, the ALTER SYSTEM commands also
have a permanent effect.
I'm not sure if there is a way for a user to provide a pre-configured set
of parameters to use during startup. You could see if setting these values
in DRILL_HOME/conf/drill-override.conf helps. A lot of the values you want
are originally set as defaults from an embedded file "drill-module.conf" ,
and drill-override.conf , well... overrides some of those values.
If this doesn't work, you can file an improvement/newFeature JIRA for this,
considering this is a nice feature to have.
~ Kunal
On Tue, Mar 12, 2019 at 3:18 AM PRAVEEN DEVERACHETTY <pr...@gmail.com>
wrote:
> Hi Kunal, But where should i keep those statements? is there any drill
> startup script that i can run these alter staements? i think these scripts
> only on the drill command prompt right?
>
> Thanks,
> Praveen
>
> On Tue, Mar 12, 2019 at 1:02 PM Kunal Khatua <ku...@apache.org> wrote:
>
> > Executing an "alter system set param=value" usually persists the value.
> > Not sure if that works for an embedded mode.
> > Could you try and let us know if that works?
> > On 3/11/2019 11:19:13 AM, PRAVEEN DEVERACHETTY <pr...@gmail.com>
> > wrote:
> > I am using apache drill on windows platform. My requirement is to udpate
> > the following parameters during apache drill startup. These parameter
> > vaules may differ in each apache drill nodes. can you share me an example
> > of how to update these in windows. I know another option is to run on
> webUI
> > or postman? i dont want to do it post installation of apache drill. i
> want
> > these changes to be reflected during apache drill startup.
> >
> > *planner.cpu_load_average - 0.7*
> > *exec.queue.enable - true*
> > *exec.queue.large - 2*
> > *exec.queue.memory_ratio - 10.0*
> > *exec.queue.memory_reserve_ratio - 0.2*
> > *exec.queue.small - 4*
> >
> > Thanks,
> > praveen
> >
>
Re: Update planner/configuration keys
Posted by PRAVEEN DEVERACHETTY <pr...@gmail.com>.
Hi Kunal, But where should i keep those statements? is there any drill
startup script that i can run these alter staements? i think these scripts
only on the drill command prompt right?
Thanks,
Praveen
On Tue, Mar 12, 2019 at 1:02 PM Kunal Khatua <ku...@apache.org> wrote:
> Executing an "alter system set param=value" usually persists the value.
> Not sure if that works for an embedded mode.
> Could you try and let us know if that works?
> On 3/11/2019 11:19:13 AM, PRAVEEN DEVERACHETTY <pr...@gmail.com>
> wrote:
> I am using apache drill on windows platform. My requirement is to udpate
> the following parameters during apache drill startup. These parameter
> vaules may differ in each apache drill nodes. can you share me an example
> of how to update these in windows. I know another option is to run on webUI
> or postman? i dont want to do it post installation of apache drill. i want
> these changes to be reflected during apache drill startup.
>
> *planner.cpu_load_average - 0.7*
> *exec.queue.enable - true*
> *exec.queue.large - 2*
> *exec.queue.memory_ratio - 10.0*
> *exec.queue.memory_reserve_ratio - 0.2*
> *exec.queue.small - 4*
>
> Thanks,
> praveen
>
Re: Update planner/configuration keys
Posted by Kunal Khatua <ku...@apache.org>.
Executing an "alter system set param=value" usually persists the value. Not sure if that works for an embedded mode.
Could you try and let us know if that works?
On 3/11/2019 11:19:13 AM, PRAVEEN DEVERACHETTY <pr...@gmail.com> wrote:
I am using apache drill on windows platform. My requirement is to udpate
the following parameters during apache drill startup. These parameter
vaules may differ in each apache drill nodes. can you share me an example
of how to update these in windows. I know another option is to run on webUI
or postman? i dont want to do it post installation of apache drill. i want
these changes to be reflected during apache drill startup.
*planner.cpu_load_average - 0.7*
*exec.queue.enable - true*
*exec.queue.large - 2*
*exec.queue.memory_ratio - 10.0*
*exec.queue.memory_reserve_ratio - 0.2*
*exec.queue.small - 4*
Thanks,
praveen