Controlling the Lifecycle of Drill using Supervisord

[Charles Nnamdi Akalugwu <cp...@gmail.com>]

Hi guys,

I would like to manage the lifecycle of my drillbits using supervisord. The
first problem is that the drillbit script get detached and does not run in
the foreground. My question is this: how can I get to run the drillbits in
the foreground to allow me use supervisord to manage them?

Best regards

Re: Controlling the Lifecycle of Drill using Supervisord

[John Omernik <jo...@omernik.com>]

So I am now worried I have messed up my own installations on Mesos, so I
dove into the two scripts, and found that main things that drillbit.sh sets
up is logging. I have moved that to my drill-env.sh which via drill-config
is called when executing the runbit.  I setup the logging etc there,
because I wanted to be in more control of what is happening with logging
and I use MapRFS as my logging destination rather than local hosts... just
makes more sense to have the logs centralized. The only others things I
noticed in the drillbit.sh were:

niceness - I don't set niceness, but per drillbit.sh it sets it to 0 and
that's the default, so I think I ok here and then the GC Server options
which I think have to do with logging more than anything.  (Maybe I am
reading that wrong) Basically, as I am reading things, as long as I setup
logging (which I do in drill-env, I should be good with runbit. If I am
missing something here, please let me know :)



On Sun, May 29, 2016 at 4:42 PM, John Omernik <jo...@omernik.com> wrote:

> So what does drillbit.sh setup that is missing from runbit? It's been
> working for me, but am I missing something? :)
>
>
> On Sunday, May 29, 2016, Paul Rogers <pr...@maprtech.com> wrote:
>
>> Note that drillbit.sh does quite a bit of setup, then turns around to
>> call runbit. You’ll have to duplicate that setup in your own wrapper
>> script. As an alternative, you can create a new “command” in drillbit.sh to
>> call runbit in the foreground.
>>
>> This same issue arose in the Drill-on-YARN work and the “run in
>> foreground” option in drillbit.sh was my first attempt at solving the
>> problem. In the end, we created a purpose-built script for running under
>> YARN to handle other YARN weirdness.
>>
>> Still, when I check in the YARN work, perhaps I should leave the “run in
>> foreground” option in drillbit.sh for cases such as this.
>>
>> Thoughts?
>>
>> Thanks,
>>
>> - Paul
>>
>> > On May 29, 2016, at 8:59 AM, Charles Nnamdi Akalugwu <
>> cprenzberg@gmail.com> wrote:
>> >
>> > ah excellent John!! That is exactly what I wanted! :)
>> >
>> > On Sun, May 29, 2016 at 5:00 PM, John Omernik <jo...@omernik.com> wrote:
>> >
>> >> Instead of using drillbit.sh use runbit. I use that with Marathon on
>> Mesos,
>> >> and it runs it in the foreground and works well.
>> >>
>> >> John
>> >>
>> >>
>> >>
>> >> On Sun, May 29, 2016 at 9:55 AM, Charles Nnamdi Akalugwu <
>> >> cprenzberg@gmail.com> wrote:
>> >>
>> >>> Hi guys,
>> >>>
>> >>> I would like to manage the lifecycle of my drillbits using
>> supervisord.
>> >> The
>> >>> first problem is that the drillbit script get detached and does not
>> run
>> >> in
>> >>> the foreground. My question is this: how can I get to run the
>> drillbits
>> >> in
>> >>> the foreground to allow me use supervisord to manage them?
>> >>>
>> >>> Best regards
>> >>>
>> >>
>>
>>
>
> --
> Sent from my iThing
>

Re: Controlling the Lifecycle of Drill using Supervisord

[Paul Rogers <pr...@maprtech.com>]

Hi John,

Most of the setup is done in drill-config.sh, which runbit loads. Drillbit.sh adds:

* Log file setup
* Java GC log file setup (which, I’d guess, most folks never use)

If you call runbit, then you probably do your own logging setup. For example in Drill-on-YARN, logging goes to the YARN log directories, not $DRILL_HOME/log. Logging is of two forms: 1) capture of stderr and stdout from runbit, and 2) Drill internal logging. Your script that calls runbit can do the stdout/stderr capture. But, runbit requires that the caller set up DRILLBIT_LOG_PATH and DRILLBIT_QUERY_LOG_PATH for internal logging. Here’s the actual execution line:

exec $JAVA -Dlog.path=$DRILLBIT_LOG_PATH -Dlog.query.path=$DRILLBIT_QUERY_LOG_PATH $DRILL_ALL_JAVA_OPTS -cp $CP org.apache.drill.exec.server.Drillbit

The two log variables are set by drillbit.sh, not drill-config.sh.

Some other things I noticed…

Today, both runbit and drillbit.sh load drill-config.sh. This can be awkward as some environment variables are additive and so we get duplicates of some settings. No harm, just messy.

Also, the scripts are supposed to allow you to set properties using the -Dname=value syntax, but those settings get lost currently.  Further, there is a bit of an issue where the sqlline client shares the same options as drillbit, causing sqlline to reserve more memory than it really needs. See DRILL-4581 for details.

For Drill-on-YARN, I cleaned up the scripts quite a bit. It is good to know that you are using runbit by itself. Sounds like we need to retain something equivalent. When you get the new Drill version with Drill-on-YARN, can you switch to using something like this:

$DRILL_HOME/bin/drillbit.sh run

Where “run” is a new option we would provide to run Drill as a child process. If you can switch, then we need only one path through the scripts. Or, should we try to make runbit work both stand-alone and when called from drillbit.sh (and the new yarn-drillbit.sh)?

Thanks,

- Paul

> On May 29, 2016, at 2:42 PM, John Omernik <jo...@omernik.com> wrote:
> 
> So what does drillbit.sh setup that is missing from runbit? It's been
> working for me, but am I missing something? :)
> 
> On Sunday, May 29, 2016, Paul Rogers <progers@maprtech.com <ma...@maprtech.com>> wrote:
> 
>> Note that drillbit.sh does quite a bit of setup, then turns around to call
>> runbit. You’ll have to duplicate that setup in your own wrapper script. As
>> an alternative, you can create a new “command” in drillbit.sh to call
>> runbit in the foreground.
>> 
>> This same issue arose in the Drill-on-YARN work and the “run in
>> foreground” option in drillbit.sh was my first attempt at solving the
>> problem. In the end, we created a purpose-built script for running under
>> YARN to handle other YARN weirdness.
>> 
>> Still, when I check in the YARN work, perhaps I should leave the “run in
>> foreground” option in drillbit.sh for cases such as this.
>> 
>> Thoughts?
>> 
>> Thanks,
>> 
>> - Paul
>> 
>>> On May 29, 2016, at 8:59 AM, Charles Nnamdi Akalugwu <
>> cprenzberg@gmail.com <javascript:;>> wrote:
>>> 
>>> ah excellent John!! That is exactly what I wanted! :)
>>> 
>>> On Sun, May 29, 2016 at 5:00 PM, John Omernik <john@omernik.com
>> <javascript:;>> wrote:
>>> 
>>>> Instead of using drillbit.sh use runbit. I use that with Marathon on
>> Mesos,
>>>> and it runs it in the foreground and works well.
>>>> 
>>>> John
>>>> 
>>>> 
>>>> 
>>>> On Sun, May 29, 2016 at 9:55 AM, Charles Nnamdi Akalugwu <
>>>> cprenzberg@gmail.com <javascript:;>> wrote:
>>>> 
>>>>> Hi guys,
>>>>> 
>>>>> I would like to manage the lifecycle of my drillbits using supervisord.
>>>> The
>>>>> first problem is that the drillbit script get detached and does not run
>>>> in
>>>>> the foreground. My question is this: how can I get to run the drillbits
>>>> in
>>>>> the foreground to allow me use supervisord to manage them?
>>>>> 
>>>>> Best regards
>>>>> 
>>>> 
>> 
>> 
> 
> -- 
> Sent from my iThing

Re: Controlling the Lifecycle of Drill using Supervisord

[John Omernik <jo...@omernik.com>]

So what does drillbit.sh setup that is missing from runbit? It's been
working for me, but am I missing something? :)

On Sunday, May 29, 2016, Paul Rogers <pr...@maprtech.com> wrote:

> Note that drillbit.sh does quite a bit of setup, then turns around to call
> runbit. You’ll have to duplicate that setup in your own wrapper script. As
> an alternative, you can create a new “command” in drillbit.sh to call
> runbit in the foreground.
>
> This same issue arose in the Drill-on-YARN work and the “run in
> foreground” option in drillbit.sh was my first attempt at solving the
> problem. In the end, we created a purpose-built script for running under
> YARN to handle other YARN weirdness.
>
> Still, when I check in the YARN work, perhaps I should leave the “run in
> foreground” option in drillbit.sh for cases such as this.
>
> Thoughts?
>
> Thanks,
>
> - Paul
>
> > On May 29, 2016, at 8:59 AM, Charles Nnamdi Akalugwu <
> cprenzberg@gmail.com <javascript:;>> wrote:
> >
> > ah excellent John!! That is exactly what I wanted! :)
> >
> > On Sun, May 29, 2016 at 5:00 PM, John Omernik <john@omernik.com
> <javascript:;>> wrote:
> >
> >> Instead of using drillbit.sh use runbit. I use that with Marathon on
> Mesos,
> >> and it runs it in the foreground and works well.
> >>
> >> John
> >>
> >>
> >>
> >> On Sun, May 29, 2016 at 9:55 AM, Charles Nnamdi Akalugwu <
> >> cprenzberg@gmail.com <javascript:;>> wrote:
> >>
> >>> Hi guys,
> >>>
> >>> I would like to manage the lifecycle of my drillbits using supervisord.
> >> The
> >>> first problem is that the drillbit script get detached and does not run
> >> in
> >>> the foreground. My question is this: how can I get to run the drillbits
> >> in
> >>> the foreground to allow me use supervisord to manage them?
> >>>
> >>> Best regards
> >>>
> >>
>
>

-- 
Sent from my iThing

Re: Controlling the Lifecycle of Drill using Supervisord

[Paul Rogers <pr...@maprtech.com>]

Note that drillbit.sh does quite a bit of setup, then turns around to call runbit. You’ll have to duplicate that setup in your own wrapper script. As an alternative, you can create a new “command” in drillbit.sh to call runbit in the foreground.

This same issue arose in the Drill-on-YARN work and the “run in foreground” option in drillbit.sh was my first attempt at solving the problem. In the end, we created a purpose-built script for running under YARN to handle other YARN weirdness.

Still, when I check in the YARN work, perhaps I should leave the “run in foreground” option in drillbit.sh for cases such as this.

Thoughts?

Thanks,

- Paul

> On May 29, 2016, at 8:59 AM, Charles Nnamdi Akalugwu <cp...@gmail.com> wrote:
> 
> ah excellent John!! That is exactly what I wanted! :)
> 
> On Sun, May 29, 2016 at 5:00 PM, John Omernik <jo...@omernik.com> wrote:
> 
>> Instead of using drillbit.sh use runbit. I use that with Marathon on Mesos,
>> and it runs it in the foreground and works well.
>> 
>> John
>> 
>> 
>> 
>> On Sun, May 29, 2016 at 9:55 AM, Charles Nnamdi Akalugwu <
>> cprenzberg@gmail.com> wrote:
>> 
>>> Hi guys,
>>> 
>>> I would like to manage the lifecycle of my drillbits using supervisord.
>> The
>>> first problem is that the drillbit script get detached and does not run
>> in
>>> the foreground. My question is this: how can I get to run the drillbits
>> in
>>> the foreground to allow me use supervisord to manage them?
>>> 
>>> Best regards
>>> 
>> 

Re: Controlling the Lifecycle of Drill using Supervisord

[Charles Nnamdi Akalugwu <cp...@gmail.com>]

ah excellent John!! That is exactly what I wanted! :)

On Sun, May 29, 2016 at 5:00 PM, John Omernik <jo...@omernik.com> wrote:

> Instead of using drillbit.sh use runbit. I use that with Marathon on Mesos,
> and it runs it in the foreground and works well.
>
> John
>
>
>
> On Sun, May 29, 2016 at 9:55 AM, Charles Nnamdi Akalugwu <
> cprenzberg@gmail.com> wrote:
>
> > Hi guys,
> >
> > I would like to manage the lifecycle of my drillbits using supervisord.
> The
> > first problem is that the drillbit script get detached and does not run
> in
> > the foreground. My question is this: how can I get to run the drillbits
> in
> > the foreground to allow me use supervisord to manage them?
> >
> > Best regards
> >
>

Re: Controlling the Lifecycle of Drill using Supervisord

[John Omernik <jo...@omernik.com>]

Instead of using drillbit.sh use runbit. I use that with Marathon on Mesos,
and it runs it in the foreground and works well.

John



On Sun, May 29, 2016 at 9:55 AM, Charles Nnamdi Akalugwu <
cprenzberg@gmail.com> wrote:

> Hi guys,
>
> I would like to manage the lifecycle of my drillbits using supervisord. The
> first problem is that the drillbit script get detached and does not run in
> the foreground. My question is this: how can I get to run the drillbits in
> the foreground to allow me use supervisord to manage them?
>
> Best regards
>