Single (embedded) drillbit as a service

[Krzysztof Kowalczyk <ko...@gmail.com>]

Hi,

I would like to have and ad-hoc deployment of Drill on a linux box working
with relatively small dataset, thus I don't want to use Zookeeper, but I
find drill shell scripts actively restricting such usage.

The `drillbit.sh start` will not work without zookeeper (or would it?) and
is explicitly discourage to use in such way in docs. I tried to installing
local zookeeper (as per some Stack Overflow answers) but that throws
exception:
org.apache.drill.exec.exception.DrillbitStartupException: Drillbit is
disallowed to bind to loopback address in distributed mode.

So I tried to use `drill-embedded`, but that does not work when run in non
interactive mode, i.e.
`nohup ./drill-embedded >> log 2>&1 &`

Is there a non hacky way of staring drill as a service in embedded mode or
starting just the drillbit without sqline?

On the side node, the FAQ suggest this email to be used in case of
questions, is there a slack, forum or any other ways of asking questions?

Regards,
Krzysztof

Re: Single (embedded) drillbit as a service

[Jiang Wu <ji...@mulesoft.com>]

We use an upstart (http://upstart.ubuntu.com/getting-started.html) script to launch drill-embedded as a service.  It works fine on local Ubuntu machines and AWS ec2 instances.  The command used inside the upstart script is similar to what you have:

exec /bin/sh "a script that eventually calls drill-embedded" >> "some log file" 2>&1

Hope this helps.

-- Jiang


On 3/13/18, 10:36 PM, "Krzysztof Kowalczyk" <ko...@gmail.com> wrote:

    Hi,
    
    I would like to have and ad-hoc deployment of Drill on a linux box working
    with relatively small dataset, thus I don't want to use Zookeeper, but I
    find drill shell scripts actively restricting such usage.
    
    The `drillbit.sh start` will not work without zookeeper (or would it?) and
    is explicitly discourage to use in such way in docs. I tried to installing
    local zookeeper (as per some Stack Overflow answers) but that throws
    exception:
    org.apache.drill.exec.exception.DrillbitStartupException: Drillbit is
    disallowed to bind to loopback address in distributed mode.
    
    So I tried to use `drill-embedded`, but that does not work when run in non
    interactive mode, i.e.
    `nohup ./drill-embedded >> log 2>&1 &`
    
    Is there a non hacky way of staring drill as a service in embedded mode or
    starting just the drillbit without sqline?
    
    On the side node, the FAQ suggest this email to be used in case of
    questions, is there a slack, forum or any other ways of asking questions?
    
    Regards,
    Krzysztof
    

Re: Single (embedded) drillbit as a service

[Kunal Khatua <ku...@apache.org>]

 The trouble with your approach is that making it a background process
pauses the process in an interactive mode, until you bring it to the
foreground.

You cannot use an Embedded Drillbit with Zookeeper. You should start it off
as a regular drillbit instance.
Zookeeper only serves the purpose of coordinating amongst different
Drillbits, so it is not avoidable.

That said, you can use Embedded Drill as a service by starting it in a
Linux screen session and then detaching from it. There should be tricks to
launch something into a screen session and detaching it via scripts.

I believe there are folks on this list who have used Embedded Drill in the
way you described your requirement. Look through the past mailing list if
you need more information.



On Tue, Mar 13, 2018 at 6:12 PM, Krzysztof Kowalczyk <
kowalczyk.krzysztof@gmail.com> wrote:

> Hi,
>
> I would like to have and ad-hoc deployment of Drill on a linux box working
> with relatively small dataset, thus I don't want to use Zookeeper, but I
> find drill shell scripts actively restricting such usage.
>
> The `drillbit.sh start` will not work without zookeeper (or would it?) and
> is explicitly discourage to use in such way in docs. I tried to installing
> local zookeeper (as per some Stack Overflow answers) but that throws
> exception:
> org.apache.drill.exec.exception.DrillbitStartupException: Drillbit is
> disallowed to bind to loopback address in distributed mode.
>
> So I tried to use `drill-embedded`, but that does not work when run in non
> interactive mode, i.e.
> `nohup ./drill-embedded >> log 2>&1 &`
>
> Is there a non hacky way of staring drill as a service in embedded mode or
> starting just the drillbit without sqline?
>
> On the side node, the FAQ suggest this email to be used in case of
> questions, is there a slack, forum or any other ways of asking questions?
>
> Regards,
> Krzysztof
>