You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@drill.apache.org by Alexandre Rodrigues <al...@gmail.com> on 2015/10/29 19:17:23 UTC

Drill queues – per foreman or across all Drill cluster?

Hi,

I'd like to confirm if Drill queues are per foreman or across all Drill
cluster?
In the case of being global, where is the information stored? Zookeeper?

Thanks
--
Alexandre Rodrigues

Re: Drill queues – per foreman or across all Drill cluster?

Posted by Alexandre Rodrigues <al...@gmail.com>.

Thanks Hanifi

--
Alexandre Rodrigues

On Thu, Oct 29, 2015 at 6:40 PM, Hanifi Gunes <hg...@maprtech.com> wrote:

> Drill queue is logical and global abstraction. It is logical because there
> is no notion of task queues or workers. It simply relies on a distributed
> semaphore that in turn relies on Zookeeper which makes queues global.
> Whoever acquires the lock, proceeds with executing the query.
>
> -Hanifi
>
> On Thu, Oct 29, 2015 at 11:17 AM, Alexandre Rodrigues <
> alex.jose.rodrigues@gmail.com> wrote:
>
> > Hi,
> >
> > I'd like to confirm if Drill queues are per foreman or across all Drill
> > cluster?
> > In the case of being global, where is the information stored? Zookeeper?
> >
> > Thanks
> > --
> > Alexandre Rodrigues
> >
>

Re: Drill queues – per foreman or across all Drill cluster?

Posted by Jacques Nadeau <ja...@dremio.com>.

Just to clarify, the queues are ordered based on request timing. e.g. the
person who tries to start the query first will be the first to get the
semaphore. This is because Curator's shared semaphore is an ordered
construct. See [1] for more info.

[1] http://curator.apache.org/curator-recipes/shared-semaphore.html

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Thu, Oct 29, 2015 at 11:40 AM, Hanifi Gunes <hg...@maprtech.com> wrote:

> Drill queue is logical and global abstraction. It is logical because there
> is no notion of task queues or workers. It simply relies on a distributed
> semaphore that in turn relies on Zookeeper which makes queues global.
> Whoever acquires the lock, proceeds with executing the query.
>
> -Hanifi
>
> On Thu, Oct 29, 2015 at 11:17 AM, Alexandre Rodrigues <
> alex.jose.rodrigues@gmail.com> wrote:
>
> > Hi,
> >
> > I'd like to confirm if Drill queues are per foreman or across all Drill
> > cluster?
> > In the case of being global, where is the information stored? Zookeeper?
> >
> > Thanks
> > --
> > Alexandre Rodrigues
> >
>

Re: Drill queues – per foreman or across all Drill cluster?

Posted by Hanifi Gunes <hg...@maprtech.com>.

Drill queue is logical and global abstraction. It is logical because there
is no notion of task queues or workers. It simply relies on a distributed
semaphore that in turn relies on Zookeeper which makes queues global.
Whoever acquires the lock, proceeds with executing the query.

-Hanifi

On Thu, Oct 29, 2015 at 11:17 AM, Alexandre Rodrigues <
alex.jose.rodrigues@gmail.com> wrote:

> Hi,
>
> I'd like to confirm if Drill queues are per foreman or across all Drill
> cluster?
> In the case of being global, where is the information stored? Zookeeper?
>
> Thanks
> --
> Alexandre Rodrigues
>