Drill Presentation - Question - Resource Manager Integration

[Jim Scott <js...@maprtech.com>]

I was giving a presentation to the St. Louis Hadoop Users Group and the
question was:
Which resource managers will Drill work with and when?

I do not know an answer for that, can anyone else help with that?

Re: Drill Presentation - Question - Resource Manager Integration

[Peter Grman <pe...@gmail.com>]

The Yarn integration, I've implemented, is based on Apache Twill and so far
basically requires only two classes, a tiny change for the default class
loader and the Twill dependencies in pom.xml.

It's good enough for starting Drill but there are some problems when
containers won't shut down and of course it would require a more dynamic
interface/api to be able to start and stop containers on the fly.

Cheers
Peter
On Sep 17, 2014 5:51 PM, "Jacques Nadeau" <ja...@apache.org> wrote:

> There is actually an available patch with an initial implementation at [1]
> for YARN.  We should take a look and see what we need to do to move that
> into Drill master.
>
> [1] https://issues.apache.org/jira/browse/DRILL-142
>
>
>
> On Wed, Sep 17, 2014 at 8:31 AM, Neeraja Rentachintala <
> nrentachintala@maprtech.com> wrote:
>
> > YARN integration is in consideration , probably by end of the year.
> > For docker intergation, sequenceiq did some work.
> > http://blog.sequenceiq.com/blog/2014/09/11/apache-drill-docker/
> >
> > On Wed, Sep 17, 2014 at 8:02 AM, Ted Dunning <te...@gmail.com>
> > wrote:
> >
> > >
> > > Well just looking at the jira traffic suggests that docker, mesos and
> > yarn
> > > are all on the table.  Exactly which ones drop when is an open
> question.
> > >
> > > Sent from my iPhone
> > >
> > > > On Sep 17, 2014, at 9:42, Jim Scott <js...@maprtech.com> wrote:
> > > >
> > > > Which resource managers will Drill work with and when?
> > >
> >
>

Re: Drill Presentation - Question - Resource Manager Integration

[Jacques Nadeau <ja...@apache.org>]

There is actually an available patch with an initial implementation at [1]
for YARN.  We should take a look and see what we need to do to move that
into Drill master.

[1] https://issues.apache.org/jira/browse/DRILL-142



On Wed, Sep 17, 2014 at 8:31 AM, Neeraja Rentachintala <
nrentachintala@maprtech.com> wrote:

> YARN integration is in consideration , probably by end of the year.
> For docker intergation, sequenceiq did some work.
> http://blog.sequenceiq.com/blog/2014/09/11/apache-drill-docker/
>
> On Wed, Sep 17, 2014 at 8:02 AM, Ted Dunning <te...@gmail.com>
> wrote:
>
> >
> > Well just looking at the jira traffic suggests that docker, mesos and
> yarn
> > are all on the table.  Exactly which ones drop when is an open question.
> >
> > Sent from my iPhone
> >
> > > On Sep 17, 2014, at 9:42, Jim Scott <js...@maprtech.com> wrote:
> > >
> > > Which resource managers will Drill work with and when?
> >
>

Re: Drill Presentation - Question - Resource Manager Integration

[Neeraja Rentachintala <nr...@maprtech.com>]

YARN integration is in consideration , probably by end of the year.
For docker intergation, sequenceiq did some work.
http://blog.sequenceiq.com/blog/2014/09/11/apache-drill-docker/

On Wed, Sep 17, 2014 at 8:02 AM, Ted Dunning <te...@gmail.com> wrote:

>
> Well just looking at the jira traffic suggests that docker, mesos and yarn
> are all on the table.  Exactly which ones drop when is an open question.
>
> Sent from my iPhone
>
> > On Sep 17, 2014, at 9:42, Jim Scott <js...@maprtech.com> wrote:
> >
> > Which resource managers will Drill work with and when?
>

Re: Drill Presentation - Question - Resource Manager Integration

[Ted Dunning <te...@gmail.com>]

Well just looking at the jira traffic suggests that docker, mesos and yarn are all on the table.  Exactly which ones drop when is an open question.  

Sent from my iPhone

> On Sep 17, 2014, at 9:42, Jim Scott <js...@maprtech.com> wrote:
> 
> Which resource managers will Drill work with and when?