You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Michael Hausenblas <mh...@maprtech.com> on 2013/10/28 14:04:19 UTC
Towards doc of distributed mode of M1
All,
I’ve put together a sandbox repo with a draft for instructions how to set up M1 in distributed mode:
https://github.com/mhausenblas/apache-drill-sandbox/tree/master/M1
Feedback welcome. Once it stabilises, we can transfer to the Confluence wiki.
Cheers,
Michael
--
Michael Hausenblas
Ireland, Europe
http://mhausenblas.info/
Re: Towards doc of distributed mode of M1
Posted by Michael Hausenblas <mh...@maprtech.com>.
> Got it, any pointers for this for further details?
Not yet. WIP. But IIRC Tim was working on something similar for AWS with Whirr.
BTW, I’ve updated https://github.com/mhausenblas/apache-drill-sandbox/tree/master/M1 now to also include CLI details …
Cheers,
Michael
--
Michael Hausenblas
Ireland, Europe
http://mhausenblas.info/
On 30 Oct 2013, at 09:27, Ashish <pa...@gmail.com> wrote:
> On Wed, Oct 30, 2013 at 2:25 PM, Michael Hausenblas <
> mhausenblas@maprtech.com> wrote:
>
>>
>>> Followed the steps and was able to run the cluster and execute query :)
>>> Thanks for this doc.
>>
>> Great! Glad it was useful.
>>
>>
>>> 1. For Zookeeper section, can point to Zookeeper docs. The setting are
>>> generic to Zookeeper installation. After reading the statement "Create a
>>> config file conf/apache-drill-zoo.cfg that holds the Apache Drill
>> specific
>>> settings" and was looking for specific needs that Drill might have, but
>>> they were generic, so I reused my existing Zookeeper setup.
>>
>> Yup, that’s right. In fact, there is something more in the pipeline (GCE
>> deployment) so this one is a bit of a future investment ;)
>>
>
> Got it, any pointers for this for further details?
>
>
>>
>>
>>> Rest all worked fine. 2nd query on sqlline ran with following warning
>>> (looks like a friendly warning message)
>>>
>>> Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
>>> WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does
>> not
>>> support DISTINCT aggregates
>>
>> Yeah, encountered that as well. Anyone wanna chime in here?
>>
>>
>> Cheers,
>> Michael
>>
>> --
>> Michael Hausenblas
>> Ireland, Europe
>> http://mhausenblas.info/
>>
>> On 30 Oct 2013, at 08:39, Ashish <pa...@gmail.com> wrote:
>>
>>> Hi Michael,
>>>
>>> Followed the steps and was able to run the cluster and execute query :)
>>> Thanks for this doc.
>>>
>>>
>>> Minor feedback
>>> 1. For Zookeeper section, can point to Zookeeper docs. The setting are
>>> generic to Zookeeper installation. After reading the statement "Create a
>>> config file conf/apache-drill-zoo.cfg that holds the Apache Drill
>> specific
>>> settings" and was looking for specific needs that Drill might have, but
>>> they were generic, so I reused my existing Zookeeper setup.
>>>
>>> Rest all worked fine. 2nd query on sqlline ran with following warning
>>> (looks like a friendly warning message)
>>>
>>> Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
>>> WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does
>> not
>>> support DISTINCT aggregates
>>>
>>>
>>>
>>> On Mon, Oct 28, 2013 at 6:34 PM, Michael Hausenblas <
>>> mhausenblas@maprtech.com> wrote:
>>>
>>>> All,
>>>>
>>>> I’ve put together a sandbox repo with a draft for instructions how to
>> set
>>>> up M1 in distributed mode:
>>>>
>>>> https://github.com/mhausenblas/apache-drill-sandbox/tree/master/M1
>>>>
>>>>
>>>> Feedback welcome. Once it stabilises, we can transfer to the Confluence
>>>> wiki.
>>>>
>>>>
>>>> Cheers,
>>>> Michael
>>>>
>>>> --
>>>> Michael Hausenblas
>>>> Ireland, Europe
>>>> http://mhausenblas.info/
>>>>
>>>>
>>>
>>>
>>> --
>>> thanks
>>> ashish
>>>
>>> Blog: http://www.ashishpaliwal.com/blog
>>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>
>>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Towards doc of distributed mode of M1
Posted by Michael Hausenblas <mh...@maprtech.com>.
> Got it, any pointers for this for further details?
Not yet. WIP. But IIRC Tim was working on something similar for AWS with Whirr.
BTW, I’ve updated https://github.com/mhausenblas/apache-drill-sandbox/tree/master/M1 now to also include CLI details …
Cheers,
Michael
--
Michael Hausenblas
Ireland, Europe
http://mhausenblas.info/
On 30 Oct 2013, at 09:27, Ashish <pa...@gmail.com> wrote:
> On Wed, Oct 30, 2013 at 2:25 PM, Michael Hausenblas <
> mhausenblas@maprtech.com> wrote:
>
>>
>>> Followed the steps and was able to run the cluster and execute query :)
>>> Thanks for this doc.
>>
>> Great! Glad it was useful.
>>
>>
>>> 1. For Zookeeper section, can point to Zookeeper docs. The setting are
>>> generic to Zookeeper installation. After reading the statement "Create a
>>> config file conf/apache-drill-zoo.cfg that holds the Apache Drill
>> specific
>>> settings" and was looking for specific needs that Drill might have, but
>>> they were generic, so I reused my existing Zookeeper setup.
>>
>> Yup, that’s right. In fact, there is something more in the pipeline (GCE
>> deployment) so this one is a bit of a future investment ;)
>>
>
> Got it, any pointers for this for further details?
>
>
>>
>>
>>> Rest all worked fine. 2nd query on sqlline ran with following warning
>>> (looks like a friendly warning message)
>>>
>>> Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
>>> WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does
>> not
>>> support DISTINCT aggregates
>>
>> Yeah, encountered that as well. Anyone wanna chime in here?
>>
>>
>> Cheers,
>> Michael
>>
>> --
>> Michael Hausenblas
>> Ireland, Europe
>> http://mhausenblas.info/
>>
>> On 30 Oct 2013, at 08:39, Ashish <pa...@gmail.com> wrote:
>>
>>> Hi Michael,
>>>
>>> Followed the steps and was able to run the cluster and execute query :)
>>> Thanks for this doc.
>>>
>>>
>>> Minor feedback
>>> 1. For Zookeeper section, can point to Zookeeper docs. The setting are
>>> generic to Zookeeper installation. After reading the statement "Create a
>>> config file conf/apache-drill-zoo.cfg that holds the Apache Drill
>> specific
>>> settings" and was looking for specific needs that Drill might have, but
>>> they were generic, so I reused my existing Zookeeper setup.
>>>
>>> Rest all worked fine. 2nd query on sqlline ran with following warning
>>> (looks like a friendly warning message)
>>>
>>> Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
>>> WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does
>> not
>>> support DISTINCT aggregates
>>>
>>>
>>>
>>> On Mon, Oct 28, 2013 at 6:34 PM, Michael Hausenblas <
>>> mhausenblas@maprtech.com> wrote:
>>>
>>>> All,
>>>>
>>>> I’ve put together a sandbox repo with a draft for instructions how to
>> set
>>>> up M1 in distributed mode:
>>>>
>>>> https://github.com/mhausenblas/apache-drill-sandbox/tree/master/M1
>>>>
>>>>
>>>> Feedback welcome. Once it stabilises, we can transfer to the Confluence
>>>> wiki.
>>>>
>>>>
>>>> Cheers,
>>>> Michael
>>>>
>>>> --
>>>> Michael Hausenblas
>>>> Ireland, Europe
>>>> http://mhausenblas.info/
>>>>
>>>>
>>>
>>>
>>> --
>>> thanks
>>> ashish
>>>
>>> Blog: http://www.ashishpaliwal.com/blog
>>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>
>>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Towards doc of distributed mode of M1
Posted by Ashish <pa...@gmail.com>.
On Wed, Oct 30, 2013 at 2:25 PM, Michael Hausenblas <
mhausenblas@maprtech.com> wrote:
>
> > Followed the steps and was able to run the cluster and execute query :)
> > Thanks for this doc.
>
> Great! Glad it was useful.
>
>
> > 1. For Zookeeper section, can point to Zookeeper docs. The setting are
> > generic to Zookeeper installation. After reading the statement "Create a
> > config file conf/apache-drill-zoo.cfg that holds the Apache Drill
> specific
> > settings" and was looking for specific needs that Drill might have, but
> > they were generic, so I reused my existing Zookeeper setup.
>
> Yup, that’s right. In fact, there is something more in the pipeline (GCE
> deployment) so this one is a bit of a future investment ;)
>
Got it, any pointers for this for further details?
>
>
> > Rest all worked fine. 2nd query on sqlline ran with following warning
> > (looks like a friendly warning message)
> >
> > Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
> > WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does
> not
> > support DISTINCT aggregates
>
> Yeah, encountered that as well. Anyone wanna chime in here?
>
>
> Cheers,
> Michael
>
> --
> Michael Hausenblas
> Ireland, Europe
> http://mhausenblas.info/
>
> On 30 Oct 2013, at 08:39, Ashish <pa...@gmail.com> wrote:
>
> > Hi Michael,
> >
> > Followed the steps and was able to run the cluster and execute query :)
> > Thanks for this doc.
> >
> >
> > Minor feedback
> > 1. For Zookeeper section, can point to Zookeeper docs. The setting are
> > generic to Zookeeper installation. After reading the statement "Create a
> > config file conf/apache-drill-zoo.cfg that holds the Apache Drill
> specific
> > settings" and was looking for specific needs that Drill might have, but
> > they were generic, so I reused my existing Zookeeper setup.
> >
> > Rest all worked fine. 2nd query on sqlline ran with following warning
> > (looks like a friendly warning message)
> >
> > Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
> > WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does
> not
> > support DISTINCT aggregates
> >
> >
> >
> > On Mon, Oct 28, 2013 at 6:34 PM, Michael Hausenblas <
> > mhausenblas@maprtech.com> wrote:
> >
> >> All,
> >>
> >> I’ve put together a sandbox repo with a draft for instructions how to
> set
> >> up M1 in distributed mode:
> >>
> >> https://github.com/mhausenblas/apache-drill-sandbox/tree/master/M1
> >>
> >>
> >> Feedback welcome. Once it stabilises, we can transfer to the Confluence
> >> wiki.
> >>
> >>
> >> Cheers,
> >> Michael
> >>
> >> --
> >> Michael Hausenblas
> >> Ireland, Europe
> >> http://mhausenblas.info/
> >>
> >>
> >
> >
> > --
> > thanks
> > ashish
> >
> > Blog: http://www.ashishpaliwal.com/blog
> > My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Towards doc of distributed mode of M1
Posted by Ashish <pa...@gmail.com>.
On Wed, Oct 30, 2013 at 2:25 PM, Michael Hausenblas <
mhausenblas@maprtech.com> wrote:
>
> > Followed the steps and was able to run the cluster and execute query :)
> > Thanks for this doc.
>
> Great! Glad it was useful.
>
>
> > 1. For Zookeeper section, can point to Zookeeper docs. The setting are
> > generic to Zookeeper installation. After reading the statement "Create a
> > config file conf/apache-drill-zoo.cfg that holds the Apache Drill
> specific
> > settings" and was looking for specific needs that Drill might have, but
> > they were generic, so I reused my existing Zookeeper setup.
>
> Yup, that’s right. In fact, there is something more in the pipeline (GCE
> deployment) so this one is a bit of a future investment ;)
>
Got it, any pointers for this for further details?
>
>
> > Rest all worked fine. 2nd query on sqlline ran with following warning
> > (looks like a friendly warning message)
> >
> > Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
> > WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does
> not
> > support DISTINCT aggregates
>
> Yeah, encountered that as well. Anyone wanna chime in here?
>
>
> Cheers,
> Michael
>
> --
> Michael Hausenblas
> Ireland, Europe
> http://mhausenblas.info/
>
> On 30 Oct 2013, at 08:39, Ashish <pa...@gmail.com> wrote:
>
> > Hi Michael,
> >
> > Followed the steps and was able to run the cluster and execute query :)
> > Thanks for this doc.
> >
> >
> > Minor feedback
> > 1. For Zookeeper section, can point to Zookeeper docs. The setting are
> > generic to Zookeeper installation. After reading the statement "Create a
> > config file conf/apache-drill-zoo.cfg that holds the Apache Drill
> specific
> > settings" and was looking for specific needs that Drill might have, but
> > they were generic, so I reused my existing Zookeeper setup.
> >
> > Rest all worked fine. 2nd query on sqlline ran with following warning
> > (looks like a friendly warning message)
> >
> > Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
> > WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does
> not
> > support DISTINCT aggregates
> >
> >
> >
> > On Mon, Oct 28, 2013 at 6:34 PM, Michael Hausenblas <
> > mhausenblas@maprtech.com> wrote:
> >
> >> All,
> >>
> >> I’ve put together a sandbox repo with a draft for instructions how to
> set
> >> up M1 in distributed mode:
> >>
> >> https://github.com/mhausenblas/apache-drill-sandbox/tree/master/M1
> >>
> >>
> >> Feedback welcome. Once it stabilises, we can transfer to the Confluence
> >> wiki.
> >>
> >>
> >> Cheers,
> >> Michael
> >>
> >> --
> >> Michael Hausenblas
> >> Ireland, Europe
> >> http://mhausenblas.info/
> >>
> >>
> >
> >
> > --
> > thanks
> > ashish
> >
> > Blog: http://www.ashishpaliwal.com/blog
> > My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Towards doc of distributed mode of M1
Posted by Michael Hausenblas <mh...@maprtech.com>.
> Followed the steps and was able to run the cluster and execute query :)
> Thanks for this doc.
Great! Glad it was useful.
> 1. For Zookeeper section, can point to Zookeeper docs. The setting are
> generic to Zookeeper installation. After reading the statement "Create a
> config file conf/apache-drill-zoo.cfg that holds the Apache Drill specific
> settings" and was looking for specific needs that Drill might have, but
> they were generic, so I reused my existing Zookeeper setup.
Yup, that’s right. In fact, there is something more in the pipeline (GCE deployment) so this one is a bit of a future investment ;)
> Rest all worked fine. 2nd query on sqlline ran with following warning
> (looks like a friendly warning message)
>
> Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
> WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does not
> support DISTINCT aggregates
Yeah, encountered that as well. Anyone wanna chime in here?
Cheers,
Michael
--
Michael Hausenblas
Ireland, Europe
http://mhausenblas.info/
On 30 Oct 2013, at 08:39, Ashish <pa...@gmail.com> wrote:
> Hi Michael,
>
> Followed the steps and was able to run the cluster and execute query :)
> Thanks for this doc.
>
>
> Minor feedback
> 1. For Zookeeper section, can point to Zookeeper docs. The setting are
> generic to Zookeeper installation. After reading the statement "Create a
> config file conf/apache-drill-zoo.cfg that holds the Apache Drill specific
> settings" and was looking for specific needs that Drill might have, but
> they were generic, so I reused my existing Zookeeper setup.
>
> Rest all worked fine. 2nd query on sqlline ran with following warning
> (looks like a friendly warning message)
>
> Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
> WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does not
> support DISTINCT aggregates
>
>
>
> On Mon, Oct 28, 2013 at 6:34 PM, Michael Hausenblas <
> mhausenblas@maprtech.com> wrote:
>
>> All,
>>
>> I’ve put together a sandbox repo with a draft for instructions how to set
>> up M1 in distributed mode:
>>
>> https://github.com/mhausenblas/apache-drill-sandbox/tree/master/M1
>>
>>
>> Feedback welcome. Once it stabilises, we can transfer to the Confluence
>> wiki.
>>
>>
>> Cheers,
>> Michael
>>
>> --
>> Michael Hausenblas
>> Ireland, Europe
>> http://mhausenblas.info/
>>
>>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Towards doc of distributed mode of M1
Posted by Michael Hausenblas <mh...@maprtech.com>.
> Followed the steps and was able to run the cluster and execute query :)
> Thanks for this doc.
Great! Glad it was useful.
> 1. For Zookeeper section, can point to Zookeeper docs. The setting are
> generic to Zookeeper installation. After reading the statement "Create a
> config file conf/apache-drill-zoo.cfg that holds the Apache Drill specific
> settings" and was looking for specific needs that Drill might have, but
> they were generic, so I reused my existing Zookeeper setup.
Yup, that’s right. In fact, there is something more in the pipeline (GCE deployment) so this one is a bit of a future investment ;)
> Rest all worked fine. 2nd query on sqlline ran with following warning
> (looks like a friendly warning message)
>
> Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
> WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does not
> support DISTINCT aggregates
Yeah, encountered that as well. Anyone wanna chime in here?
Cheers,
Michael
--
Michael Hausenblas
Ireland, Europe
http://mhausenblas.info/
On 30 Oct 2013, at 08:39, Ashish <pa...@gmail.com> wrote:
> Hi Michael,
>
> Followed the steps and was able to run the cluster and execute query :)
> Thanks for this doc.
>
>
> Minor feedback
> 1. For Zookeeper section, can point to Zookeeper docs. The setting are
> generic to Zookeeper installation. After reading the statement "Create a
> config file conf/apache-drill-zoo.cfg that holds the Apache Drill specific
> settings" and was looking for specific needs that Drill might have, but
> they were generic, so I reused my existing Zookeeper setup.
>
> Rest all worked fine. 2nd query on sqlline ran with following warning
> (looks like a friendly warning message)
>
> Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
> WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does not
> support DISTINCT aggregates
>
>
>
> On Mon, Oct 28, 2013 at 6:34 PM, Michael Hausenblas <
> mhausenblas@maprtech.com> wrote:
>
>> All,
>>
>> I’ve put together a sandbox repo with a draft for instructions how to set
>> up M1 in distributed mode:
>>
>> https://github.com/mhausenblas/apache-drill-sandbox/tree/master/M1
>>
>>
>> Feedback welcome. Once it stabilises, we can transfer to the Confluence
>> wiki.
>>
>>
>> Cheers,
>> Michael
>>
>> --
>> Michael Hausenblas
>> Ireland, Europe
>> http://mhausenblas.info/
>>
>>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Towards doc of distributed mode of M1
Posted by Ashish <pa...@gmail.com>.
Hi Michael,
Followed the steps and was able to run the cluster and execute query :)
Thanks for this doc.
Minor feedback
1. For Zookeeper section, can point to Zookeeper docs. The setting are
generic to Zookeeper installation. After reading the statement "Create a
config file conf/apache-drill-zoo.cfg that holds the Apache Drill specific
settings" and was looking for specific needs that Drill might have, but
they were generic, so I reused my existing Zookeeper setup.
Rest all worked fine. 2nd query on sqlline ran with following warning
(looks like a friendly warning message)
Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does not
support DISTINCT aggregates
On Mon, Oct 28, 2013 at 6:34 PM, Michael Hausenblas <
mhausenblas@maprtech.com> wrote:
> All,
>
> I’ve put together a sandbox repo with a draft for instructions how to set
> up M1 in distributed mode:
>
> https://github.com/mhausenblas/apache-drill-sandbox/tree/master/M1
>
>
> Feedback welcome. Once it stabilises, we can transfer to the Confluence
> wiki.
>
>
> Cheers,
> Michael
>
> --
> Michael Hausenblas
> Ireland, Europe
> http://mhausenblas.info/
>
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Towards doc of distributed mode of M1
Posted by Ashish <pa...@gmail.com>.
Hi Michael,
Followed the steps and was able to run the cluster and execute query :)
Thanks for this doc.
Minor feedback
1. For Zookeeper section, can point to Zookeeper docs. The setting are
generic to Zookeeper installation. After reading the statement "Create a
config file conf/apache-drill-zoo.cfg that holds the Apache Drill specific
settings" and was looking for specific needs that Drill might have, but
they were generic, so I reused my existing Zookeeper setup.
Rest all worked fine. 2nd query on sqlline ran with following warning
(looks like a friendly warning message)
Oct 30, 2013 2:05:16 PM org.apache.drill.optiq.DrillAggregateRule onMatch
WARNING: org.eigenbase.rel.InvalidRelException: DrillAggregateRel does not
support DISTINCT aggregates
On Mon, Oct 28, 2013 at 6:34 PM, Michael Hausenblas <
mhausenblas@maprtech.com> wrote:
> All,
>
> I’ve put together a sandbox repo with a draft for instructions how to set
> up M1 in distributed mode:
>
> https://github.com/mhausenblas/apache-drill-sandbox/tree/master/M1
>
>
> Feedback welcome. Once it stabilises, we can transfer to the Confluence
> wiki.
>
>
> Cheers,
> Michael
>
> --
> Michael Hausenblas
> Ireland, Europe
> http://mhausenblas.info/
>
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal