You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hawq.apache.org by Jemish Patel <je...@gmail.com> on 2016/07/08 17:18:36 UTC
Re: Any interest in running HAWQ on n Docker containers?
Hi bigtop devs,
Wanted to see if there was interest and how we can work together to have
HAWQ run in Docker containers for the larger Hadoop community.
Let me know.
Jemish
>
> ---------- Forwarded message ----------
> From: Lei Chang <le...@apache.org>
> Date: Fri, Jun 24, 2016 at 6:37 PM
> Subject: Re: Any interest in running HAWQ on n Docker containers?
> To: dev <de...@hawq.incubator.apache.org>, dev@bigtop.apache.org
>
>
> @Roman, looks you forgot adding the mailing list.
>
> + dev@bigtop.apache.org
>
> Cheers
> Lei
>
>
> On Sat, Jun 25, 2016 at 6:30 AM, Roman Shaposhnik <ro...@shaposhnik.org>
> wrote:
>
> > Awesome! Let me add dev@bigtop to the original message then.
> >
> > Thanks,
> > Roman.
>
> >
> > On Fri, Jun 24, 2016 at 12:09 PM, Jemish Patel <je...@gmail.com>
> wrote:
> > > Yes I have added on top of the ones that Jon points to.
> > >
> > > Also agree that working with Bigtop makes sense.
> > >
> > >
> > > ---------- Forwarded message ----------
> > > From: Kavinder Dhaliwal <kd...@pivotal.io>
> > > Date: Fri, Jun 24, 2016 at 11:51 AM
> > > Subject: Re: Any interest in running HAWQ on n Docker containers?
> > > To: dev@hawq.incubator.apache.org
> > >
> > >
> > > I can vouch that Jemish's efforts on creating a Docker environment for
> > HAWQ
> > > are really helpful. They're an improvement on the links Jon referenced
> > > above. It'd be great to have the docker files part of the main
> repository
> > > or at least documented on the wiki
> > >
> > > I'm in favor of having any HAWQ related docker work to be in line with
> > the
> > > larger hadoop community so working with Bigtop makes good sense.
> > >
> > > Let me know how and we can get this integration effort on the way.
> > >
> > > Jemish Patel
> > >
> > > On Thu, Jun 23, 2016 at 4:58 PM, Roman Shaposhnik <
> roman@shaposhnik.org>
> > > wrote:
> > >
> > >> On Thu, Jun 23, 2016 at 4:46 PM, Jon Ernster <je...@zdatainc.com>
> > >> wrote:
> > >> > Check out:
> > >> >
> > >> > https://github.com/wangzw/hawq-devel-env
> > >> > https://hub.docker.com/r/mayjojo/hawq-test/
> > >> > https://hub.docker.com/r/mayjojo/hawq-devel/
> > >>
> > >> Great starting points! And perhaps something that we can coalesce with
> > >> Jemish's efforts as well. More comments inline:
> > >>
> > >> > On Thu, Jun 23, 2016 at 3:13 PM, Jemish Patel <jp...@pivotal.io>
> > wrote:
> > >> >
> > >> >> I am working on configuring /installing HAWQ on a really
> bare-bones 4
> > >> >> container docker environment to make it easy for dev/test.
> > >> >>
> > >> >> The advantages for devs are that it is light weight and one can get
> > to
> > >> >> feature development or reproducing an issue , etc and abstracting
> > away
> > >> all
> > >> >> the needed installation/configuration.
> > >> >>
> > >> >> Users could benefit as well as they can get a running HAWQ
> > environment
> > >> >> with ORCA, PXF, PL/python, PL/R and MADLib that they can use right
> > away
> > >> to
> > >> >> prove out a theory or even simply try it out.
> > >> >>
> > >> >> Would this be of interest to anyone?
> > >>
> > >> I think this is super interesting. And in fact it reminds me a lot of
> > >> the efforts that
> > >> Bigtop is pursuing for all the Bigdata projects. In fact, quite a few
> > >> projects in Hadoop
> > >> ecosystem are leveraging Bigtop for that type of integration. See the
> > >> following
> > >> excellent presentation by Trend Micro on the work that is being done:
> > >>
> > >>
> > >
> >
> http://schd.ws/hosted_files/apachebigdata2016/4e/Dockerized%20Hadoop%20Platform%20and%20Recent%20Updates%20in%20Apache%20Bigtop.pdf
> > >>
> > >> Recently Apache Apex leveraged Bigtop to create demo VM, etc. Zeppelin
> > >> folks are also pretty active.
> > >>
> > >> So perhaps, one suggestion here would be to do this integration work
> in
> > >> Bigtop
> > >> itself.
> > >>
> > >> What do you guys think?
> > >>
> > >> Thanks,
> > >> Roman.
> > >>
> >
>
Re: Any interest in running HAWQ on n Docker containers?
Posted by Jemish Patel <je...@gmail.com>.
Sounds good Roman.
Thanks.
Jemish
On Fri, Jul 8, 2016 at 10:26 AM Roman Shaposhnik <ro...@shaposhnik.org>
wrote:
> On Fri, Jul 8, 2016 at 10:18 AM, Jemish Patel <je...@gmail.com> wrote:
> > Hi bigtop devs,
> >
> > Wanted to see if there was interest and how we can work together to have
> > HAWQ run in Docker containers for the larger Hadoop community.
>
> I think there's definitely a lot of interest, but perhaps with a slightly
> bigger
> scope. I'd love to see HAWQ join Bigtop as one of the supported components.
> For that to happen, there's quite a bit of work that still needs to be
> done on
> HAWQ's side, starting with publishing a reasonable release.
>
> This is not to say that you couldn't start with the Docker container
> first and build
> it out from there. Jemish, any chance you can play with Bigtop Docker
> provisioner
> and take a look what would it take to hook HAWQ up to it?
>
> Does this sounds like a good plan?
>
> Thanks,
> Roman.
>
Re: Any interest in running HAWQ on n Docker containers?
Posted by Jemish Patel <je...@gmail.com>.
Sounds good Roman.
Thanks.
Jemish
On Fri, Jul 8, 2016 at 10:26 AM Roman Shaposhnik <ro...@shaposhnik.org>
wrote:
> On Fri, Jul 8, 2016 at 10:18 AM, Jemish Patel <je...@gmail.com> wrote:
> > Hi bigtop devs,
> >
> > Wanted to see if there was interest and how we can work together to have
> > HAWQ run in Docker containers for the larger Hadoop community.
>
> I think there's definitely a lot of interest, but perhaps with a slightly
> bigger
> scope. I'd love to see HAWQ join Bigtop as one of the supported components.
> For that to happen, there's quite a bit of work that still needs to be
> done on
> HAWQ's side, starting with publishing a reasonable release.
>
> This is not to say that you couldn't start with the Docker container
> first and build
> it out from there. Jemish, any chance you can play with Bigtop Docker
> provisioner
> and take a look what would it take to hook HAWQ up to it?
>
> Does this sounds like a good plan?
>
> Thanks,
> Roman.
>
Re: Any interest in running HAWQ on n Docker containers?
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Fri, Jul 8, 2016 at 10:18 AM, Jemish Patel <je...@gmail.com> wrote:
> Hi bigtop devs,
>
> Wanted to see if there was interest and how we can work together to have
> HAWQ run in Docker containers for the larger Hadoop community.
I think there's definitely a lot of interest, but perhaps with a slightly bigger
scope. I'd love to see HAWQ join Bigtop as one of the supported components.
For that to happen, there's quite a bit of work that still needs to be done on
HAWQ's side, starting with publishing a reasonable release.
This is not to say that you couldn't start with the Docker container
first and build
it out from there. Jemish, any chance you can play with Bigtop Docker
provisioner
and take a look what would it take to hook HAWQ up to it?
Does this sounds like a good plan?
Thanks,
Roman.
Re: Any interest in running HAWQ on n Docker containers?
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Fri, Jul 8, 2016 at 10:18 AM, Jemish Patel <je...@gmail.com> wrote:
> Hi bigtop devs,
>
> Wanted to see if there was interest and how we can work together to have
> HAWQ run in Docker containers for the larger Hadoop community.
I think there's definitely a lot of interest, but perhaps with a slightly bigger
scope. I'd love to see HAWQ join Bigtop as one of the supported components.
For that to happen, there's quite a bit of work that still needs to be done on
HAWQ's side, starting with publishing a reasonable release.
This is not to say that you couldn't start with the Docker container
first and build
it out from there. Jemish, any chance you can play with Bigtop Docker
provisioner
and take a look what would it take to hook HAWQ up to it?
Does this sounds like a good plan?
Thanks,
Roman.