You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Scott Kinney <sc...@stem.com> on 2016/07/22 20:20:14 UTC
deploy dockerized drill cluster
I have built a drill docker images very much like https://github.com/bigstepinc/apache-drill/blob/master/Dockerfile
I volume mount a drill-override.conf file that looks like:
drill.exec:{
cluster-id: drill1,
zk.connect: 192.1.1.1:2181",
sys.store.provider.local.path: "/drill-storage",
store.json.all_text_mode: True,
store.json.read_numbers_as_double: True
}
it seems to be connecting to the zookeeper (otherwise it would fail right? i should actually confirm this).
I know it is picking up my s3 plugin that i volume mount inside /drill-storage but it's not setting the json all_text_mode and read_numbers_as_double.
I have a drill-embeded instance that seems to pick these json setting from the drill-override file just fine.
Can you see what I'm doing wrong?
________________________________
Scott Kinney | DevOps
stem <http://www.stem.com/> | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended recipient(s). Any review, use or distribution that has not been expressly authorized by Stem, Inc. is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. Thank you.
Re: deploy dockerized drill cluster
Posted by John Omernik <jo...@omernik.com>.
So I think part of the problem here is the options on json are stored in
the zookeeper. Are you running the Zookeepers in the docker containers, in
separate docker containers, or completely separate? For those settings,
I'd boot strap the cluster by itself, get it up and running, and then use
the ALTER SYSTEM set command, either from SQLLINE or using the Rest API.
Then they will persist int he Zookeeper information.
I know I asked a similar question, because I wanted new clusters to be able
to be setup to use specified admin users. (I.e. I wanted to provide my list
of users that were admins prior to the first drill bit starting, and there
wasn't an easy way to do that at the time I asked).
On Sun, Jul 31, 2016 at 8:07 PM, Jesse Yates <je...@gmail.com>
wrote:
> Ran into this myself today, so gonna try and close the loop.
>
> Looking at the 1.6 codeline (assuming its the same in 1.7), on startup the
> SystemOptionManager on pulls out the sys.options line from the config and
> uses that to build the PersistentStoreConfig (which get initialized from
> the store in #init()). However, this obviously doesn't take into account
> the config level options, only the startup options since those get pulled
> out to create everything else.
>
> It seems simple enough to wrap the SystemOptionManager into a
> FallbackOptionManager that falls back to the config file, if its not in the
> persistent store (or something like that).
>
> Unless someone already filed a JIRA on this?
>
> On Mon, Jul 25, 2016 at 10:05 AM Scott Kinney <sc...@stem.com>
> wrote:
>
> > it's still not picking up the store.json* config changes
> >
> > The only way I can see to set these is with running ALTER SYSTEM query
> > after drill api is up.
> >
> >
> > ________________________________
> > Scott Kinney | DevOps
> > stem | m 510.282.1299
> > 100 Rollins Road, Millbrae, California 94030
> >
> > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > proprietary information and material for the sole use of the intended
> > recipient(s). Any review, use or distribution that has not been expressly
> > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > intended recipient, please contact the sender and delete all copies.
> Thank
> > you.
> >
> > ________________________________________
> > From: John Omernik <jo...@omernik.com>
> > Sent: Monday, July 25, 2016 8:21 AM
> > To: user
> > Subject: Re: deploy dockerized drill cluster
> >
> > Try (for the sake of the conversation here) using host networking, and
> see
> > if it changes how successful your setup is. (I know bridged is
> preferred,
> > but try the host side and see what happens)
> >
> > John
> >
> > On Mon, Jul 25, 2016 at 10:06 AM, Scott Kinney <sc...@stem.com>
> > wrote:
> >
> > > I'm running the docker in bridged network mode.
> > >
> > >
> > > ________________________________
> > > Scott Kinney | DevOps
> > > stem | m 510.282.1299
> > > 100 Rollins Road, Millbrae, California 94030
> > >
> > > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > > proprietary information and material for the sole use of the intended
> > > recipient(s). Any review, use or distribution that has not been
> expressly
> > > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > > intended recipient, please contact the sender and delete all copies.
> > Thank
> > > you.
> > >
> > > ________________________________________
> > > From: John Omernik <jo...@omernik.com>
> > > Sent: Sunday, July 24, 2016 8:28 AM
> > > To: user
> > > Subject: Re: deploy dockerized drill cluster
> > >
> > > Are you running Drill in host networking or bridged networking?
> > >
> > > On Sat, Jul 23, 2016 at 1:21 PM, Scott Kinney <sc...@stem.com>
> > > wrote:
> > >
> > > > Hm, i must have set those another way in embeded mode. I can't see
> > where.
> > > > Those settings persist between drill restarts.
> > > >
> > > >
> > > >
> > > > ________________________________
> > > > Scott Kinney | DevOps
> > > > stem | m 510.282.1299
> > > > 100 Rollins Road, Millbrae, California 94030
> > > >
> > > > This e-mail and/or any attachments contain Stem, Inc. confidential
> and
> > > > proprietary information and material for the sole use of the intended
> > > > recipient(s). Any review, use or distribution that has not been
> > expressly
> > > > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > > > intended recipient, please contact the sender and delete all copies.
> > > Thank
> > > > you.
> > > >
> > > > ________________________________________
> > > > From: Abhishek Girish <ab...@gmail.com>
> > > > Sent: Friday, July 22, 2016 1:57 PM
> > > > To: Drill User List
> > > > Subject: Re: deploy dockerized drill cluster
> > > >
> > > > You can set boot level start-up options in drill-override.conf [1].
> > But I
> > > > don't think we can do the same with the system options. Someone else
> > can
> > > > comment if there is a workaround.
> > > >
> > > > On why it works for you with drill-embedded, is something I'm trying
> to
> > > > understand. I attempted this and couldn't manage to get those options
> > to
> > > > show up in embedded mode.
> > > >
> > > > [1] https://drill.apache.org/docs/start-up-options/
> > > >
> > > > On Fri, Jul 22, 2016 at 1:20 PM, Scott Kinney <scott.kinney@stem.com
> >
> > > > wrote:
> > > >
> > > > > I have built a drill docker images very much like
> > > > > https://github.com/bigstepinc/apache-drill/blob/master/Dockerfile
> > > > >
> > > > >
> > > > > I volume mount a drill-override.conf file that looks like:
> > > > >
> > > > > drill.exec:{
> > > > > cluster-id: drill1,
> > > > > zk.connect: 192.1.1.1:2181",
> > > > > sys.store.provider.local.path: "/drill-storage",
> > > > > store.json.all_text_mode: True,
> > > > > store.json.read_numbers_as_double: True
> > > > > }
> > > > >
> > > > > it seems to be connecting to the zookeeper (otherwise it would fail
> > > > right?
> > > > > i should actually confirm this).
> > > > >
> > > > > I know it is picking up my s3 plugin that i volume mount inside
> > > > > /drill-storage but it's not setting the json all_text_mode and
> > > > > read_numbers_as_double.
> > > > >
> > > > > I have a drill-embeded instance that seems to pick these json
> setting
> > > > from
> > > > > the drill-override file just fine.
> > > > >
> > > > > Can you see what I'm doing wrong?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ________________________________
> > > > > Scott Kinney | DevOps
> > > > > stem <http://www.stem.com/> | m 510.282.1299
> > > > > 100 Rollins Road, Millbrae, California 94030
> > > > >
> > > > > This e-mail and/or any attachments contain Stem, Inc. confidential
> > and
> > > > > proprietary information and material for the sole use of the
> intended
> > > > > recipient(s). Any review, use or distribution that has not been
> > > expressly
> > > > > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > > > > intended recipient, please contact the sender and delete all
> copies.
> > > > Thank
> > > > > you.
> > > > >
> > > >
> > >
> >
> --
> Jesse Yates
> Founder/CEO Fineo.io
>
Re: deploy dockerized drill cluster
Posted by Jesse Yates <je...@gmail.com>.
Ran into this myself today, so gonna try and close the loop.
Looking at the 1.6 codeline (assuming its the same in 1.7), on startup the
SystemOptionManager on pulls out the sys.options line from the config and
uses that to build the PersistentStoreConfig (which get initialized from
the store in #init()). However, this obviously doesn't take into account
the config level options, only the startup options since those get pulled
out to create everything else.
It seems simple enough to wrap the SystemOptionManager into a
FallbackOptionManager that falls back to the config file, if its not in the
persistent store (or something like that).
Unless someone already filed a JIRA on this?
On Mon, Jul 25, 2016 at 10:05 AM Scott Kinney <sc...@stem.com> wrote:
> it's still not picking up the store.json* config changes
>
> The only way I can see to set these is with running ALTER SYSTEM query
> after drill api is up.
>
>
> ________________________________
> Scott Kinney | DevOps
> stem | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>
> ________________________________________
> From: John Omernik <jo...@omernik.com>
> Sent: Monday, July 25, 2016 8:21 AM
> To: user
> Subject: Re: deploy dockerized drill cluster
>
> Try (for the sake of the conversation here) using host networking, and see
> if it changes how successful your setup is. (I know bridged is preferred,
> but try the host side and see what happens)
>
> John
>
> On Mon, Jul 25, 2016 at 10:06 AM, Scott Kinney <sc...@stem.com>
> wrote:
>
> > I'm running the docker in bridged network mode.
> >
> >
> > ________________________________
> > Scott Kinney | DevOps
> > stem | m 510.282.1299
> > 100 Rollins Road, Millbrae, California 94030
> >
> > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > proprietary information and material for the sole use of the intended
> > recipient(s). Any review, use or distribution that has not been expressly
> > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > intended recipient, please contact the sender and delete all copies.
> Thank
> > you.
> >
> > ________________________________________
> > From: John Omernik <jo...@omernik.com>
> > Sent: Sunday, July 24, 2016 8:28 AM
> > To: user
> > Subject: Re: deploy dockerized drill cluster
> >
> > Are you running Drill in host networking or bridged networking?
> >
> > On Sat, Jul 23, 2016 at 1:21 PM, Scott Kinney <sc...@stem.com>
> > wrote:
> >
> > > Hm, i must have set those another way in embeded mode. I can't see
> where.
> > > Those settings persist between drill restarts.
> > >
> > >
> > >
> > > ________________________________
> > > Scott Kinney | DevOps
> > > stem | m 510.282.1299
> > > 100 Rollins Road, Millbrae, California 94030
> > >
> > > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > > proprietary information and material for the sole use of the intended
> > > recipient(s). Any review, use or distribution that has not been
> expressly
> > > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > > intended recipient, please contact the sender and delete all copies.
> > Thank
> > > you.
> > >
> > > ________________________________________
> > > From: Abhishek Girish <ab...@gmail.com>
> > > Sent: Friday, July 22, 2016 1:57 PM
> > > To: Drill User List
> > > Subject: Re: deploy dockerized drill cluster
> > >
> > > You can set boot level start-up options in drill-override.conf [1].
> But I
> > > don't think we can do the same with the system options. Someone else
> can
> > > comment if there is a workaround.
> > >
> > > On why it works for you with drill-embedded, is something I'm trying to
> > > understand. I attempted this and couldn't manage to get those options
> to
> > > show up in embedded mode.
> > >
> > > [1] https://drill.apache.org/docs/start-up-options/
> > >
> > > On Fri, Jul 22, 2016 at 1:20 PM, Scott Kinney <sc...@stem.com>
> > > wrote:
> > >
> > > > I have built a drill docker images very much like
> > > > https://github.com/bigstepinc/apache-drill/blob/master/Dockerfile
> > > >
> > > >
> > > > I volume mount a drill-override.conf file that looks like:
> > > >
> > > > drill.exec:{
> > > > cluster-id: drill1,
> > > > zk.connect: 192.1.1.1:2181",
> > > > sys.store.provider.local.path: "/drill-storage",
> > > > store.json.all_text_mode: True,
> > > > store.json.read_numbers_as_double: True
> > > > }
> > > >
> > > > it seems to be connecting to the zookeeper (otherwise it would fail
> > > right?
> > > > i should actually confirm this).
> > > >
> > > > I know it is picking up my s3 plugin that i volume mount inside
> > > > /drill-storage but it's not setting the json all_text_mode and
> > > > read_numbers_as_double.
> > > >
> > > > I have a drill-embeded instance that seems to pick these json setting
> > > from
> > > > the drill-override file just fine.
> > > >
> > > > Can you see what I'm doing wrong?
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > ________________________________
> > > > Scott Kinney | DevOps
> > > > stem <http://www.stem.com/> | m 510.282.1299
> > > > 100 Rollins Road, Millbrae, California 94030
> > > >
> > > > This e-mail and/or any attachments contain Stem, Inc. confidential
> and
> > > > proprietary information and material for the sole use of the intended
> > > > recipient(s). Any review, use or distribution that has not been
> > expressly
> > > > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > > > intended recipient, please contact the sender and delete all copies.
> > > Thank
> > > > you.
> > > >
> > >
> >
>
--
Jesse Yates
Founder/CEO Fineo.io
Re: deploy dockerized drill cluster
Posted by Scott Kinney <sc...@stem.com>.
it's still not picking up the store.json* config changes
The only way I can see to set these is with running ALTER SYSTEM query after drill api is up.
________________________________
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended recipient(s). Any review, use or distribution that has not been expressly authorized by Stem, Inc. is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. Thank you.
________________________________________
From: John Omernik <jo...@omernik.com>
Sent: Monday, July 25, 2016 8:21 AM
To: user
Subject: Re: deploy dockerized drill cluster
Try (for the sake of the conversation here) using host networking, and see
if it changes how successful your setup is. (I know bridged is preferred,
but try the host side and see what happens)
John
On Mon, Jul 25, 2016 at 10:06 AM, Scott Kinney <sc...@stem.com>
wrote:
> I'm running the docker in bridged network mode.
>
>
> ________________________________
> Scott Kinney | DevOps
> stem | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>
> ________________________________________
> From: John Omernik <jo...@omernik.com>
> Sent: Sunday, July 24, 2016 8:28 AM
> To: user
> Subject: Re: deploy dockerized drill cluster
>
> Are you running Drill in host networking or bridged networking?
>
> On Sat, Jul 23, 2016 at 1:21 PM, Scott Kinney <sc...@stem.com>
> wrote:
>
> > Hm, i must have set those another way in embeded mode. I can't see where.
> > Those settings persist between drill restarts.
> >
> >
> >
> > ________________________________
> > Scott Kinney | DevOps
> > stem | m 510.282.1299
> > 100 Rollins Road, Millbrae, California 94030
> >
> > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > proprietary information and material for the sole use of the intended
> > recipient(s). Any review, use or distribution that has not been expressly
> > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > intended recipient, please contact the sender and delete all copies.
> Thank
> > you.
> >
> > ________________________________________
> > From: Abhishek Girish <ab...@gmail.com>
> > Sent: Friday, July 22, 2016 1:57 PM
> > To: Drill User List
> > Subject: Re: deploy dockerized drill cluster
> >
> > You can set boot level start-up options in drill-override.conf [1]. But I
> > don't think we can do the same with the system options. Someone else can
> > comment if there is a workaround.
> >
> > On why it works for you with drill-embedded, is something I'm trying to
> > understand. I attempted this and couldn't manage to get those options to
> > show up in embedded mode.
> >
> > [1] https://drill.apache.org/docs/start-up-options/
> >
> > On Fri, Jul 22, 2016 at 1:20 PM, Scott Kinney <sc...@stem.com>
> > wrote:
> >
> > > I have built a drill docker images very much like
> > > https://github.com/bigstepinc/apache-drill/blob/master/Dockerfile
> > >
> > >
> > > I volume mount a drill-override.conf file that looks like:
> > >
> > > drill.exec:{
> > > cluster-id: drill1,
> > > zk.connect: 192.1.1.1:2181",
> > > sys.store.provider.local.path: "/drill-storage",
> > > store.json.all_text_mode: True,
> > > store.json.read_numbers_as_double: True
> > > }
> > >
> > > it seems to be connecting to the zookeeper (otherwise it would fail
> > right?
> > > i should actually confirm this).
> > >
> > > I know it is picking up my s3 plugin that i volume mount inside
> > > /drill-storage but it's not setting the json all_text_mode and
> > > read_numbers_as_double.
> > >
> > > I have a drill-embeded instance that seems to pick these json setting
> > from
> > > the drill-override file just fine.
> > >
> > > Can you see what I'm doing wrong?
> > >
> > >
> > >
> > >
> > >
> > > ________________________________
> > > Scott Kinney | DevOps
> > > stem <http://www.stem.com/> | m 510.282.1299
> > > 100 Rollins Road, Millbrae, California 94030
> > >
> > > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > > proprietary information and material for the sole use of the intended
> > > recipient(s). Any review, use or distribution that has not been
> expressly
> > > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > > intended recipient, please contact the sender and delete all copies.
> > Thank
> > > you.
> > >
> >
>
Re: deploy dockerized drill cluster
Posted by John Omernik <jo...@omernik.com>.
Try (for the sake of the conversation here) using host networking, and see
if it changes how successful your setup is. (I know bridged is preferred,
but try the host side and see what happens)
John
On Mon, Jul 25, 2016 at 10:06 AM, Scott Kinney <sc...@stem.com>
wrote:
> I'm running the docker in bridged network mode.
>
>
> ________________________________
> Scott Kinney | DevOps
> stem | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>
> ________________________________________
> From: John Omernik <jo...@omernik.com>
> Sent: Sunday, July 24, 2016 8:28 AM
> To: user
> Subject: Re: deploy dockerized drill cluster
>
> Are you running Drill in host networking or bridged networking?
>
> On Sat, Jul 23, 2016 at 1:21 PM, Scott Kinney <sc...@stem.com>
> wrote:
>
> > Hm, i must have set those another way in embeded mode. I can't see where.
> > Those settings persist between drill restarts.
> >
> >
> >
> > ________________________________
> > Scott Kinney | DevOps
> > stem | m 510.282.1299
> > 100 Rollins Road, Millbrae, California 94030
> >
> > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > proprietary information and material for the sole use of the intended
> > recipient(s). Any review, use or distribution that has not been expressly
> > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > intended recipient, please contact the sender and delete all copies.
> Thank
> > you.
> >
> > ________________________________________
> > From: Abhishek Girish <ab...@gmail.com>
> > Sent: Friday, July 22, 2016 1:57 PM
> > To: Drill User List
> > Subject: Re: deploy dockerized drill cluster
> >
> > You can set boot level start-up options in drill-override.conf [1]. But I
> > don't think we can do the same with the system options. Someone else can
> > comment if there is a workaround.
> >
> > On why it works for you with drill-embedded, is something I'm trying to
> > understand. I attempted this and couldn't manage to get those options to
> > show up in embedded mode.
> >
> > [1] https://drill.apache.org/docs/start-up-options/
> >
> > On Fri, Jul 22, 2016 at 1:20 PM, Scott Kinney <sc...@stem.com>
> > wrote:
> >
> > > I have built a drill docker images very much like
> > > https://github.com/bigstepinc/apache-drill/blob/master/Dockerfile
> > >
> > >
> > > I volume mount a drill-override.conf file that looks like:
> > >
> > > drill.exec:{
> > > cluster-id: drill1,
> > > zk.connect: 192.1.1.1:2181",
> > > sys.store.provider.local.path: "/drill-storage",
> > > store.json.all_text_mode: True,
> > > store.json.read_numbers_as_double: True
> > > }
> > >
> > > it seems to be connecting to the zookeeper (otherwise it would fail
> > right?
> > > i should actually confirm this).
> > >
> > > I know it is picking up my s3 plugin that i volume mount inside
> > > /drill-storage but it's not setting the json all_text_mode and
> > > read_numbers_as_double.
> > >
> > > I have a drill-embeded instance that seems to pick these json setting
> > from
> > > the drill-override file just fine.
> > >
> > > Can you see what I'm doing wrong?
> > >
> > >
> > >
> > >
> > >
> > > ________________________________
> > > Scott Kinney | DevOps
> > > stem <http://www.stem.com/> | m 510.282.1299
> > > 100 Rollins Road, Millbrae, California 94030
> > >
> > > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > > proprietary information and material for the sole use of the intended
> > > recipient(s). Any review, use or distribution that has not been
> expressly
> > > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > > intended recipient, please contact the sender and delete all copies.
> > Thank
> > > you.
> > >
> >
>
Re: deploy dockerized drill cluster
Posted by Scott Kinney <sc...@stem.com>.
I'm running the docker in bridged network mode.
________________________________
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended recipient(s). Any review, use or distribution that has not been expressly authorized by Stem, Inc. is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. Thank you.
________________________________________
From: John Omernik <jo...@omernik.com>
Sent: Sunday, July 24, 2016 8:28 AM
To: user
Subject: Re: deploy dockerized drill cluster
Are you running Drill in host networking or bridged networking?
On Sat, Jul 23, 2016 at 1:21 PM, Scott Kinney <sc...@stem.com> wrote:
> Hm, i must have set those another way in embeded mode. I can't see where.
> Those settings persist between drill restarts.
>
>
>
> ________________________________
> Scott Kinney | DevOps
> stem | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>
> ________________________________________
> From: Abhishek Girish <ab...@gmail.com>
> Sent: Friday, July 22, 2016 1:57 PM
> To: Drill User List
> Subject: Re: deploy dockerized drill cluster
>
> You can set boot level start-up options in drill-override.conf [1]. But I
> don't think we can do the same with the system options. Someone else can
> comment if there is a workaround.
>
> On why it works for you with drill-embedded, is something I'm trying to
> understand. I attempted this and couldn't manage to get those options to
> show up in embedded mode.
>
> [1] https://drill.apache.org/docs/start-up-options/
>
> On Fri, Jul 22, 2016 at 1:20 PM, Scott Kinney <sc...@stem.com>
> wrote:
>
> > I have built a drill docker images very much like
> > https://github.com/bigstepinc/apache-drill/blob/master/Dockerfile
> >
> >
> > I volume mount a drill-override.conf file that looks like:
> >
> > drill.exec:{
> > cluster-id: drill1,
> > zk.connect: 192.1.1.1:2181",
> > sys.store.provider.local.path: "/drill-storage",
> > store.json.all_text_mode: True,
> > store.json.read_numbers_as_double: True
> > }
> >
> > it seems to be connecting to the zookeeper (otherwise it would fail
> right?
> > i should actually confirm this).
> >
> > I know it is picking up my s3 plugin that i volume mount inside
> > /drill-storage but it's not setting the json all_text_mode and
> > read_numbers_as_double.
> >
> > I have a drill-embeded instance that seems to pick these json setting
> from
> > the drill-override file just fine.
> >
> > Can you see what I'm doing wrong?
> >
> >
> >
> >
> >
> > ________________________________
> > Scott Kinney | DevOps
> > stem <http://www.stem.com/> | m 510.282.1299
> > 100 Rollins Road, Millbrae, California 94030
> >
> > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > proprietary information and material for the sole use of the intended
> > recipient(s). Any review, use or distribution that has not been expressly
> > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > intended recipient, please contact the sender and delete all copies.
> Thank
> > you.
> >
>
Re: deploy dockerized drill cluster
Posted by John Omernik <jo...@omernik.com>.
Are you running Drill in host networking or bridged networking?
On Sat, Jul 23, 2016 at 1:21 PM, Scott Kinney <sc...@stem.com> wrote:
> Hm, i must have set those another way in embeded mode. I can't see where.
> Those settings persist between drill restarts.
>
>
>
> ________________________________
> Scott Kinney | DevOps
> stem | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>
> ________________________________________
> From: Abhishek Girish <ab...@gmail.com>
> Sent: Friday, July 22, 2016 1:57 PM
> To: Drill User List
> Subject: Re: deploy dockerized drill cluster
>
> You can set boot level start-up options in drill-override.conf [1]. But I
> don't think we can do the same with the system options. Someone else can
> comment if there is a workaround.
>
> On why it works for you with drill-embedded, is something I'm trying to
> understand. I attempted this and couldn't manage to get those options to
> show up in embedded mode.
>
> [1] https://drill.apache.org/docs/start-up-options/
>
> On Fri, Jul 22, 2016 at 1:20 PM, Scott Kinney <sc...@stem.com>
> wrote:
>
> > I have built a drill docker images very much like
> > https://github.com/bigstepinc/apache-drill/blob/master/Dockerfile
> >
> >
> > I volume mount a drill-override.conf file that looks like:
> >
> > drill.exec:{
> > cluster-id: drill1,
> > zk.connect: 192.1.1.1:2181",
> > sys.store.provider.local.path: "/drill-storage",
> > store.json.all_text_mode: True,
> > store.json.read_numbers_as_double: True
> > }
> >
> > it seems to be connecting to the zookeeper (otherwise it would fail
> right?
> > i should actually confirm this).
> >
> > I know it is picking up my s3 plugin that i volume mount inside
> > /drill-storage but it's not setting the json all_text_mode and
> > read_numbers_as_double.
> >
> > I have a drill-embeded instance that seems to pick these json setting
> from
> > the drill-override file just fine.
> >
> > Can you see what I'm doing wrong?
> >
> >
> >
> >
> >
> > ________________________________
> > Scott Kinney | DevOps
> > stem <http://www.stem.com/> | m 510.282.1299
> > 100 Rollins Road, Millbrae, California 94030
> >
> > This e-mail and/or any attachments contain Stem, Inc. confidential and
> > proprietary information and material for the sole use of the intended
> > recipient(s). Any review, use or distribution that has not been expressly
> > authorized by Stem, Inc. is strictly prohibited. If you are not the
> > intended recipient, please contact the sender and delete all copies.
> Thank
> > you.
> >
>
Re: deploy dockerized drill cluster
Posted by Scott Kinney <sc...@stem.com>.
Hm, i must have set those another way in embeded mode. I can't see where. Those settings persist between drill restarts.
________________________________
Scott Kinney | DevOps
stem | m 510.282.1299
100 Rollins Road, Millbrae, California 94030
This e-mail and/or any attachments contain Stem, Inc. confidential and proprietary information and material for the sole use of the intended recipient(s). Any review, use or distribution that has not been expressly authorized by Stem, Inc. is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. Thank you.
________________________________________
From: Abhishek Girish <ab...@gmail.com>
Sent: Friday, July 22, 2016 1:57 PM
To: Drill User List
Subject: Re: deploy dockerized drill cluster
You can set boot level start-up options in drill-override.conf [1]. But I
don't think we can do the same with the system options. Someone else can
comment if there is a workaround.
On why it works for you with drill-embedded, is something I'm trying to
understand. I attempted this and couldn't manage to get those options to
show up in embedded mode.
[1] https://drill.apache.org/docs/start-up-options/
On Fri, Jul 22, 2016 at 1:20 PM, Scott Kinney <sc...@stem.com> wrote:
> I have built a drill docker images very much like
> https://github.com/bigstepinc/apache-drill/blob/master/Dockerfile
>
>
> I volume mount a drill-override.conf file that looks like:
>
> drill.exec:{
> cluster-id: drill1,
> zk.connect: 192.1.1.1:2181",
> sys.store.provider.local.path: "/drill-storage",
> store.json.all_text_mode: True,
> store.json.read_numbers_as_double: True
> }
>
> it seems to be connecting to the zookeeper (otherwise it would fail right?
> i should actually confirm this).
>
> I know it is picking up my s3 plugin that i volume mount inside
> /drill-storage but it's not setting the json all_text_mode and
> read_numbers_as_double.
>
> I have a drill-embeded instance that seems to pick these json setting from
> the drill-override file just fine.
>
> Can you see what I'm doing wrong?
>
>
>
>
>
> ________________________________
> Scott Kinney | DevOps
> stem <http://www.stem.com/> | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>
Re: deploy dockerized drill cluster
Posted by Abhishek Girish <ab...@gmail.com>.
You can set boot level start-up options in drill-override.conf [1]. But I
don't think we can do the same with the system options. Someone else can
comment if there is a workaround.
On why it works for you with drill-embedded, is something I'm trying to
understand. I attempted this and couldn't manage to get those options to
show up in embedded mode.
[1] https://drill.apache.org/docs/start-up-options/
On Fri, Jul 22, 2016 at 1:20 PM, Scott Kinney <sc...@stem.com> wrote:
> I have built a drill docker images very much like
> https://github.com/bigstepinc/apache-drill/blob/master/Dockerfile
>
>
> I volume mount a drill-override.conf file that looks like:
>
> drill.exec:{
> cluster-id: drill1,
> zk.connect: 192.1.1.1:2181",
> sys.store.provider.local.path: "/drill-storage",
> store.json.all_text_mode: True,
> store.json.read_numbers_as_double: True
> }
>
> it seems to be connecting to the zookeeper (otherwise it would fail right?
> i should actually confirm this).
>
> I know it is picking up my s3 plugin that i volume mount inside
> /drill-storage but it's not setting the json all_text_mode and
> read_numbers_as_double.
>
> I have a drill-embeded instance that seems to pick these json setting from
> the drill-override file just fine.
>
> Can you see what I'm doing wrong?
>
>
>
>
>
> ________________________________
> Scott Kinney | DevOps
> stem <http://www.stem.com/> | m 510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>