Questions about configuration and deployment

[Burton Williams <bu...@vimeo.com>]

Hi -

I have recently decided to use drill and i'm in the process of configuring
it. I am not clear on if there is a central point for configuring things
such as mem etc or do i have to do it individually for all drillbits. Can
someone shed some light on this please. Do i need to configure one drillbit
then replicate it (install and copy configs and start the drillbit) on all
my data nodes? Thank you.


-B

Re: Questions about configuration and deployment

[Steven Phillips <st...@dremio.com>]

Hi Burton,

Like John said, I would strongly recommend using the same configuration for
each drillbit.

The reason memory settings are in drill-env is that we use the standard
java options to limit the amount of memory the jvm can use.
drill-override.conf contains Drill specific settings.

Let me know if you have any more questions.


On Wed, Dec 9, 2015 at 1:26 PM, John Omernik <jo...@omernik.com> wrote:

> The way I run it is using marathon on mesos, but I share my configuration
> so it's the same on every instance.
>
> Basically, you can use a shared location for the config using the --config
> flag. I use runbit but others probably have different scenarios for how
> they use it
>
> $DRILL_HOME/bin/runbit --config /path/to/shared/location/dir
>
> Now, thats if you want to have the same config (memory etc) on every bit...
> I think it's recommended to do that, but I could be wrong on that.  The
> drill-override.conf should be the same on everynode, that's how the
> drillbits know to form a cluster.  The memory settings are in drill-env
> which I believe was setup like this because drill-override needs to be the
> same, but drill-env CAN be the same, but doesn't need to be.
>
> Not sure if this helps or hurts, but it's how I think of it when I am
> running things, happy to clarify more if you have questions.
>
> John
>
>
>
> On Wed, Dec 9, 2015 at 3:16 PM, Burton Williams <bu...@vimeo.com> wrote:
>
> > Hi -
> >
> > I have recently decided to use drill and i'm in the process of
> configuring
> > it. I am not clear on if there is a central point for configuring things
> > such as mem etc or do i have to do it individually for all drillbits. Can
> > someone shed some light on this please. Do i need to configure one
> drillbit
> > then replicate it (install and copy configs and start the drillbit) on
> all
> > my data nodes? Thank you.
> >
> >
> > -B
> >
>

Re: Questions about configuration and deployment

[John Omernik <jo...@omernik.com>]

The way I run it is using marathon on mesos, but I share my configuration
so it's the same on every instance.

Basically, you can use a shared location for the config using the --config
flag. I use runbit but others probably have different scenarios for how
they use it

$DRILL_HOME/bin/runbit --config /path/to/shared/location/dir

Now, thats if you want to have the same config (memory etc) on every bit...
I think it's recommended to do that, but I could be wrong on that.  The
drill-override.conf should be the same on everynode, that's how the
drillbits know to form a cluster.  The memory settings are in drill-env
which I believe was setup like this because drill-override needs to be the
same, but drill-env CAN be the same, but doesn't need to be.

Not sure if this helps or hurts, but it's how I think of it when I am
running things, happy to clarify more if you have questions.

John



On Wed, Dec 9, 2015 at 3:16 PM, Burton Williams <bu...@vimeo.com> wrote:

> Hi -
>
> I have recently decided to use drill and i'm in the process of configuring
> it. I am not clear on if there is a central point for configuring things
> such as mem etc or do i have to do it individually for all drillbits. Can
> someone shed some light on this please. Do i need to configure one drillbit
> then replicate it (install and copy configs and start the drillbit) on all
> my data nodes? Thank you.
>
>
> -B
>