dynamic configuration loading

[Ted Yu <yu...@gmail.com>]

>From Stack on HBASE-3866:

Seeing it makes me want to work on making Configuration settable in a live
server, sort of like the log level servlet so that we can set
hbase.balancer.max.regions.periteration one way for one minute and then come
along later and change it.

Currently master classes receive Configuration upon construction.
I think we should enable dynamic loading of Configuration at runtime. This
would be very helpful for developers and users. I think with this feature,
master would be more robust.

Please comment.

Re: dynamic configuration loading

[Andrew Purtell <ap...@apache.org>]

On the jira, there was a -1 to this idea by Todd followed quickly by another by Ted. I don't have a strong feeling to mount a defense of the idea, so let's forget it.


--- On Fri, 5/20/11, Stack <st...@duboce.net> wrote:

> From: Stack <st...@duboce.net>
> Subject: Re: dynamic configuration loading
> To: dev@hbase.apache.org, apurtell@apache.org
> Date: Friday, May 20, 2011, 9:26 PM
> On Fri, May 20, 2011 at 8:03 PM,
> Andrew Purtell <ap...@apache.org>
> wrote:
> > Given that Hadoop does not require ZooKeeper, but we
> do anyway, I wonder if it makes more sense to go our own
> route and host all of configuration in the ZooKeeper
> namespace.
> >
> >
> 
> This could work.  On configuration update, we could
> write the
> difference to the filesystem on each
> server.   We could write each
> individual diff as its own file.  We could change the
> Configuration to
> pick up hbase-site.xml + hbase-default.xml plus any edits
> made since
> last restart (edits could be written to conf or to an edits
> dir?)
> 
> Or, move Configuration into zk altogether.
> 
> St.Ack
> 

Re: dynamic configuration loading

[Stack <st...@duboce.net>]

On Fri, May 20, 2011 at 8:03 PM, Andrew Purtell <ap...@apache.org> wrote:
> Given that Hadoop does not require ZooKeeper, but we do anyway, I wonder if it makes more sense to go our own route and host all of configuration in the ZooKeeper namespace.
>
>

This could work.  On configuration update, we could write the
difference to the filesystem on each server.   We could write each
individual diff as its own file.  We could change the Configuration to
pick up hbase-site.xml + hbase-default.xml plus any edits made since
last restart (edits could be written to conf or to an edits dir?)

Or, move Configuration into zk altogether.

St.Ack

Re: dynamic configuration loading

[Andrew Purtell <ap...@apache.org>]

Given that Hadoop does not require ZooKeeper, but we do anyway, I wonder if it makes more sense to go our own route and host all of configuration in the ZooKeeper namespace.


--- On Fri, 5/20/11, Stack <st...@duboce.net> wrote:

> From: Stack <st...@duboce.net>
> Subject: Re: dynamic configuration loading
> To: dev@hbase.apache.org
> Date: Friday, May 20, 2011, 1:03 PM
> On Fri, May 20, 2011 at 12:15 PM,
> Todd Lipcon <to...@cloudera.com>
> wrote:
> > Hi Ted,
> >
> > You should check out HADOOP-7001 - it would be nice if
> we followed a
> > similar pattern in HBase.
> >
> Agree.  I made hbase-3909.
> St.Ack
> 

Re: dynamic configuration loading

[Stack <st...@duboce.net>]

On Fri, May 20, 2011 at 12:15 PM, Todd Lipcon <to...@cloudera.com> wrote:
> Hi Ted,
>
> You should check out HADOOP-7001 - it would be nice if we followed a
> similar pattern in HBase.
>
Agree.  I made hbase-3909.
St.Ack

Re: dynamic configuration loading

[Todd Lipcon <to...@cloudera.com>]

Hi Ted,

You should check out HADOOP-7001 - it would be nice if we followed a
similar pattern in HBase.

On Fri, May 20, 2011 at 11:38 AM, Ted Yu <yu...@gmail.com> wrote:
> From Stack on HBASE-3866:
>
> Seeing it makes me want to work on making Configuration settable in a live
> server, sort of like the log level servlet so that we can set
> hbase.balancer.max.regions.periteration one way for one minute and then come
> along later and change it.
>
> Currently master classes receive Configuration upon construction.
> I think we should enable dynamic loading of Configuration at runtime. This
> would be very helpful for developers and users. I think with this feature,
> master would be more robust.
>
> Please comment.
>



-- 
Todd Lipcon
Software Engineer, Cloudera