You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Alexander Rojas <al...@mesosphere.io> on 2015/11/23 14:08:39 UTC
Configuration file?
Hey guys,
Over the time I’ve been involved in Mesos I’ve seen that we went from a handful of flags to around 42 supported flags in the master. At this point I’m wondering if perhaps we should support a configuration file in conjunction (or instead of) with all the command flags.
My intuition is that it will make it easier for operators as well as for debuggers to be able to replicate configurations easier.
Any comments on this idea?
Re: Configuration file?
Posted by Weitao <zh...@gmail.com>.
not bad! I like it
> 在 2015年11月25日，00:48，Marco Massenzio <ma...@mesosphere.io> 写道：
>
> Thank you for asking :)
>
> The idea is pretty simple (and not even original, it's the "Facade
> pattern"): essentially we would define a YAML syntax for the various
> configuration flags that Mesos ({Master, Agent}) define, possibly grouping
> them by "topic" and then write a few python classes, that read in the YAML
> configuration and emit a set of flags that reflect the caller intent, then
> invoke the appropriate ./bin/mesos-{master,slave}.sh script.
>
> (variation on the theme for installed packages)
>
> E.g.:
>
> # Configuration /opt/mesos/config-master.yml
>
> ip: 192.168.1.101
>
> zk:
> url: zk://192.168.1.220:2181
> quorum: 3
>
> directories:
> work: /var/run/mesos
> log: /var/log/mesos/master.log
>
> Execute with:
>
> ./mesos-wrapper.py --run master --config /opt/mesos/config-master.yml
> --no-ssl \
> [other script-specific options]
>
> would generate something along the lines of:
>
> /opt/mesos/bin/mesos-master.sh --ip=192.168.1.101 \
> --zk=zk://192.168.1.220:2181 --quorum=3 \
> --work_dir=/var/run/mesos --log_dir=/var/log/mesos/master.log
>
> (example greatly contrived and probably pointless, but hopefully gives you
> the idea).
>
> The point is that leveraging Python and the existing YAML libraries, this
> would be pretty quick to implement, would leave Mesos per se completely
> untouched, and would be easy to maintain (add/remove flags as needed;
> enforce deprecation cycles; etc.)
>
> There's probably a ton of detail I'm missing, but you get the point.
>
> Cheers,
> M.
>
>> On Tuesday, November 24, 2015, tommy xiao <xi...@gmail.com> wrote:
>>
>> how to do that? Marco
>>
>> 2015-11-24 3:54 GMT+08:00 Marco Massenzio <marco@mesosphere.io
>> <javascript:;>>:
>>
>>> I was thinking along the same lines, with a slightly more "modern"
>> approach
>>> :)
>>>
>>> My idea was to write a thin Python layer that reads a configuration YAML
>>> file (possibly with "override" flags) and then invokes
>> mesos-{master,slave}
>>> with the appropriate flags.
>>>
>>> By using YAML, we would also gain the ability to have more intuitive
>>> syntax; grouping flags by function; and make it extensible.
>>>
>>> Any takers who may want to work together on this one?
>>>
>>> --
>>> *Marco Massenzio*
>>> Distributed Systems Engineer
>>> http://codetrips.com
>>>
>>>> On Mon, Nov 23, 2015 at 11:18 AM, Vinod Kone <vinodkone@gmail.com
>>> <javascript:;>> wrote:
>>>
>>>> We had this discussion a long while ago and the argument was that we
>>>> already have a configuration file of sorts. It is a bash script with
>> one
>>>> flag per line that sets the environment. Is this not sufficient?
>>>>
>>>> Example:
>>>>
>>>> config.sh
>>>> ------------
>>>> MESOS_WORK_DIR="/var/run/mesos"
>>>> MESOS_QUORUM=2
>>>>
>>>> $ source config.sh
>>>> $ ./bin/mesos-master.sh
>>>>
>>>>
>>>>
>>>> On Mon, Nov 23, 2015 at 10:19 AM, Jojy Varghese <jojy@mesosphere.io
>> <javascript:;>>
>>>> wrote:
>>>>
>>>>> Thanks for bringing this topic up Alex. I have been thinking about
>> the
>>>> same
>>>>> and was wondering if we can have subsystem specific flags and some
>>> scheme
>>>>> where there can be a namespace instead of a flat space. And as alex
>>>>> suggested, a configuration to represent this?
>>>>>
>>>>> -jojy
>>>>> On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <xiaods@gmail.com
>> <javascript:;>> wrote:
>>>>>
>>>>>> please file a issue to tracking this proposal
>>>>>>
>>>>>> 2015-11-23 21:53 GMT+08:00 Klaus Ma <klaus1982.cn@gmail.com
>> <javascript:;>>:
>>>>>>
>>>>>>> +1, that's helpful :).
>>>>>>>
>>>>>>> For the detail of implementing such as auto re-load, I think we
>> can
>>>> let
>>>>>>> owner/shepherd to decide :).
>>>>>>>
>>>>>>> ----
>>>>>>> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
>>>>>>> Platform Symphony/DCOS Development & Support, STG, IBM GCG
>>>>>>> +86-10-8245 4084 | klaus1982.cn@gmail.com <javascript:;> |
>> http://k82.me
>>>>>>>
>>>>>>> On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <adam@avil.la
>> <javascript:;>> wrote:
>>>>>>>
>>>>>>>> +1 I think it would be helpful.
>>>>>>>>
>>>>>>>> This may be orthogonal / feature creep, but would it be
>> possible
>>> to
>>>>>> have
>>>>>>>> the config file be able to be safely reloaded with a HUP or
>>>>> appropriate
>>>>>>>> signal?
>>>>>>>>
>>>>>>>> On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <
>> gyliu513@gmail.com <javascript:;>
>>>>
>>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1000, introducing a new configuration file for mesos master
>>> and
>>>>>> slave
>>>>>>>> can
>>>>>>>>> help end user take the configuration file as the source of
>> all
>>>>> flags.
>>>>>>>>>
>>>>>>>>> The OpenStack is also using same way to manage all of the
>>> flags,
>>>> it
>>>>>> is
>>>>>>>>> putting all flags into a configuration file and the
>>> configuration
>>>>>> file
>>>>>>>>> including all flag examples. Most of the flags are disabled
>> by
>>>>>> default
>>>>>>>> and
>>>>>>>>> the end user can just enable those flags based on his
>>>> requirement.
>>>>>>>>>
>>>>>>>>> Also the flags in the configuration file can be classified to
>>>>>> different
>>>>>>>>> groups for a better management, and mesos can also follow
>> this
>>> to
>>>>>>>> classify
>>>>>>>>> those flags to different groups, such as ACL, Cluster,
>>> framework
>>>>> etc.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
>>>>>>>> alexander@mesosphere.io <javascript:;>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hey guys,
>>>>>>>>>>
>>>>>>>>>> Over the time I’ve been involved in Mesos I’ve seen that we
>>>> went
>>>>>>> from a
>>>>>>>>>> handful of flags to around 42 supported flags in the
>> master.
>>> At
>>>>>> this
>>>>>>>>> point
>>>>>>>>>> I’m wondering if perhaps we should support a configuration
>>> file
>>>>> in
>>>>>>>>>> conjunction (or instead of) with all the command flags.
>>>>>>>>>>
>>>>>>>>>> My intuition is that it will make it easier for operators
>> as
>>>> well
>>>>>> as
>>>>>>>> for
>>>>>>>>>> debuggers to be able to replicate configurations easier.
>>>>>>>>>>
>>>>>>>>>> Any comments on this idea?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> /adam
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Deshi Xiao
>>>>>> Twitter: xds2000
>>>>>> E-mail: xiaods(AT)gmail.com
>>
>>
>>
>> --
>> Deshi Xiao
>> Twitter: xds2000
>> E-mail: xiaods(AT)gmail.com
>
>
> --
> --
> *Marco Massenzio*
> Distributed Systems Engineer
> http://codetrips.com
Re: Configuration file?
Posted by Marco Massenzio <ma...@mesosphere.io>.
ah, ok - sorry, I do use Ngnix, but a bit like a caveman :)
we (as in, Mesosphere) recommend that Mesos (master/agents) are run under
supervision (either systemd or upstart or supervisord or your favorite
supervisor) so that when the process gets killed (by accident / bug /
planned restart) the supervisor will restart it (as the case may be, with
the new configuration).
Maybe that is the best paradigm then. Use HA setup with Zookeeper and
roll config to one master, restart, wash, rinse, repeat?
yup!
look, ma', no downtime :)
--
*Marco Massenzio*
Distributed Systems Engineer
http://codetrips.com
On Tue, Nov 24, 2015 at 8:20 PM, Adam Avilla <ad...@avil.la> wrote:
> On Tue, Nov 24, 2015 at 3:15 PM, Marco Massenzio <ma...@mesosphere.io>
> wrote:
>
> > On Tue, Nov 24, 2015 at 10:45 AM, Adam Avilla <ad...@avil.la> wrote:
> >
> > > Is there any way to support HUP reloading of config with this design
> > > paradigm?
> >
> >
> > Can you please elaborate what do you mean by "support HUP reloading"?
> > I don't suppose anyone really runs Mesos in production inside a terminal?
> >
>
> I mean like how webservers and other systems work. NGINX is an example of
> the paradigm:
>
> http://nginx.org/en/docs/control.html
>
> you will see HUP spawns new workers with new config, gracefully shutting
> down old workers.
>
> Use case is config management system copies new configs to masters, HUP
> to reload, no downtime. Maybe this is already possible? If so, excuse my
> ignorance :$.
>
>
> > > Is HUPing mesos even possible / helpful? From an end user
> > > perspective it sounds useful, but I am not sure if even possible for
> > mesos
> > > to accomplish safely.
> > >
> >
> > If you are running Mesos in multi-Master configuration (using, eg,
> > Zookeeper) yes, it is safe to re-start the process, so long as you always
> > maintain a --quorum of servers running.
> > In fact, that's the recommended way to do a Mesos cluster upgrade.
> >
> > But, again, I don't really think this is what you meant by "HUPing
> Mesos"?
> > (I mean, if you `kill -1 <PID>` this seems to just terminate Master - at
> > least on OSX)
> >
>
> Maybe that is the best paradigm then. Use HA setup with Zookeeper and
> roll config to one master, restart, wash, rinse, repeat?
>
Re: Configuration file?
Posted by Adam Avilla <ad...@avil.la>.
On Tue, Nov 24, 2015 at 3:15 PM, Marco Massenzio <ma...@mesosphere.io>
wrote:
> On Tue, Nov 24, 2015 at 10:45 AM, Adam Avilla <ad...@avil.la> wrote:
>
> > Is there any way to support HUP reloading of config with this design
> > paradigm?
>
>
> Can you please elaborate what do you mean by "support HUP reloading"?
> I don't suppose anyone really runs Mesos in production inside a terminal?
>
I mean like how webservers and other systems work. NGINX is an example of
the paradigm:
http://nginx.org/en/docs/control.html
you will see HUP spawns new workers with new config, gracefully shutting
down old workers.
Use case is config management system copies new configs to masters, HUP
to reload, no downtime. Maybe this is already possible? If so, excuse my
ignorance :$.
> > Is HUPing mesos even possible / helpful? From an end user
> > perspective it sounds useful, but I am not sure if even possible for
> mesos
> > to accomplish safely.
> >
>
> If you are running Mesos in multi-Master configuration (using, eg,
> Zookeeper) yes, it is safe to re-start the process, so long as you always
> maintain a --quorum of servers running.
> In fact, that's the recommended way to do a Mesos cluster upgrade.
>
> But, again, I don't really think this is what you meant by "HUPing Mesos"?
> (I mean, if you `kill -1 <PID>` this seems to just terminate Master - at
> least on OSX)
>
Maybe that is the best paradigm then. Use HA setup with Zookeeper and
roll config to one master, restart, wash, rinse, repeat?
Re: Configuration file?
Posted by Marco Massenzio <ma...@mesosphere.io>.
On Tue, Nov 24, 2015 at 10:45 AM, Adam Avilla <ad...@avil.la> wrote:
> Is there any way to support HUP reloading of config with this design
> paradigm?
Can you please elaborate what do you mean by "support HUP reloading"?
I don't suppose anyone really runs Mesos in production inside a terminal?
At any rate, being a facade, this will support exactly what Mesos supports
(and, currently, no, it does not support dynamic configuration reload).
> Is HUPing mesos even possible / helpful? From an end user
> perspective it sounds useful, but I am not sure if even possible for mesos
> to accomplish safely.
>
If you are running Mesos in multi-Master configuration (using, eg,
Zookeeper) yes, it is safe to re-start the process, so long as you always
maintain a --quorum of servers running.
In fact, that's the recommended way to do a Mesos cluster upgrade.
But, again, I don't really think this is what you meant by "HUPing Mesos"?
(I mean, if you `kill -1 <PID>` this seems to just terminate Master - at
least on OSX)
> On Nov 24, 2015 10:04 AM, "haosdent" <ha...@gmail.com> wrote:
>
> > Use YAML sounds great. Suppose I am a framework developer, use a widely
> > used format would be helpful. Because framework maybe written by
> different
> > programming languages, if use some rare formats, I have to write some
> code
> > to parse the configuration file when I want to get some fields from it.
> >
> > On Wed, Nov 25, 2015 at 1:57 AM, Tomek Janiszewski <ja...@gmail.com>
> > wrote:
> >
> > > Hi all
> > >
> > > It looks simple for the first sight. Marco, I can work on it.
> > >
> > > -
> > > Tomek
> > >
> > > wt., 24 lis 2015, 17:49 Marco Massenzio użytkownik <
> marco@mesosphere.io>
> > > napisał:
> > >
> > > > Thank you for asking :)
> > > >
> > > > The idea is pretty simple (and not even original, it's the "Facade
> > > > pattern"): essentially we would define a YAML syntax for the various
> > > > configuration flags that Mesos ({Master, Agent}) define, possibly
> > > grouping
> > > > them by "topic" and then write a few python classes, that read in the
> > > YAML
> > > > configuration and emit a set of flags that reflect the caller intent,
> > > then
> > > > invoke the appropriate ./bin/mesos-{master,slave}.sh script.
> > > >
> > > > (variation on the theme for installed packages)
> > > >
> > > > E.g.:
> > > >
> > > > # Configuration /opt/mesos/config-master.yml
> > > >
> > > > ip: 192.168.1.101
> > > >
> > > > zk:
> > > > url: zk://192.168.1.220:2181
> > > > quorum: 3
> > > >
> > > > directories:
> > > > work: /var/run/mesos
> > > > log: /var/log/mesos/master.log
> > > >
> > > > Execute with:
> > > >
> > > > ./mesos-wrapper.py --run master --config
> /opt/mesos/config-master.yml
> > > > --no-ssl \
> > > > [other script-specific options]
> > > >
> > > > would generate something along the lines of:
> > > >
> > > > /opt/mesos/bin/mesos-master.sh --ip=192.168.1.101 \
> > > > --zk=zk://192.168.1.220:2181 --quorum=3 \
> > > > --work_dir=/var/run/mesos --log_dir=/var/log/mesos/master.log
> > > >
> > > > (example greatly contrived and probably pointless, but hopefully
> gives
> > > you
> > > > the idea).
> > > >
> > > > The point is that leveraging Python and the existing YAML libraries,
> > this
> > > > would be pretty quick to implement, would leave Mesos per se
> completely
> > > > untouched, and would be easy to maintain (add/remove flags as needed;
> > > > enforce deprecation cycles; etc.)
> > > >
> > > > There's probably a ton of detail I'm missing, but you get the point.
> > > >
> > > > Cheers,
> > > > M.
> > > >
> > > > On Tuesday, November 24, 2015, tommy xiao <xi...@gmail.com> wrote:
> > > >
> > > > > how to do that? Marco
> > > > >
> > > > > 2015-11-24 3:54 GMT+08:00 Marco Massenzio <marco@mesosphere.io
> > > > > <javascript:;>>:
> > > > >
> > > > > > I was thinking along the same lines, with a slightly more
> "modern"
> > > > > approach
> > > > > > :)
> > > > > >
> > > > > > My idea was to write a thin Python layer that reads a
> configuration
> > > > YAML
> > > > > > file (possibly with "override" flags) and then invokes
> > > > > mesos-{master,slave}
> > > > > > with the appropriate flags.
> > > > > >
> > > > > > By using YAML, we would also gain the ability to have more
> > intuitive
> > > > > > syntax; grouping flags by function; and make it extensible.
> > > > > >
> > > > > > Any takers who may want to work together on this one?
> > > > > >
> > > > > > --
> > > > > > *Marco Massenzio*
> > > > > > Distributed Systems Engineer
> > > > > > http://codetrips.com
> > > > > >
> > > > > > On Mon, Nov 23, 2015 at 11:18 AM, Vinod Kone <
> vinodkone@gmail.com
> > > > > <javascript:;>> wrote:
> > > > > >
> > > > > > > We had this discussion a long while ago and the argument was
> that
> > > we
> > > > > > > already have a configuration file of sorts. It is a bash script
> > > with
> > > > > one
> > > > > > > flag per line that sets the environment. Is this not
> sufficient?
> > > > > > >
> > > > > > > Example:
> > > > > > >
> > > > > > > config.sh
> > > > > > > ------------
> > > > > > > MESOS_WORK_DIR="/var/run/mesos"
> > > > > > > MESOS_QUORUM=2
> > > > > > >
> > > > > > > $ source config.sh
> > > > > > > $ ./bin/mesos-master.sh
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Nov 23, 2015 at 10:19 AM, Jojy Varghese <
> > > jojy@mesosphere.io
> > > > > <javascript:;>>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Thanks for bringing this topic up Alex. I have been thinking
> > > about
> > > > > the
> > > > > > > same
> > > > > > > > and was wondering if we can have subsystem specific flags and
> > > some
> > > > > > scheme
> > > > > > > > where there can be a namespace instead of a flat space. And
> as
> > > alex
> > > > > > > > suggested, a configuration to represent this?
> > > > > > > >
> > > > > > > > -jojy
> > > > > > > > On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <xiaods@gmail.com
> > > > > <javascript:;>> wrote:
> > > > > > > >
> > > > > > > > > please file a issue to tracking this proposal
> > > > > > > > >
> > > > > > > > > 2015-11-23 21:53 GMT+08:00 Klaus Ma <
> klaus1982.cn@gmail.com
> > > > > <javascript:;>>:
> > > > > > > > >
> > > > > > > > > > +1, that's helpful :).
> > > > > > > > > >
> > > > > > > > > > For the detail of implementing such as auto re-load, I
> > think
> > > we
> > > > > can
> > > > > > > let
> > > > > > > > > > owner/shepherd to decide :).
> > > > > > > > > >
> > > > > > > > > > ----
> > > > > > > > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > > > > > > > > > Platform Symphony/DCOS Development & Support, STG, IBM
> GCG
> > > > > > > > > > +86-10-8245 4084 | klaus1982.cn@gmail.com
> <javascript:;> |
> > > > > http://k82.me
> > > > > > > > > >
> > > > > > > > > > On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <
> adam@avil.la
> > > > > <javascript:;>> wrote:
> > > > > > > > > >
> > > > > > > > > > > +1 I think it would be helpful.
> > > > > > > > > > >
> > > > > > > > > > > This may be orthogonal / feature creep, but would it be
> > > > > possible
> > > > > > to
> > > > > > > > > have
> > > > > > > > > > > the config file be able to be safely reloaded with a
> HUP
> > or
> > > > > > > > appropriate
> > > > > > > > > > > signal?
> > > > > > > > > > >
> > > > > > > > > > > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <
> > > > > gyliu513@gmail.com <javascript:;>
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > +1000, introducing a new configuration file for mesos
> > > > master
> > > > > > and
> > > > > > > > > slave
> > > > > > > > > > > can
> > > > > > > > > > > > help end user take the configuration file as the
> source
> > > of
> > > > > all
> > > > > > > > flags.
> > > > > > > > > > > >
> > > > > > > > > > > > The OpenStack is also using same way to manage all of
> > the
> > > > > > flags,
> > > > > > > it
> > > > > > > > > is
> > > > > > > > > > > > putting all flags into a configuration file and the
> > > > > > configuration
> > > > > > > > > file
> > > > > > > > > > > > including all flag examples. Most of the flags are
> > > disabled
> > > > > by
> > > > > > > > > default
> > > > > > > > > > > and
> > > > > > > > > > > > the end user can just enable those flags based on his
> > > > > > > requirement.
> > > > > > > > > > > >
> > > > > > > > > > > > Also the flags in the configuration file can be
> > > classified
> > > > to
> > > > > > > > > different
> > > > > > > > > > > > groups for a better management, and mesos can also
> > follow
> > > > > this
> > > > > > to
> > > > > > > > > > > classify
> > > > > > > > > > > > those flags to different groups, such as ACL,
> Cluster,
> > > > > > framework
> > > > > > > > etc.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> > > > > > > > > > > alexander@mesosphere.io <javascript:;>>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hey guys,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Over the time I’ve been involved in Mesos I’ve seen
> > > that
> > > > we
> > > > > > > went
> > > > > > > > > > from a
> > > > > > > > > > > > > handful of flags to around 42 supported flags in
> the
> > > > > master.
> > > > > > At
> > > > > > > > > this
> > > > > > > > > > > > point
> > > > > > > > > > > > > I’m wondering if perhaps we should support a
> > > > configuration
> > > > > > file
> > > > > > > > in
> > > > > > > > > > > > > conjunction (or instead of) with all the command
> > flags.
> > > > > > > > > > > > >
> > > > > > > > > > > > > My intuition is that it will make it easier for
> > > operators
> > > > > as
> > > > > > > well
> > > > > > > > > as
> > > > > > > > > > > for
> > > > > > > > > > > > > debuggers to be able to replicate configurations
> > > easier.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Any comments on this idea?
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > /adam
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Deshi Xiao
> > > > > > > > > Twitter: xds2000
> > > > > > > > > E-mail: xiaods(AT)gmail.com
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Deshi Xiao
> > > > > Twitter: xds2000
> > > > > E-mail: xiaods(AT)gmail.com
> > > > >
> > > >
> > > >
> > > > --
> > > > --
> > > > *Marco Massenzio*
> > > > Distributed Systems Engineer
> > > > http://codetrips.com
> > > >
> > >
> >
> >
> >
> > --
> > Best Regards,
> > Haosdent Huang
> >
>
Re: Configuration file?
Posted by Adam Avilla <ad...@avil.la>.
Is there any way to support HUP reloading of config with this design
paradigm? Is HUPing mesos even possible / helpful? From an end user
perspective it sounds useful, but I am not sure if even possible for mesos
to accomplish safely.
On Nov 24, 2015 10:04 AM, "haosdent" <ha...@gmail.com> wrote:
> Use YAML sounds great. Suppose I am a framework developer, use a widely
> used format would be helpful. Because framework maybe written by different
> programming languages, if use some rare formats, I have to write some code
> to parse the configuration file when I want to get some fields from it.
>
> On Wed, Nov 25, 2015 at 1:57 AM, Tomek Janiszewski <ja...@gmail.com>
> wrote:
>
> > Hi all
> >
> > It looks simple for the first sight. Marco, I can work on it.
> >
> > -
> > Tomek
> >
> > wt., 24 lis 2015, 17:49 Marco Massenzio użytkownik <ma...@mesosphere.io>
> > napisał:
> >
> > > Thank you for asking :)
> > >
> > > The idea is pretty simple (and not even original, it's the "Facade
> > > pattern"): essentially we would define a YAML syntax for the various
> > > configuration flags that Mesos ({Master, Agent}) define, possibly
> > grouping
> > > them by "topic" and then write a few python classes, that read in the
> > YAML
> > > configuration and emit a set of flags that reflect the caller intent,
> > then
> > > invoke the appropriate ./bin/mesos-{master,slave}.sh script.
> > >
> > > (variation on the theme for installed packages)
> > >
> > > E.g.:
> > >
> > > # Configuration /opt/mesos/config-master.yml
> > >
> > > ip: 192.168.1.101
> > >
> > > zk:
> > > url: zk://192.168.1.220:2181
> > > quorum: 3
> > >
> > > directories:
> > > work: /var/run/mesos
> > > log: /var/log/mesos/master.log
> > >
> > > Execute with:
> > >
> > > ./mesos-wrapper.py --run master --config /opt/mesos/config-master.yml
> > > --no-ssl \
> > > [other script-specific options]
> > >
> > > would generate something along the lines of:
> > >
> > > /opt/mesos/bin/mesos-master.sh --ip=192.168.1.101 \
> > > --zk=zk://192.168.1.220:2181 --quorum=3 \
> > > --work_dir=/var/run/mesos --log_dir=/var/log/mesos/master.log
> > >
> > > (example greatly contrived and probably pointless, but hopefully gives
> > you
> > > the idea).
> > >
> > > The point is that leveraging Python and the existing YAML libraries,
> this
> > > would be pretty quick to implement, would leave Mesos per se completely
> > > untouched, and would be easy to maintain (add/remove flags as needed;
> > > enforce deprecation cycles; etc.)
> > >
> > > There's probably a ton of detail I'm missing, but you get the point.
> > >
> > > Cheers,
> > > M.
> > >
> > > On Tuesday, November 24, 2015, tommy xiao <xi...@gmail.com> wrote:
> > >
> > > > how to do that? Marco
> > > >
> > > > 2015-11-24 3:54 GMT+08:00 Marco Massenzio <marco@mesosphere.io
> > > > <javascript:;>>:
> > > >
> > > > > I was thinking along the same lines, with a slightly more "modern"
> > > > approach
> > > > > :)
> > > > >
> > > > > My idea was to write a thin Python layer that reads a configuration
> > > YAML
> > > > > file (possibly with "override" flags) and then invokes
> > > > mesos-{master,slave}
> > > > > with the appropriate flags.
> > > > >
> > > > > By using YAML, we would also gain the ability to have more
> intuitive
> > > > > syntax; grouping flags by function; and make it extensible.
> > > > >
> > > > > Any takers who may want to work together on this one?
> > > > >
> > > > > --
> > > > > *Marco Massenzio*
> > > > > Distributed Systems Engineer
> > > > > http://codetrips.com
> > > > >
> > > > > On Mon, Nov 23, 2015 at 11:18 AM, Vinod Kone <vinodkone@gmail.com
> > > > <javascript:;>> wrote:
> > > > >
> > > > > > We had this discussion a long while ago and the argument was that
> > we
> > > > > > already have a configuration file of sorts. It is a bash script
> > with
> > > > one
> > > > > > flag per line that sets the environment. Is this not sufficient?
> > > > > >
> > > > > > Example:
> > > > > >
> > > > > > config.sh
> > > > > > ------------
> > > > > > MESOS_WORK_DIR="/var/run/mesos"
> > > > > > MESOS_QUORUM=2
> > > > > >
> > > > > > $ source config.sh
> > > > > > $ ./bin/mesos-master.sh
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, Nov 23, 2015 at 10:19 AM, Jojy Varghese <
> > jojy@mesosphere.io
> > > > <javascript:;>>
> > > > > > wrote:
> > > > > >
> > > > > > > Thanks for bringing this topic up Alex. I have been thinking
> > about
> > > > the
> > > > > > same
> > > > > > > and was wondering if we can have subsystem specific flags and
> > some
> > > > > scheme
> > > > > > > where there can be a namespace instead of a flat space. And as
> > alex
> > > > > > > suggested, a configuration to represent this?
> > > > > > >
> > > > > > > -jojy
> > > > > > > On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <xiaods@gmail.com
> > > > <javascript:;>> wrote:
> > > > > > >
> > > > > > > > please file a issue to tracking this proposal
> > > > > > > >
> > > > > > > > 2015-11-23 21:53 GMT+08:00 Klaus Ma <klaus1982.cn@gmail.com
> > > > <javascript:;>>:
> > > > > > > >
> > > > > > > > > +1, that's helpful :).
> > > > > > > > >
> > > > > > > > > For the detail of implementing such as auto re-load, I
> think
> > we
> > > > can
> > > > > > let
> > > > > > > > > owner/shepherd to decide :).
> > > > > > > > >
> > > > > > > > > ----
> > > > > > > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > > > > > > > > Platform Symphony/DCOS Development & Support, STG, IBM GCG
> > > > > > > > > +86-10-8245 4084 | klaus1982.cn@gmail.com <javascript:;> |
> > > > http://k82.me
> > > > > > > > >
> > > > > > > > > On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <adam@avil.la
> > > > <javascript:;>> wrote:
> > > > > > > > >
> > > > > > > > > > +1 I think it would be helpful.
> > > > > > > > > >
> > > > > > > > > > This may be orthogonal / feature creep, but would it be
> > > > possible
> > > > > to
> > > > > > > > have
> > > > > > > > > > the config file be able to be safely reloaded with a HUP
> or
> > > > > > > appropriate
> > > > > > > > > > signal?
> > > > > > > > > >
> > > > > > > > > > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <
> > > > gyliu513@gmail.com <javascript:;>
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > +1000, introducing a new configuration file for mesos
> > > master
> > > > > and
> > > > > > > > slave
> > > > > > > > > > can
> > > > > > > > > > > help end user take the configuration file as the source
> > of
> > > > all
> > > > > > > flags.
> > > > > > > > > > >
> > > > > > > > > > > The OpenStack is also using same way to manage all of
> the
> > > > > flags,
> > > > > > it
> > > > > > > > is
> > > > > > > > > > > putting all flags into a configuration file and the
> > > > > configuration
> > > > > > > > file
> > > > > > > > > > > including all flag examples. Most of the flags are
> > disabled
> > > > by
> > > > > > > > default
> > > > > > > > > > and
> > > > > > > > > > > the end user can just enable those flags based on his
> > > > > > requirement.
> > > > > > > > > > >
> > > > > > > > > > > Also the flags in the configuration file can be
> > classified
> > > to
> > > > > > > > different
> > > > > > > > > > > groups for a better management, and mesos can also
> follow
> > > > this
> > > > > to
> > > > > > > > > > classify
> > > > > > > > > > > those flags to different groups, such as ACL, Cluster,
> > > > > framework
> > > > > > > etc.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> > > > > > > > > > alexander@mesosphere.io <javascript:;>>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hey guys,
> > > > > > > > > > > >
> > > > > > > > > > > > Over the time I’ve been involved in Mesos I’ve seen
> > that
> > > we
> > > > > > went
> > > > > > > > > from a
> > > > > > > > > > > > handful of flags to around 42 supported flags in the
> > > > master.
> > > > > At
> > > > > > > > this
> > > > > > > > > > > point
> > > > > > > > > > > > I’m wondering if perhaps we should support a
> > > configuration
> > > > > file
> > > > > > > in
> > > > > > > > > > > > conjunction (or instead of) with all the command
> flags.
> > > > > > > > > > > >
> > > > > > > > > > > > My intuition is that it will make it easier for
> > operators
> > > > as
> > > > > > well
> > > > > > > > as
> > > > > > > > > > for
> > > > > > > > > > > > debuggers to be able to replicate configurations
> > easier.
> > > > > > > > > > > >
> > > > > > > > > > > > Any comments on this idea?
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > /adam
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Deshi Xiao
> > > > > > > > Twitter: xds2000
> > > > > > > > E-mail: xiaods(AT)gmail.com
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Deshi Xiao
> > > > Twitter: xds2000
> > > > E-mail: xiaods(AT)gmail.com
> > > >
> > >
> > >
> > > --
> > > --
> > > *Marco Massenzio*
> > > Distributed Systems Engineer
> > > http://codetrips.com
> > >
> >
>
>
>
> --
> Best Regards,
> Haosdent Huang
>
Re: Configuration file?
Posted by haosdent <ha...@gmail.com>.
Use YAML sounds great. Suppose I am a framework developer, use a widely
used format would be helpful. Because framework maybe written by different
programming languages, if use some rare formats, I have to write some code
to parse the configuration file when I want to get some fields from it.
On Wed, Nov 25, 2015 at 1:57 AM, Tomek Janiszewski <ja...@gmail.com>
wrote:
> Hi all
>
> It looks simple for the first sight. Marco, I can work on it.
>
> -
> Tomek
>
> wt., 24 lis 2015, 17:49 Marco Massenzio użytkownik <ma...@mesosphere.io>
> napisał:
>
> > Thank you for asking :)
> >
> > The idea is pretty simple (and not even original, it's the "Facade
> > pattern"): essentially we would define a YAML syntax for the various
> > configuration flags that Mesos ({Master, Agent}) define, possibly
> grouping
> > them by "topic" and then write a few python classes, that read in the
> YAML
> > configuration and emit a set of flags that reflect the caller intent,
> then
> > invoke the appropriate ./bin/mesos-{master,slave}.sh script.
> >
> > (variation on the theme for installed packages)
> >
> > E.g.:
> >
> > # Configuration /opt/mesos/config-master.yml
> >
> > ip: 192.168.1.101
> >
> > zk:
> > url: zk://192.168.1.220:2181
> > quorum: 3
> >
> > directories:
> > work: /var/run/mesos
> > log: /var/log/mesos/master.log
> >
> > Execute with:
> >
> > ./mesos-wrapper.py --run master --config /opt/mesos/config-master.yml
> > --no-ssl \
> > [other script-specific options]
> >
> > would generate something along the lines of:
> >
> > /opt/mesos/bin/mesos-master.sh --ip=192.168.1.101 \
> > --zk=zk://192.168.1.220:2181 --quorum=3 \
> > --work_dir=/var/run/mesos --log_dir=/var/log/mesos/master.log
> >
> > (example greatly contrived and probably pointless, but hopefully gives
> you
> > the idea).
> >
> > The point is that leveraging Python and the existing YAML libraries, this
> > would be pretty quick to implement, would leave Mesos per se completely
> > untouched, and would be easy to maintain (add/remove flags as needed;
> > enforce deprecation cycles; etc.)
> >
> > There's probably a ton of detail I'm missing, but you get the point.
> >
> > Cheers,
> > M.
> >
> > On Tuesday, November 24, 2015, tommy xiao <xi...@gmail.com> wrote:
> >
> > > how to do that? Marco
> > >
> > > 2015-11-24 3:54 GMT+08:00 Marco Massenzio <marco@mesosphere.io
> > > <javascript:;>>:
> > >
> > > > I was thinking along the same lines, with a slightly more "modern"
> > > approach
> > > > :)
> > > >
> > > > My idea was to write a thin Python layer that reads a configuration
> > YAML
> > > > file (possibly with "override" flags) and then invokes
> > > mesos-{master,slave}
> > > > with the appropriate flags.
> > > >
> > > > By using YAML, we would also gain the ability to have more intuitive
> > > > syntax; grouping flags by function; and make it extensible.
> > > >
> > > > Any takers who may want to work together on this one?
> > > >
> > > > --
> > > > *Marco Massenzio*
> > > > Distributed Systems Engineer
> > > > http://codetrips.com
> > > >
> > > > On Mon, Nov 23, 2015 at 11:18 AM, Vinod Kone <vinodkone@gmail.com
> > > <javascript:;>> wrote:
> > > >
> > > > > We had this discussion a long while ago and the argument was that
> we
> > > > > already have a configuration file of sorts. It is a bash script
> with
> > > one
> > > > > flag per line that sets the environment. Is this not sufficient?
> > > > >
> > > > > Example:
> > > > >
> > > > > config.sh
> > > > > ------------
> > > > > MESOS_WORK_DIR="/var/run/mesos"
> > > > > MESOS_QUORUM=2
> > > > >
> > > > > $ source config.sh
> > > > > $ ./bin/mesos-master.sh
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Nov 23, 2015 at 10:19 AM, Jojy Varghese <
> jojy@mesosphere.io
> > > <javascript:;>>
> > > > > wrote:
> > > > >
> > > > > > Thanks for bringing this topic up Alex. I have been thinking
> about
> > > the
> > > > > same
> > > > > > and was wondering if we can have subsystem specific flags and
> some
> > > > scheme
> > > > > > where there can be a namespace instead of a flat space. And as
> alex
> > > > > > suggested, a configuration to represent this?
> > > > > >
> > > > > > -jojy
> > > > > > On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <xiaods@gmail.com
> > > <javascript:;>> wrote:
> > > > > >
> > > > > > > please file a issue to tracking this proposal
> > > > > > >
> > > > > > > 2015-11-23 21:53 GMT+08:00 Klaus Ma <klaus1982.cn@gmail.com
> > > <javascript:;>>:
> > > > > > >
> > > > > > > > +1, that's helpful :).
> > > > > > > >
> > > > > > > > For the detail of implementing such as auto re-load, I think
> we
> > > can
> > > > > let
> > > > > > > > owner/shepherd to decide :).
> > > > > > > >
> > > > > > > > ----
> > > > > > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > > > > > > > Platform Symphony/DCOS Development & Support, STG, IBM GCG
> > > > > > > > +86-10-8245 4084 | klaus1982.cn@gmail.com <javascript:;> |
> > > http://k82.me
> > > > > > > >
> > > > > > > > On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <adam@avil.la
> > > <javascript:;>> wrote:
> > > > > > > >
> > > > > > > > > +1 I think it would be helpful.
> > > > > > > > >
> > > > > > > > > This may be orthogonal / feature creep, but would it be
> > > possible
> > > > to
> > > > > > > have
> > > > > > > > > the config file be able to be safely reloaded with a HUP or
> > > > > > appropriate
> > > > > > > > > signal?
> > > > > > > > >
> > > > > > > > > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <
> > > gyliu513@gmail.com <javascript:;>
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > +1000, introducing a new configuration file for mesos
> > master
> > > > and
> > > > > > > slave
> > > > > > > > > can
> > > > > > > > > > help end user take the configuration file as the source
> of
> > > all
> > > > > > flags.
> > > > > > > > > >
> > > > > > > > > > The OpenStack is also using same way to manage all of the
> > > > flags,
> > > > > it
> > > > > > > is
> > > > > > > > > > putting all flags into a configuration file and the
> > > > configuration
> > > > > > > file
> > > > > > > > > > including all flag examples. Most of the flags are
> disabled
> > > by
> > > > > > > default
> > > > > > > > > and
> > > > > > > > > > the end user can just enable those flags based on his
> > > > > requirement.
> > > > > > > > > >
> > > > > > > > > > Also the flags in the configuration file can be
> classified
> > to
> > > > > > > different
> > > > > > > > > > groups for a better management, and mesos can also follow
> > > this
> > > > to
> > > > > > > > > classify
> > > > > > > > > > those flags to different groups, such as ACL, Cluster,
> > > > framework
> > > > > > etc.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> > > > > > > > > alexander@mesosphere.io <javascript:;>>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hey guys,
> > > > > > > > > > >
> > > > > > > > > > > Over the time I’ve been involved in Mesos I’ve seen
> that
> > we
> > > > > went
> > > > > > > > from a
> > > > > > > > > > > handful of flags to around 42 supported flags in the
> > > master.
> > > > At
> > > > > > > this
> > > > > > > > > > point
> > > > > > > > > > > I’m wondering if perhaps we should support a
> > configuration
> > > > file
> > > > > > in
> > > > > > > > > > > conjunction (or instead of) with all the command flags.
> > > > > > > > > > >
> > > > > > > > > > > My intuition is that it will make it easier for
> operators
> > > as
> > > > > well
> > > > > > > as
> > > > > > > > > for
> > > > > > > > > > > debuggers to be able to replicate configurations
> easier.
> > > > > > > > > > >
> > > > > > > > > > > Any comments on this idea?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > /adam
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Deshi Xiao
> > > > > > > Twitter: xds2000
> > > > > > > E-mail: xiaods(AT)gmail.com
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Deshi Xiao
> > > Twitter: xds2000
> > > E-mail: xiaods(AT)gmail.com
> > >
> >
> >
> > --
> > --
> > *Marco Massenzio*
> > Distributed Systems Engineer
> > http://codetrips.com
> >
>
--
Best Regards,
Haosdent Huang
Re: Configuration file?
Posted by Tomek Janiszewski <ja...@gmail.com>.
Hi all
It looks simple for the first sight. Marco, I can work on it.
-
Tomek
wt., 24 lis 2015, 17:49 Marco Massenzio użytkownik <ma...@mesosphere.io>
napisał:
> Thank you for asking :)
>
> The idea is pretty simple (and not even original, it's the "Facade
> pattern"): essentially we would define a YAML syntax for the various
> configuration flags that Mesos ({Master, Agent}) define, possibly grouping
> them by "topic" and then write a few python classes, that read in the YAML
> configuration and emit a set of flags that reflect the caller intent, then
> invoke the appropriate ./bin/mesos-{master,slave}.sh script.
>
> (variation on the theme for installed packages)
>
> E.g.:
>
> # Configuration /opt/mesos/config-master.yml
>
> ip: 192.168.1.101
>
> zk:
> url: zk://192.168.1.220:2181
> quorum: 3
>
> directories:
> work: /var/run/mesos
> log: /var/log/mesos/master.log
>
> Execute with:
>
> ./mesos-wrapper.py --run master --config /opt/mesos/config-master.yml
> --no-ssl \
> [other script-specific options]
>
> would generate something along the lines of:
>
> /opt/mesos/bin/mesos-master.sh --ip=192.168.1.101 \
> --zk=zk://192.168.1.220:2181 --quorum=3 \
> --work_dir=/var/run/mesos --log_dir=/var/log/mesos/master.log
>
> (example greatly contrived and probably pointless, but hopefully gives you
> the idea).
>
> The point is that leveraging Python and the existing YAML libraries, this
> would be pretty quick to implement, would leave Mesos per se completely
> untouched, and would be easy to maintain (add/remove flags as needed;
> enforce deprecation cycles; etc.)
>
> There's probably a ton of detail I'm missing, but you get the point.
>
> Cheers,
> M.
>
> On Tuesday, November 24, 2015, tommy xiao <xi...@gmail.com> wrote:
>
> > how to do that? Marco
> >
> > 2015-11-24 3:54 GMT+08:00 Marco Massenzio <marco@mesosphere.io
> > <javascript:;>>:
> >
> > > I was thinking along the same lines, with a slightly more "modern"
> > approach
> > > :)
> > >
> > > My idea was to write a thin Python layer that reads a configuration
> YAML
> > > file (possibly with "override" flags) and then invokes
> > mesos-{master,slave}
> > > with the appropriate flags.
> > >
> > > By using YAML, we would also gain the ability to have more intuitive
> > > syntax; grouping flags by function; and make it extensible.
> > >
> > > Any takers who may want to work together on this one?
> > >
> > > --
> > > *Marco Massenzio*
> > > Distributed Systems Engineer
> > > http://codetrips.com
> > >
> > > On Mon, Nov 23, 2015 at 11:18 AM, Vinod Kone <vinodkone@gmail.com
> > <javascript:;>> wrote:
> > >
> > > > We had this discussion a long while ago and the argument was that we
> > > > already have a configuration file of sorts. It is a bash script with
> > one
> > > > flag per line that sets the environment. Is this not sufficient?
> > > >
> > > > Example:
> > > >
> > > > config.sh
> > > > ------------
> > > > MESOS_WORK_DIR="/var/run/mesos"
> > > > MESOS_QUORUM=2
> > > >
> > > > $ source config.sh
> > > > $ ./bin/mesos-master.sh
> > > >
> > > >
> > > >
> > > > On Mon, Nov 23, 2015 at 10:19 AM, Jojy Varghese <jojy@mesosphere.io
> > <javascript:;>>
> > > > wrote:
> > > >
> > > > > Thanks for bringing this topic up Alex. I have been thinking about
> > the
> > > > same
> > > > > and was wondering if we can have subsystem specific flags and some
> > > scheme
> > > > > where there can be a namespace instead of a flat space. And as alex
> > > > > suggested, a configuration to represent this?
> > > > >
> > > > > -jojy
> > > > > On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <xiaods@gmail.com
> > <javascript:;>> wrote:
> > > > >
> > > > > > please file a issue to tracking this proposal
> > > > > >
> > > > > > 2015-11-23 21:53 GMT+08:00 Klaus Ma <klaus1982.cn@gmail.com
> > <javascript:;>>:
> > > > > >
> > > > > > > +1, that's helpful :).
> > > > > > >
> > > > > > > For the detail of implementing such as auto re-load, I think we
> > can
> > > > let
> > > > > > > owner/shepherd to decide :).
> > > > > > >
> > > > > > > ----
> > > > > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > > > > > > Platform Symphony/DCOS Development & Support, STG, IBM GCG
> > > > > > > +86-10-8245 4084 | klaus1982.cn@gmail.com <javascript:;> |
> > http://k82.me
> > > > > > >
> > > > > > > On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <adam@avil.la
> > <javascript:;>> wrote:
> > > > > > >
> > > > > > > > +1 I think it would be helpful.
> > > > > > > >
> > > > > > > > This may be orthogonal / feature creep, but would it be
> > possible
> > > to
> > > > > > have
> > > > > > > > the config file be able to be safely reloaded with a HUP or
> > > > > appropriate
> > > > > > > > signal?
> > > > > > > >
> > > > > > > > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <
> > gyliu513@gmail.com <javascript:;>
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1000, introducing a new configuration file for mesos
> master
> > > and
> > > > > > slave
> > > > > > > > can
> > > > > > > > > help end user take the configuration file as the source of
> > all
> > > > > flags.
> > > > > > > > >
> > > > > > > > > The OpenStack is also using same way to manage all of the
> > > flags,
> > > > it
> > > > > > is
> > > > > > > > > putting all flags into a configuration file and the
> > > configuration
> > > > > > file
> > > > > > > > > including all flag examples. Most of the flags are disabled
> > by
> > > > > > default
> > > > > > > > and
> > > > > > > > > the end user can just enable those flags based on his
> > > > requirement.
> > > > > > > > >
> > > > > > > > > Also the flags in the configuration file can be classified
> to
> > > > > > different
> > > > > > > > > groups for a better management, and mesos can also follow
> > this
> > > to
> > > > > > > > classify
> > > > > > > > > those flags to different groups, such as ACL, Cluster,
> > > framework
> > > > > etc.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> > > > > > > > alexander@mesosphere.io <javascript:;>>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hey guys,
> > > > > > > > > >
> > > > > > > > > > Over the time I’ve been involved in Mesos I’ve seen that
> we
> > > > went
> > > > > > > from a
> > > > > > > > > > handful of flags to around 42 supported flags in the
> > master.
> > > At
> > > > > > this
> > > > > > > > > point
> > > > > > > > > > I’m wondering if perhaps we should support a
> configuration
> > > file
> > > > > in
> > > > > > > > > > conjunction (or instead of) with all the command flags.
> > > > > > > > > >
> > > > > > > > > > My intuition is that it will make it easier for operators
> > as
> > > > well
> > > > > > as
> > > > > > > > for
> > > > > > > > > > debuggers to be able to replicate configurations easier.
> > > > > > > > > >
> > > > > > > > > > Any comments on this idea?
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > /adam
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Deshi Xiao
> > > > > > Twitter: xds2000
> > > > > > E-mail: xiaods(AT)gmail.com
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Deshi Xiao
> > Twitter: xds2000
> > E-mail: xiaods(AT)gmail.com
> >
>
>
> --
> --
> *Marco Massenzio*
> Distributed Systems Engineer
> http://codetrips.com
>
Re: Configuration file?
Posted by Marco Massenzio <ma...@mesosphere.io>.
Thank you for asking :)
The idea is pretty simple (and not even original, it's the "Facade
pattern"): essentially we would define a YAML syntax for the various
configuration flags that Mesos ({Master, Agent}) define, possibly grouping
them by "topic" and then write a few python classes, that read in the YAML
configuration and emit a set of flags that reflect the caller intent, then
invoke the appropriate ./bin/mesos-{master,slave}.sh script.
(variation on the theme for installed packages)
E.g.:
# Configuration /opt/mesos/config-master.yml
ip: 192.168.1.101
zk:
url: zk://192.168.1.220:2181
quorum: 3
directories:
work: /var/run/mesos
log: /var/log/mesos/master.log
Execute with:
./mesos-wrapper.py --run master --config /opt/mesos/config-master.yml
--no-ssl \
[other script-specific options]
would generate something along the lines of:
/opt/mesos/bin/mesos-master.sh --ip=192.168.1.101 \
--zk=zk://192.168.1.220:2181 --quorum=3 \
--work_dir=/var/run/mesos --log_dir=/var/log/mesos/master.log
(example greatly contrived and probably pointless, but hopefully gives you
the idea).
The point is that leveraging Python and the existing YAML libraries, this
would be pretty quick to implement, would leave Mesos per se completely
untouched, and would be easy to maintain (add/remove flags as needed;
enforce deprecation cycles; etc.)
There's probably a ton of detail I'm missing, but you get the point.
Cheers,
M.
On Tuesday, November 24, 2015, tommy xiao <xi...@gmail.com> wrote:
> how to do that? Marco
>
> 2015-11-24 3:54 GMT+08:00 Marco Massenzio <marco@mesosphere.io
> <javascript:;>>:
>
> > I was thinking along the same lines, with a slightly more "modern"
> approach
> > :)
> >
> > My idea was to write a thin Python layer that reads a configuration YAML
> > file (possibly with "override" flags) and then invokes
> mesos-{master,slave}
> > with the appropriate flags.
> >
> > By using YAML, we would also gain the ability to have more intuitive
> > syntax; grouping flags by function; and make it extensible.
> >
> > Any takers who may want to work together on this one?
> >
> > --
> > *Marco Massenzio*
> > Distributed Systems Engineer
> > http://codetrips.com
> >
> > On Mon, Nov 23, 2015 at 11:18 AM, Vinod Kone <vinodkone@gmail.com
> <javascript:;>> wrote:
> >
> > > We had this discussion a long while ago and the argument was that we
> > > already have a configuration file of sorts. It is a bash script with
> one
> > > flag per line that sets the environment. Is this not sufficient?
> > >
> > > Example:
> > >
> > > config.sh
> > > ------------
> > > MESOS_WORK_DIR="/var/run/mesos"
> > > MESOS_QUORUM=2
> > >
> > > $ source config.sh
> > > $ ./bin/mesos-master.sh
> > >
> > >
> > >
> > > On Mon, Nov 23, 2015 at 10:19 AM, Jojy Varghese <jojy@mesosphere.io
> <javascript:;>>
> > > wrote:
> > >
> > > > Thanks for bringing this topic up Alex. I have been thinking about
> the
> > > same
> > > > and was wondering if we can have subsystem specific flags and some
> > scheme
> > > > where there can be a namespace instead of a flat space. And as alex
> > > > suggested, a configuration to represent this?
> > > >
> > > > -jojy
> > > > On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <xiaods@gmail.com
> <javascript:;>> wrote:
> > > >
> > > > > please file a issue to tracking this proposal
> > > > >
> > > > > 2015-11-23 21:53 GMT+08:00 Klaus Ma <klaus1982.cn@gmail.com
> <javascript:;>>:
> > > > >
> > > > > > +1, that's helpful :).
> > > > > >
> > > > > > For the detail of implementing such as auto re-load, I think we
> can
> > > let
> > > > > > owner/shepherd to decide :).
> > > > > >
> > > > > > ----
> > > > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > > > > > Platform Symphony/DCOS Development & Support, STG, IBM GCG
> > > > > > +86-10-8245 4084 | klaus1982.cn@gmail.com <javascript:;> |
> http://k82.me
> > > > > >
> > > > > > On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <adam@avil.la
> <javascript:;>> wrote:
> > > > > >
> > > > > > > +1 I think it would be helpful.
> > > > > > >
> > > > > > > This may be orthogonal / feature creep, but would it be
> possible
> > to
> > > > > have
> > > > > > > the config file be able to be safely reloaded with a HUP or
> > > > appropriate
> > > > > > > signal?
> > > > > > >
> > > > > > > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <
> gyliu513@gmail.com <javascript:;>
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > +1000, introducing a new configuration file for mesos master
> > and
> > > > > slave
> > > > > > > can
> > > > > > > > help end user take the configuration file as the source of
> all
> > > > flags.
> > > > > > > >
> > > > > > > > The OpenStack is also using same way to manage all of the
> > flags,
> > > it
> > > > > is
> > > > > > > > putting all flags into a configuration file and the
> > configuration
> > > > > file
> > > > > > > > including all flag examples. Most of the flags are disabled
> by
> > > > > default
> > > > > > > and
> > > > > > > > the end user can just enable those flags based on his
> > > requirement.
> > > > > > > >
> > > > > > > > Also the flags in the configuration file can be classified to
> > > > > different
> > > > > > > > groups for a better management, and mesos can also follow
> this
> > to
> > > > > > > classify
> > > > > > > > those flags to different groups, such as ACL, Cluster,
> > framework
> > > > etc.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> > > > > > > alexander@mesosphere.io <javascript:;>>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hey guys,
> > > > > > > > >
> > > > > > > > > Over the time I’ve been involved in Mesos I’ve seen that we
> > > went
> > > > > > from a
> > > > > > > > > handful of flags to around 42 supported flags in the
> master.
> > At
> > > > > this
> > > > > > > > point
> > > > > > > > > I’m wondering if perhaps we should support a configuration
> > file
> > > > in
> > > > > > > > > conjunction (or instead of) with all the command flags.
> > > > > > > > >
> > > > > > > > > My intuition is that it will make it easier for operators
> as
> > > well
> > > > > as
> > > > > > > for
> > > > > > > > > debuggers to be able to replicate configurations easier.
> > > > > > > > >
> > > > > > > > > Any comments on this idea?
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > /adam
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Deshi Xiao
> > > > > Twitter: xds2000
> > > > > E-mail: xiaods(AT)gmail.com
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com
>
--
--
*Marco Massenzio*
Distributed Systems Engineer
http://codetrips.com
Re: Configuration file?
Posted by tommy xiao <xi...@gmail.com>.
how to do that? Marco
2015-11-24 3:54 GMT+08:00 Marco Massenzio <ma...@mesosphere.io>:
> I was thinking along the same lines, with a slightly more "modern" approach
> :)
>
> My idea was to write a thin Python layer that reads a configuration YAML
> file (possibly with "override" flags) and then invokes mesos-{master,slave}
> with the appropriate flags.
>
> By using YAML, we would also gain the ability to have more intuitive
> syntax; grouping flags by function; and make it extensible.
>
> Any takers who may want to work together on this one?
>
> --
> *Marco Massenzio*
> Distributed Systems Engineer
> http://codetrips.com
>
> On Mon, Nov 23, 2015 at 11:18 AM, Vinod Kone <vi...@gmail.com> wrote:
>
> > We had this discussion a long while ago and the argument was that we
> > already have a configuration file of sorts. It is a bash script with one
> > flag per line that sets the environment. Is this not sufficient?
> >
> > Example:
> >
> > config.sh
> > ------------
> > MESOS_WORK_DIR="/var/run/mesos"
> > MESOS_QUORUM=2
> >
> > $ source config.sh
> > $ ./bin/mesos-master.sh
> >
> >
> >
> > On Mon, Nov 23, 2015 at 10:19 AM, Jojy Varghese <jo...@mesosphere.io>
> > wrote:
> >
> > > Thanks for bringing this topic up Alex. I have been thinking about the
> > same
> > > and was wondering if we can have subsystem specific flags and some
> scheme
> > > where there can be a namespace instead of a flat space. And as alex
> > > suggested, a configuration to represent this?
> > >
> > > -jojy
> > > On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <xi...@gmail.com> wrote:
> > >
> > > > please file a issue to tracking this proposal
> > > >
> > > > 2015-11-23 21:53 GMT+08:00 Klaus Ma <kl...@gmail.com>:
> > > >
> > > > > +1, that's helpful :).
> > > > >
> > > > > For the detail of implementing such as auto re-load, I think we can
> > let
> > > > > owner/shepherd to decide :).
> > > > >
> > > > > ----
> > > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > > > > Platform Symphony/DCOS Development & Support, STG, IBM GCG
> > > > > +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
> > > > >
> > > > > On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <ad...@avil.la> wrote:
> > > > >
> > > > > > +1 I think it would be helpful.
> > > > > >
> > > > > > This may be orthogonal / feature creep, but would it be possible
> to
> > > > have
> > > > > > the config file be able to be safely reloaded with a HUP or
> > > appropriate
> > > > > > signal?
> > > > > >
> > > > > > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <gyliu513@gmail.com
> >
> > > > wrote:
> > > > > >
> > > > > > > +1000, introducing a new configuration file for mesos master
> and
> > > > slave
> > > > > > can
> > > > > > > help end user take the configuration file as the source of all
> > > flags.
> > > > > > >
> > > > > > > The OpenStack is also using same way to manage all of the
> flags,
> > it
> > > > is
> > > > > > > putting all flags into a configuration file and the
> configuration
> > > > file
> > > > > > > including all flag examples. Most of the flags are disabled by
> > > > default
> > > > > > and
> > > > > > > the end user can just enable those flags based on his
> > requirement.
> > > > > > >
> > > > > > > Also the flags in the configuration file can be classified to
> > > > different
> > > > > > > groups for a better management, and mesos can also follow this
> to
> > > > > > classify
> > > > > > > those flags to different groups, such as ACL, Cluster,
> framework
> > > etc.
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> > > > > > alexander@mesosphere.io>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hey guys,
> > > > > > > >
> > > > > > > > Over the time I’ve been involved in Mesos I’ve seen that we
> > went
> > > > > from a
> > > > > > > > handful of flags to around 42 supported flags in the master.
> At
> > > > this
> > > > > > > point
> > > > > > > > I’m wondering if perhaps we should support a configuration
> file
> > > in
> > > > > > > > conjunction (or instead of) with all the command flags.
> > > > > > > >
> > > > > > > > My intuition is that it will make it easier for operators as
> > well
> > > > as
> > > > > > for
> > > > > > > > debuggers to be able to replicate configurations easier.
> > > > > > > >
> > > > > > > > Any comments on this idea?
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > /adam
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Deshi Xiao
> > > > Twitter: xds2000
> > > > E-mail: xiaods(AT)gmail.com
> > > >
> > >
> >
>
--
Deshi Xiao
Twitter: xds2000
E-mail: xiaods(AT)gmail.com
Re: Configuration file?
Posted by Marco Massenzio <ma...@mesosphere.io>.
I was thinking along the same lines, with a slightly more "modern" approach
:)
My idea was to write a thin Python layer that reads a configuration YAML
file (possibly with "override" flags) and then invokes mesos-{master,slave}
with the appropriate flags.
By using YAML, we would also gain the ability to have more intuitive
syntax; grouping flags by function; and make it extensible.
Any takers who may want to work together on this one?
--
*Marco Massenzio*
Distributed Systems Engineer
http://codetrips.com
On Mon, Nov 23, 2015 at 11:18 AM, Vinod Kone <vi...@gmail.com> wrote:
> We had this discussion a long while ago and the argument was that we
> already have a configuration file of sorts. It is a bash script with one
> flag per line that sets the environment. Is this not sufficient?
>
> Example:
>
> config.sh
> ------------
> MESOS_WORK_DIR="/var/run/mesos"
> MESOS_QUORUM=2
>
> $ source config.sh
> $ ./bin/mesos-master.sh
>
>
>
> On Mon, Nov 23, 2015 at 10:19 AM, Jojy Varghese <jo...@mesosphere.io>
> wrote:
>
> > Thanks for bringing this topic up Alex. I have been thinking about the
> same
> > and was wondering if we can have subsystem specific flags and some scheme
> > where there can be a namespace instead of a flat space. And as alex
> > suggested, a configuration to represent this?
> >
> > -jojy
> > On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <xi...@gmail.com> wrote:
> >
> > > please file a issue to tracking this proposal
> > >
> > > 2015-11-23 21:53 GMT+08:00 Klaus Ma <kl...@gmail.com>:
> > >
> > > > +1, that's helpful :).
> > > >
> > > > For the detail of implementing such as auto re-load, I think we can
> let
> > > > owner/shepherd to decide :).
> > > >
> > > > ----
> > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > > > Platform Symphony/DCOS Development & Support, STG, IBM GCG
> > > > +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
> > > >
> > > > On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <ad...@avil.la> wrote:
> > > >
> > > > > +1 I think it would be helpful.
> > > > >
> > > > > This may be orthogonal / feature creep, but would it be possible to
> > > have
> > > > > the config file be able to be safely reloaded with a HUP or
> > appropriate
> > > > > signal?
> > > > >
> > > > > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <gy...@gmail.com>
> > > wrote:
> > > > >
> > > > > > +1000, introducing a new configuration file for mesos master and
> > > slave
> > > > > can
> > > > > > help end user take the configuration file as the source of all
> > flags.
> > > > > >
> > > > > > The OpenStack is also using same way to manage all of the flags,
> it
> > > is
> > > > > > putting all flags into a configuration file and the configuration
> > > file
> > > > > > including all flag examples. Most of the flags are disabled by
> > > default
> > > > > and
> > > > > > the end user can just enable those flags based on his
> requirement.
> > > > > >
> > > > > > Also the flags in the configuration file can be classified to
> > > different
> > > > > > groups for a better management, and mesos can also follow this to
> > > > > classify
> > > > > > those flags to different groups, such as ACL, Cluster, framework
> > etc.
> > > > > >
> > > > > >
> > > > > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> > > > > alexander@mesosphere.io>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey guys,
> > > > > > >
> > > > > > > Over the time I’ve been involved in Mesos I’ve seen that we
> went
> > > > from a
> > > > > > > handful of flags to around 42 supported flags in the master. At
> > > this
> > > > > > point
> > > > > > > I’m wondering if perhaps we should support a configuration file
> > in
> > > > > > > conjunction (or instead of) with all the command flags.
> > > > > > >
> > > > > > > My intuition is that it will make it easier for operators as
> well
> > > as
> > > > > for
> > > > > > > debuggers to be able to replicate configurations easier.
> > > > > > >
> > > > > > > Any comments on this idea?
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > /adam
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Deshi Xiao
> > > Twitter: xds2000
> > > E-mail: xiaods(AT)gmail.com
> > >
> >
>
Re: Configuration file?
Posted by Vinod Kone <vi...@gmail.com>.
We had this discussion a long while ago and the argument was that we
already have a configuration file of sorts. It is a bash script with one
flag per line that sets the environment. Is this not sufficient?
Example:
config.sh
------------
MESOS_WORK_DIR="/var/run/mesos"
MESOS_QUORUM=2
$ source config.sh
$ ./bin/mesos-master.sh
On Mon, Nov 23, 2015 at 10:19 AM, Jojy Varghese <jo...@mesosphere.io> wrote:
> Thanks for bringing this topic up Alex. I have been thinking about the same
> and was wondering if we can have subsystem specific flags and some scheme
> where there can be a namespace instead of a flat space. And as alex
> suggested, a configuration to represent this?
>
> -jojy
> On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <xi...@gmail.com> wrote:
>
> > please file a issue to tracking this proposal
> >
> > 2015-11-23 21:53 GMT+08:00 Klaus Ma <kl...@gmail.com>:
> >
> > > +1, that's helpful :).
> > >
> > > For the detail of implementing such as auto re-load, I think we can let
> > > owner/shepherd to decide :).
> > >
> > > ----
> > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > > Platform Symphony/DCOS Development & Support, STG, IBM GCG
> > > +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
> > >
> > > On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <ad...@avil.la> wrote:
> > >
> > > > +1 I think it would be helpful.
> > > >
> > > > This may be orthogonal / feature creep, but would it be possible to
> > have
> > > > the config file be able to be safely reloaded with a HUP or
> appropriate
> > > > signal?
> > > >
> > > > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <gy...@gmail.com>
> > wrote:
> > > >
> > > > > +1000, introducing a new configuration file for mesos master and
> > slave
> > > > can
> > > > > help end user take the configuration file as the source of all
> flags.
> > > > >
> > > > > The OpenStack is also using same way to manage all of the flags, it
> > is
> > > > > putting all flags into a configuration file and the configuration
> > file
> > > > > including all flag examples. Most of the flags are disabled by
> > default
> > > > and
> > > > > the end user can just enable those flags based on his requirement.
> > > > >
> > > > > Also the flags in the configuration file can be classified to
> > different
> > > > > groups for a better management, and mesos can also follow this to
> > > > classify
> > > > > those flags to different groups, such as ACL, Cluster, framework
> etc.
> > > > >
> > > > >
> > > > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> > > > alexander@mesosphere.io>
> > > > > wrote:
> > > > >
> > > > > > Hey guys,
> > > > > >
> > > > > > Over the time I’ve been involved in Mesos I’ve seen that we went
> > > from a
> > > > > > handful of flags to around 42 supported flags in the master. At
> > this
> > > > > point
> > > > > > I’m wondering if perhaps we should support a configuration file
> in
> > > > > > conjunction (or instead of) with all the command flags.
> > > > > >
> > > > > > My intuition is that it will make it easier for operators as well
> > as
> > > > for
> > > > > > debuggers to be able to replicate configurations easier.
> > > > > >
> > > > > > Any comments on this idea?
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > /adam
> > > >
> > >
> >
> >
> >
> > --
> > Deshi Xiao
> > Twitter: xds2000
> > E-mail: xiaods(AT)gmail.com
> >
>
Re: Configuration file?
Posted by Jojy Varghese <jo...@mesosphere.io>.
Thanks for bringing this topic up Alex. I have been thinking about the same
and was wondering if we can have subsystem specific flags and some scheme
where there can be a namespace instead of a flat space. And as alex
suggested, a configuration to represent this?
-jojy
On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <xi...@gmail.com> wrote:
> please file a issue to tracking this proposal
>
> 2015-11-23 21:53 GMT+08:00 Klaus Ma <kl...@gmail.com>:
>
> > +1, that's helpful :).
> >
> > For the detail of implementing such as auto re-load, I think we can let
> > owner/shepherd to decide :).
> >
> > ----
> > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > Platform Symphony/DCOS Development & Support, STG, IBM GCG
> > +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
> >
> > On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <ad...@avil.la> wrote:
> >
> > > +1 I think it would be helpful.
> > >
> > > This may be orthogonal / feature creep, but would it be possible to
> have
> > > the config file be able to be safely reloaded with a HUP or appropriate
> > > signal?
> > >
> > > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <gy...@gmail.com>
> wrote:
> > >
> > > > +1000, introducing a new configuration file for mesos master and
> slave
> > > can
> > > > help end user take the configuration file as the source of all flags.
> > > >
> > > > The OpenStack is also using same way to manage all of the flags, it
> is
> > > > putting all flags into a configuration file and the configuration
> file
> > > > including all flag examples. Most of the flags are disabled by
> default
> > > and
> > > > the end user can just enable those flags based on his requirement.
> > > >
> > > > Also the flags in the configuration file can be classified to
> different
> > > > groups for a better management, and mesos can also follow this to
> > > classify
> > > > those flags to different groups, such as ACL, Cluster, framework etc.
> > > >
> > > >
> > > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> > > alexander@mesosphere.io>
> > > > wrote:
> > > >
> > > > > Hey guys,
> > > > >
> > > > > Over the time I’ve been involved in Mesos I’ve seen that we went
> > from a
> > > > > handful of flags to around 42 supported flags in the master. At
> this
> > > > point
> > > > > I’m wondering if perhaps we should support a configuration file in
> > > > > conjunction (or instead of) with all the command flags.
> > > > >
> > > > > My intuition is that it will make it easier for operators as well
> as
> > > for
> > > > > debuggers to be able to replicate configurations easier.
> > > > >
> > > > > Any comments on this idea?
> > > >
> > >
> > >
> > >
> > > --
> > > /adam
> > >
> >
>
>
>
> --
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com
>
Re: Configuration file?
Posted by tommy xiao <xi...@gmail.com>.
please file a issue to tracking this proposal
2015-11-23 21:53 GMT+08:00 Klaus Ma <kl...@gmail.com>:
> +1, that's helpful :).
>
> For the detail of implementing such as auto re-load, I think we can let
> owner/shepherd to decide :).
>
> ----
> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> Platform Symphony/DCOS Development & Support, STG, IBM GCG
> +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
>
> On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <ad...@avil.la> wrote:
>
> > +1 I think it would be helpful.
> >
> > This may be orthogonal / feature creep, but would it be possible to have
> > the config file be able to be safely reloaded with a HUP or appropriate
> > signal?
> >
> > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <gy...@gmail.com> wrote:
> >
> > > +1000, introducing a new configuration file for mesos master and slave
> > can
> > > help end user take the configuration file as the source of all flags.
> > >
> > > The OpenStack is also using same way to manage all of the flags, it is
> > > putting all flags into a configuration file and the configuration file
> > > including all flag examples. Most of the flags are disabled by default
> > and
> > > the end user can just enable those flags based on his requirement.
> > >
> > > Also the flags in the configuration file can be classified to different
> > > groups for a better management, and mesos can also follow this to
> > classify
> > > those flags to different groups, such as ACL, Cluster, framework etc.
> > >
> > >
> > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> > alexander@mesosphere.io>
> > > wrote:
> > >
> > > > Hey guys,
> > > >
> > > > Over the time I’ve been involved in Mesos I’ve seen that we went
> from a
> > > > handful of flags to around 42 supported flags in the master. At this
> > > point
> > > > I’m wondering if perhaps we should support a configuration file in
> > > > conjunction (or instead of) with all the command flags.
> > > >
> > > > My intuition is that it will make it easier for operators as well as
> > for
> > > > debuggers to be able to replicate configurations easier.
> > > >
> > > > Any comments on this idea?
> > >
> >
> >
> >
> > --
> > /adam
> >
>
--
Deshi Xiao
Twitter: xds2000
E-mail: xiaods(AT)gmail.com
Re: Configuration file?
Posted by Klaus Ma <kl...@gmail.com>.
+1, that's helpful :).
For the detail of implementing such as auto re-load, I think we can let
owner/shepherd to decide :).
----
Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
Platform Symphony/DCOS Development & Support, STG, IBM GCG
+86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <ad...@avil.la> wrote:
> +1 I think it would be helpful.
>
> This may be orthogonal / feature creep, but would it be possible to have
> the config file be able to be safely reloaded with a HUP or appropriate
> signal?
>
> On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <gy...@gmail.com> wrote:
>
> > +1000, introducing a new configuration file for mesos master and slave
> can
> > help end user take the configuration file as the source of all flags.
> >
> > The OpenStack is also using same way to manage all of the flags, it is
> > putting all flags into a configuration file and the configuration file
> > including all flag examples. Most of the flags are disabled by default
> and
> > the end user can just enable those flags based on his requirement.
> >
> > Also the flags in the configuration file can be classified to different
> > groups for a better management, and mesos can also follow this to
> classify
> > those flags to different groups, such as ACL, Cluster, framework etc.
> >
> >
> > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> alexander@mesosphere.io>
> > wrote:
> >
> > > Hey guys,
> > >
> > > Over the time I’ve been involved in Mesos I’ve seen that we went from a
> > > handful of flags to around 42 supported flags in the master. At this
> > point
> > > I’m wondering if perhaps we should support a configuration file in
> > > conjunction (or instead of) with all the command flags.
> > >
> > > My intuition is that it will make it easier for operators as well as
> for
> > > debuggers to be able to replicate configurations easier.
> > >
> > > Any comments on this idea?
> >
>
>
>
> --
> /adam
>
Re: Configuration file?
Posted by Adam Avilla <ad...@avil.la>.
+1 I think it would be helpful.
This may be orthogonal / feature creep, but would it be possible to have
the config file be able to be safely reloaded with a HUP or appropriate
signal?
On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <gy...@gmail.com> wrote:
> +1000, introducing a new configuration file for mesos master and slave can
> help end user take the configuration file as the source of all flags.
>
> The OpenStack is also using same way to manage all of the flags, it is
> putting all flags into a configuration file and the configuration file
> including all flag examples. Most of the flags are disabled by default and
> the end user can just enable those flags based on his requirement.
>
> Also the flags in the configuration file can be classified to different
> groups for a better management, and mesos can also follow this to classify
> those flags to different groups, such as ACL, Cluster, framework etc.
>
>
> On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <al...@mesosphere.io>
> wrote:
>
> > Hey guys,
> >
> > Over the time I’ve been involved in Mesos I’ve seen that we went from a
> > handful of flags to around 42 supported flags in the master. At this
> point
> > I’m wondering if perhaps we should support a configuration file in
> > conjunction (or instead of) with all the command flags.
> >
> > My intuition is that it will make it easier for operators as well as for
> > debuggers to be able to replicate configurations easier.
> >
> > Any comments on this idea?
>
--
/adam
Re: Configuration file?
Posted by Guangya Liu <gy...@gmail.com>.
+1000, introducing a new configuration file for mesos master and slave can
help end user take the configuration file as the source of all flags.
The OpenStack is also using same way to manage all of the flags, it is
putting all flags into a configuration file and the configuration file
including all flag examples. Most of the flags are disabled by default and
the end user can just enable those flags based on his requirement.
Also the flags in the configuration file can be classified to different
groups for a better management, and mesos can also follow this to classify
those flags to different groups, such as ACL, Cluster, framework etc.
On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <al...@mesosphere.io>
wrote:
> Hey guys,
>
> Over the time I’ve been involved in Mesos I’ve seen that we went from a
> handful of flags to around 42 supported flags in the master. At this point
> I’m wondering if perhaps we should support a configuration file in
> conjunction (or instead of) with all the command flags.
>
> My intuition is that it will make it easier for operators as well as for
> debuggers to be able to replicate configurations easier.
>
> Any comments on this idea?