You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Benjamin Hindman <be...@berkeley.edu> on 2012/03/26 23:38:18 UTC
Review Request: Updated deploy scripts (see description).
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/
-----------------------------------------------------------
Review request for mesos, Andy Konwinski and Matei Zaharia.
Summary
-------
Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
Diffs
-----
README 8166b04
configure.ac 7a0d89f
src/Makefile.am 090c07a
src/conf/deploy-env.sh.template a4565e2
src/configurator/configurator.hpp ecd1365
src/deploy/deploy-to-slaves 0e10486
src/deploy/mesos-common-env.sh.in PRE-CREATION
src/deploy/mesos-daemon b3144c0
src/deploy/mesos-daemon.sh.in PRE-CREATION
src/deploy/mesos-deploy-env.sh.in PRE-CREATION
src/deploy/mesos-env.sh c3dfda2
src/deploy/mesos-masters-env.sh.in PRE-CREATION
src/deploy/mesos-slaves-env.sh.in PRE-CREATION
src/deploy/mesos-start-masters.sh.in PRE-CREATION
src/deploy/mesos-start-slaves.sh.in PRE-CREATION
src/deploy/mesos-start.sh.in PRE-CREATION
src/deploy/mesos-stop-masters.sh.in PRE-CREATION
src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
src/deploy/mesos-stop.sh.in PRE-CREATION
src/deploy/start-masters a450ee0
src/deploy/start-mesos 9e5b922
src/deploy/start-slaves b3b931f
src/deploy/stop-masters f2aa9e2
src/deploy/stop-mesos 679a09e
src/deploy/stop-slaves 141acd4
src/master/main.cpp f31212b
src/tests/master_detector_tests.cpp PRE-CREATION
Diff: https://reviews.apache.org/r/4491/diff
Testing
-------
N/A
Thanks,
Benjamin
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6373
-----------------------------------------------------------
src/deploy/mesos-start-slaves.sh.in
<https://reviews.apache.org/r/4491/#comment13844>
Added '.'.
src/deploy/mesos-start-slaves.sh.in
<https://reviews.apache.org/r/4491/#comment13843>
Fixed.
- Benjamin
On 2012-03-26 21:38:18, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-26 21:38:18)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.in PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.in PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.in PRE-CREATION
> src/deploy/mesos-slaves-env.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-start.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
> src/tests/master_detector_tests.cpp PRE-CREATION
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Andy Konwinski <an...@berkeley.edu>.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > README, line 151
> > <https://reviews.apache.org/r/4491/diff/1/?file=95961#file95961line151>
> >
> > Why is this not called start-mesos anymore? If it's to keep it in line with the rest of the binaries we put on the PATH, I would go for something like mesos-start-cluster.sh, which seems more grammatically correct than just "start".
I like this idea.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > README, line 141
> > <https://reviews.apache.org/r/4491/diff/1/?file=95961#file95961line141>
> >
> > But the slaves-env.sh file needs to be on all the nodes, right? If the intent is to ship the options automatically, that would make it hard for the user to configure, say, different numbers of CPUs and memory per host, or a different LIBPROCESS_IP, or stuff like that.
I chatted with Matei offline about this. The current flow does not ship the slaves-env.sh file over. If people wanted to set up unique per-machine slave configurations, they can create a config file somewhere on each such machine containing the special settings for that machine, and then set their slaves-env.sh to point to that via the --conf=/location/of/mesos/conf/file/on/slave/machines, and the values set in those files would override anything passed over via the deploy scripts as env variables.
This raises the important issue of what the order of precedence is for options that are set in multipole of {env variable, configuration file, command line} (note that this is not a new issue). https://github.com/mesos/mesos/wiki/Configuration currently says "Configuration values are searched for first on the command line, then in the environment, and then in the config file." but we should expand that comment to say explicitly what happens if a variable is set in more than one place (maybe even give an example).
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > src/deploy/mesos-start-masters.sh.in, line 36
> > <https://reviews.apache.org/r/4491/diff/1/?file=95974#file95974line36>
> >
> > What does this do? If you're trying to ship the env variables from the local config files to the machine, shouldn't you do export ${var} = value? Also, gotta be careful with quoting here.
I believe the `var` command prints one "KEY=value" pair per line so the way Ben has it should work. Perhaps renaming the bash variable here from var to var_and_value or var_value_pair would make it more clear.
Also, I think you're right Matei that the way we're passing the environment variables right now might break if any of those environment variables contain double quotes (e.g. <code>export MESOS_CONFIG_VAR="say \"hi\""</code>). One thing we could do is just state in the documentation (on the wiki, in the README, and in the *-env.sh files themselves) that users should not include any strings with any quotes (i.e. \") in them.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > src/Makefile.am, line 40
> > <https://reviews.apache.org/r/4491/diff/1/?file=95963#file95963line40>
> >
> > Just to double-check, if someone does a make install, then modifies some config files, and then downloads a new version of Mesos and does make install again, will their directories be overwritten? Or does install protect the existing files?
This is a great point, and we should be very careful about this. I'm not sure what the exact process for re-installing mesos is actually, but I think it is simply to run make install again after building a newer version of Mesos. In that case, perhaps we should only install *-env.sh.template files instead of *.-env.sh files.
- Andy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6383
-----------------------------------------------------------
On 2012-03-26 21:38:18, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-26 21:38:18)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.in PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.in PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.in PRE-CREATION
> src/deploy/mesos-slaves-env.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-start.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
> src/tests/master_detector_tests.cpp PRE-CREATION
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > README, line 141
> > <https://reviews.apache.org/r/4491/diff/1/?file=95961#file95961line141>
> >
> > But the slaves-env.sh file needs to be on all the nodes, right? If the intent is to ship the options automatically, that would make it hard for the user to configure, say, different numbers of CPUs and memory per host, or a different LIBPROCESS_IP, or stuff like that.
>
> Andy Konwinski wrote:
> I chatted with Matei offline about this. The current flow does not ship the slaves-env.sh file over. If people wanted to set up unique per-machine slave configurations, they can create a config file somewhere on each such machine containing the special settings for that machine, and then set their slaves-env.sh to point to that via the --conf=/location/of/mesos/conf/file/on/slave/machines, and the values set in those files would override anything passed over via the deploy scripts as env variables.
>
> This raises the important issue of what the order of precedence is for options that are set in multipole of {env variable, configuration file, command line} (note that this is not a new issue). https://github.com/mesos/mesos/wiki/Configuration currently says "Configuration values are searched for first on the command line, then in the environment, and then in the config file." but we should expand that comment to say explicitly what happens if a variable is set in more than one place (maybe even give an example).
>
> Matei Zaharia wrote:
> Just to make it clear, the goal of the deploy scripts was never to ship the configuration automatically to all the machines. The old deploy-to-slaves script happened to do that, but its main purpose is to let people developing Mesos modify and recompile the code locally, then sync it to all the machines. I think it's fine to tell the user to configure each machine separately -- in fact they won't have to do a single thing by default because most users don't set any config settings on slaves. Otherwise, the mechanisms for "global" and "local" configuration should be consistent (both environment variables or both files).
By the way, my suggestion with this kind of stuff is that if you don't want to spend a long time thinking about how to do it, find another project that uses a similar configuration process, and copy what it does. Otherwise, you will repeat mistakes that others have found and fixed in the past. This is why the old scripts were based on per-machine config files in a key-value pair format -- because a lot of other cluster software does that. Same thing with the localstatedir / .template file issue. If in doubt go with .template and tell the user to copy it.
- Matei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6383
-----------------------------------------------------------
On 2012-03-26 21:38:18, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-26 21:38:18)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.in PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.in PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.in PRE-CREATION
> src/deploy/mesos-slaves-env.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-start.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
> src/tests/master_detector_tests.cpp PRE-CREATION
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Andy Konwinski <an...@berkeley.edu>.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > README, line 130
> > <https://reviews.apache.org/r/4491/diff/1/?file=95961#file95961line130>
> >
> > Mention that the scripts also need Mesos to be installed to the same prefix on all machines.
Looks like this was done too (by Ben).
- Andy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6383
-----------------------------------------------------------
On 2012-03-27 04:38:04, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 04:38:04)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.template PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.template PRE-CREATION
> src/deploy/mesos-slaves-env.sh.template PRE-CREATION
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > README, line 141
> > <https://reviews.apache.org/r/4491/diff/1/?file=95961#file95961line141>
> >
> > But the slaves-env.sh file needs to be on all the nodes, right? If the intent is to ship the options automatically, that would make it hard for the user to configure, say, different numbers of CPUs and memory per host, or a different LIBPROCESS_IP, or stuff like that.
>
> Andy Konwinski wrote:
> I chatted with Matei offline about this. The current flow does not ship the slaves-env.sh file over. If people wanted to set up unique per-machine slave configurations, they can create a config file somewhere on each such machine containing the special settings for that machine, and then set their slaves-env.sh to point to that via the --conf=/location/of/mesos/conf/file/on/slave/machines, and the values set in those files would override anything passed over via the deploy scripts as env variables.
>
> This raises the important issue of what the order of precedence is for options that are set in multipole of {env variable, configuration file, command line} (note that this is not a new issue). https://github.com/mesos/mesos/wiki/Configuration currently says "Configuration values are searched for first on the command line, then in the environment, and then in the config file." but we should expand that comment to say explicitly what happens if a variable is set in more than one place (maybe even give an example).
Just to make it clear, the goal of the deploy scripts was never to ship the configuration automatically to all the machines. The old deploy-to-slaves script happened to do that, but its main purpose is to let people developing Mesos modify and recompile the code locally, then sync it to all the machines. I think it's fine to tell the user to configure each machine separately -- in fact they won't have to do a single thing by default because most users don't set any config settings on slaves. Otherwise, the mechanisms for "global" and "local" configuration should be consistent (both environment variables or both files).
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > src/deploy/mesos-start-masters.sh.in, line 36
> > <https://reviews.apache.org/r/4491/diff/1/?file=95974#file95974line36>
> >
> > What does this do? If you're trying to ship the env variables from the local config files to the machine, shouldn't you do export ${var} = value? Also, gotta be careful with quoting here.
>
> Andy Konwinski wrote:
> I believe the `var` command prints one "KEY=value" pair per line so the way Ben has it should work. Perhaps renaming the bash variable here from var to var_and_value or var_value_pair would make it more clear.
>
> Also, I think you're right Matei that the way we're passing the environment variables right now might break if any of those environment variables contain double quotes (e.g. <code>export MESOS_CONFIG_VAR="say \"hi\""</code>). One thing we could do is just state in the documentation (on the wiki, in the README, and in the *-env.sh files themselves) that users should not include any strings with any quotes (i.e. \") in them.
This is pretty hacky IMO -- would prefer to avoid this. Just having it in the documentation doesn't mean it will be easy to find (a user might still spend a while debugging). The problem is not only quotes but also spaces in the variables, because env doesn't quite its outputs. For example:
matei@mbk:~$ export MYVAR="hello world"
matei@mbk:~$ env | grep MYVAR
MYVAR=hello world
- Matei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6383
-----------------------------------------------------------
On 2012-03-26 21:38:18, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-26 21:38:18)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.in PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.in PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.in PRE-CREATION
> src/deploy/mesos-slaves-env.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-start.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
> src/tests/master_detector_tests.cpp PRE-CREATION
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > README, line 141
> > <https://reviews.apache.org/r/4491/diff/1/?file=95961#file95961line141>
> >
> > But the slaves-env.sh file needs to be on all the nodes, right? If the intent is to ship the options automatically, that would make it hard for the user to configure, say, different numbers of CPUs and memory per host, or a different LIBPROCESS_IP, or stuff like that.
>
> Andy Konwinski wrote:
> I chatted with Matei offline about this. The current flow does not ship the slaves-env.sh file over. If people wanted to set up unique per-machine slave configurations, they can create a config file somewhere on each such machine containing the special settings for that machine, and then set their slaves-env.sh to point to that via the --conf=/location/of/mesos/conf/file/on/slave/machines, and the values set in those files would override anything passed over via the deploy scripts as env variables.
>
> This raises the important issue of what the order of precedence is for options that are set in multipole of {env variable, configuration file, command line} (note that this is not a new issue). https://github.com/mesos/mesos/wiki/Configuration currently says "Configuration values are searched for first on the command line, then in the environment, and then in the config file." but we should expand that comment to say explicitly what happens if a variable is set in more than one place (maybe even give an example).
>
> Matei Zaharia wrote:
> Just to make it clear, the goal of the deploy scripts was never to ship the configuration automatically to all the machines. The old deploy-to-slaves script happened to do that, but its main purpose is to let people developing Mesos modify and recompile the code locally, then sync it to all the machines. I think it's fine to tell the user to configure each machine separately -- in fact they won't have to do a single thing by default because most users don't set any config settings on slaves. Otherwise, the mechanisms for "global" and "local" configuration should be consistent (both environment variables or both files).
>
> Matei Zaharia wrote:
> By the way, my suggestion with this kind of stuff is that if you don't want to spend a long time thinking about how to do it, find another project that uses a similar configuration process, and copy what it does. Otherwise, you will repeat mistakes that others have found and fixed in the past. This is why the old scripts were based on per-machine config files in a key-value pair format -- because a lot of other cluster software does that. Same thing with the localstatedir / .template file issue. If in doubt go with .template and tell the user to copy it.
>
> Benjamin Hindman wrote:
> Woah, hold on a second. I was trying to get the _same_ semantics as the _old_ scripts. In the old scripts, if you had a local configuration file which had the 'url' field set the deploy scripts would actually use that when launching masters and slaves. That is to say, local configuration was used to create command line options for remote execution! I've stayed consistent with this methodology, just using mesos-*env.sh and environment variables rather than a configuration file in order to deal with naming conflicts of options between slaves and masters.
>
> But to be clear, as Andy mentioned, by setting MESOS_CONF any local configuration will get overridden by remote hosts configuration files. I would have been happy not implementing this, but I was _trying_ to keep the old workflow.
Got it. I think the old scripts were doing this wrong for the URL then, by assuming that the URL is the same on all the machines. The sudo was similarly a special case because of the way we launched commands (we needed to run ssh <node> sudo <command> instead of doing ssh <node> <command> and having the <command> script call sudo. I don't think this is the right way to go.
Again, I think setting the --conf flag to specify local options is a bad idea because it requires the user to learn two forms of configuration.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > src/deploy/mesos-start-masters.sh.in, line 36
> > <https://reviews.apache.org/r/4491/diff/1/?file=95974#file95974line36>
> >
> > What does this do? If you're trying to ship the env variables from the local config files to the machine, shouldn't you do export ${var} = value? Also, gotta be careful with quoting here.
>
> Andy Konwinski wrote:
> I believe the `var` command prints one "KEY=value" pair per line so the way Ben has it should work. Perhaps renaming the bash variable here from var to var_and_value or var_value_pair would make it more clear.
>
> Also, I think you're right Matei that the way we're passing the environment variables right now might break if any of those environment variables contain double quotes (e.g. <code>export MESOS_CONFIG_VAR="say \"hi\""</code>). One thing we could do is just state in the documentation (on the wiki, in the README, and in the *-env.sh files themselves) that users should not include any strings with any quotes (i.e. \") in them.
>
> Matei Zaharia wrote:
> This is pretty hacky IMO -- would prefer to avoid this. Just having it in the documentation doesn't mean it will be easy to find (a user might still spend a while debugging). The problem is not only quotes but also spaces in the variables, because env doesn't quite its outputs. For example:
>
> matei@mbk:~$ export MYVAR="hello world"
> matei@mbk:~$ env | grep MYVAR
> MYVAR=hello world
>
> Benjamin Hindman wrote:
> I've updated the scripts to handle the quoting better. Again, this is an attempt to match the old semantics. However, in the old semantics, not all the configuration options specified in the local configuration would be used, only those that had been added to the deploy scripts and determined via 'mesos-getconf'. This sounds horrible to me, because it means as we add configuration options we also need to remember to add them into the deploy scripts (just satisfying all of the current options would have required a lot more work).
As mentioned above, this was more of a mistake for URL and deploy_with_sudo than an intended goal. The intent was that you can configure each machine separately.
- Matei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6383
-----------------------------------------------------------
On 2012-03-27 04:38:04, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 04:38:04)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.template PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.template PRE-CREATION
> src/deploy/mesos-slaves-env.sh.template PRE-CREATION
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > README, line 121
> > <https://reviews.apache.org/r/4491/diff/1/?file=95961#file95961line121>
> >
> > You might want to provide an example of installing to a non-local prefix, because some users will probably not be familiar with the concept. Then refer back to this at the start of the next section.
Done.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > README, line 129
> > <https://reviews.apache.org/r/4491/diff/1/?file=95961#file95961line129>
> >
> > Put prefix in [brackets]
Done, here and everywhere else.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > README, line 151
> > <https://reviews.apache.org/r/4491/diff/1/?file=95961#file95961line151>
> >
> > Why is this not called start-mesos anymore? If it's to keep it in line with the rest of the binaries we put on the PATH, I would go for something like mesos-start-cluster.sh, which seems more grammatically correct than just "start".
>
> Andy Konwinski wrote:
> I like this idea.
Fixed.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > src/Makefile.am, line 40
> > <https://reviews.apache.org/r/4491/diff/1/?file=95963#file95963line40>
> >
> > Just to double-check, if someone does a make install, then modifies some config files, and then downloads a new version of Mesos and does make install again, will their directories be overwritten? Or does install protect the existing files?
>
> Andy Konwinski wrote:
> This is a great point, and we should be very careful about this. I'm not sure what the exact process for re-installing mesos is actually, but I think it is simply to run make install again after building a newer version of Mesos. In that case, perhaps we should only install *-env.sh.template files instead of *.-env.sh files.
Changed to install *.template. The deploy scripts don't barf if none exist either.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > src/deploy/mesos-deploy-env.sh.in, line 5
> > <https://reviews.apache.org/r/4491/diff/1/?file=95970#file95970line5>
> >
> > Would be nice to have a list of the variables that can be set here.
There are only two for now, those listed.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > src/deploy/mesos-masters-env.sh.in, line 6
> > <https://reviews.apache.org/r/4491/diff/1/?file=95972#file95972line6>
> >
> > Say that you can run mesos-master --help for a list of parameters.
Done.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > src/tests/master_detector_tests.cpp, line 90
> > <https://reviews.apache.org/r/4491/diff/1/?file=95987#file95987line90>
> >
> > Was this included in the patch by mistake?
Yes. Removed for another patch.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > README, line 141
> > <https://reviews.apache.org/r/4491/diff/1/?file=95961#file95961line141>
> >
> > But the slaves-env.sh file needs to be on all the nodes, right? If the intent is to ship the options automatically, that would make it hard for the user to configure, say, different numbers of CPUs and memory per host, or a different LIBPROCESS_IP, or stuff like that.
>
> Andy Konwinski wrote:
> I chatted with Matei offline about this. The current flow does not ship the slaves-env.sh file over. If people wanted to set up unique per-machine slave configurations, they can create a config file somewhere on each such machine containing the special settings for that machine, and then set their slaves-env.sh to point to that via the --conf=/location/of/mesos/conf/file/on/slave/machines, and the values set in those files would override anything passed over via the deploy scripts as env variables.
>
> This raises the important issue of what the order of precedence is for options that are set in multipole of {env variable, configuration file, command line} (note that this is not a new issue). https://github.com/mesos/mesos/wiki/Configuration currently says "Configuration values are searched for first on the command line, then in the environment, and then in the config file." but we should expand that comment to say explicitly what happens if a variable is set in more than one place (maybe even give an example).
>
> Matei Zaharia wrote:
> Just to make it clear, the goal of the deploy scripts was never to ship the configuration automatically to all the machines. The old deploy-to-slaves script happened to do that, but its main purpose is to let people developing Mesos modify and recompile the code locally, then sync it to all the machines. I think it's fine to tell the user to configure each machine separately -- in fact they won't have to do a single thing by default because most users don't set any config settings on slaves. Otherwise, the mechanisms for "global" and "local" configuration should be consistent (both environment variables or both files).
>
> Matei Zaharia wrote:
> By the way, my suggestion with this kind of stuff is that if you don't want to spend a long time thinking about how to do it, find another project that uses a similar configuration process, and copy what it does. Otherwise, you will repeat mistakes that others have found and fixed in the past. This is why the old scripts were based on per-machine config files in a key-value pair format -- because a lot of other cluster software does that. Same thing with the localstatedir / .template file issue. If in doubt go with .template and tell the user to copy it.
Woah, hold on a second. I was trying to get the _same_ semantics as the _old_ scripts. In the old scripts, if you had a local configuration file which had the 'url' field set the deploy scripts would actually use that when launching masters and slaves. That is to say, local configuration was used to create command line options for remote execution! I've stayed consistent with this methodology, just using mesos-*env.sh and environment variables rather than a configuration file in order to deal with naming conflicts of options between slaves and masters.
But to be clear, as Andy mentioned, by setting MESOS_CONF any local configuration will get overridden by remote hosts configuration files. I would have been happy not implementing this, but I was _trying_ to keep the old workflow.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > src/deploy/mesos-start-masters.sh.in, line 36
> > <https://reviews.apache.org/r/4491/diff/1/?file=95974#file95974line36>
> >
> > What does this do? If you're trying to ship the env variables from the local config files to the machine, shouldn't you do export ${var} = value? Also, gotta be careful with quoting here.
>
> Andy Konwinski wrote:
> I believe the `var` command prints one "KEY=value" pair per line so the way Ben has it should work. Perhaps renaming the bash variable here from var to var_and_value or var_value_pair would make it more clear.
>
> Also, I think you're right Matei that the way we're passing the environment variables right now might break if any of those environment variables contain double quotes (e.g. <code>export MESOS_CONFIG_VAR="say \"hi\""</code>). One thing we could do is just state in the documentation (on the wiki, in the README, and in the *-env.sh files themselves) that users should not include any strings with any quotes (i.e. \") in them.
>
> Matei Zaharia wrote:
> This is pretty hacky IMO -- would prefer to avoid this. Just having it in the documentation doesn't mean it will be easy to find (a user might still spend a while debugging). The problem is not only quotes but also spaces in the variables, because env doesn't quite its outputs. For example:
>
> matei@mbk:~$ export MYVAR="hello world"
> matei@mbk:~$ env | grep MYVAR
> MYVAR=hello world
I've updated the scripts to handle the quoting better. Again, this is an attempt to match the old semantics. However, in the old semantics, not all the configuration options specified in the local configuration would be used, only those that had been added to the deploy scripts and determined via 'mesos-getconf'. This sounds horrible to me, because it means as we add configuration options we also need to remember to add them into the deploy scripts (just satisfying all of the current options would have required a lot more work).
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6383
-----------------------------------------------------------
On 2012-03-26 21:38:18, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-26 21:38:18)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.in PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.in PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.in PRE-CREATION
> src/deploy/mesos-slaves-env.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-start.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
> src/tests/master_detector_tests.cpp PRE-CREATION
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
> On 2012-03-26 23:09:55, Matei Zaharia wrote:
> > README, line 141
> > <https://reviews.apache.org/r/4491/diff/1/?file=95961#file95961line141>
> >
> > But the slaves-env.sh file needs to be on all the nodes, right? If the intent is to ship the options automatically, that would make it hard for the user to configure, say, different numbers of CPUs and memory per host, or a different LIBPROCESS_IP, or stuff like that.
>
> Andy Konwinski wrote:
> I chatted with Matei offline about this. The current flow does not ship the slaves-env.sh file over. If people wanted to set up unique per-machine slave configurations, they can create a config file somewhere on each such machine containing the special settings for that machine, and then set their slaves-env.sh to point to that via the --conf=/location/of/mesos/conf/file/on/slave/machines, and the values set in those files would override anything passed over via the deploy scripts as env variables.
>
> This raises the important issue of what the order of precedence is for options that are set in multipole of {env variable, configuration file, command line} (note that this is not a new issue). https://github.com/mesos/mesos/wiki/Configuration currently says "Configuration values are searched for first on the command line, then in the environment, and then in the config file." but we should expand that comment to say explicitly what happens if a variable is set in more than one place (maybe even give an example).
>
> Matei Zaharia wrote:
> Just to make it clear, the goal of the deploy scripts was never to ship the configuration automatically to all the machines. The old deploy-to-slaves script happened to do that, but its main purpose is to let people developing Mesos modify and recompile the code locally, then sync it to all the machines. I think it's fine to tell the user to configure each machine separately -- in fact they won't have to do a single thing by default because most users don't set any config settings on slaves. Otherwise, the mechanisms for "global" and "local" configuration should be consistent (both environment variables or both files).
>
> Matei Zaharia wrote:
> By the way, my suggestion with this kind of stuff is that if you don't want to spend a long time thinking about how to do it, find another project that uses a similar configuration process, and copy what it does. Otherwise, you will repeat mistakes that others have found and fixed in the past. This is why the old scripts were based on per-machine config files in a key-value pair format -- because a lot of other cluster software does that. Same thing with the localstatedir / .template file issue. If in doubt go with .template and tell the user to copy it.
>
> Benjamin Hindman wrote:
> Woah, hold on a second. I was trying to get the _same_ semantics as the _old_ scripts. In the old scripts, if you had a local configuration file which had the 'url' field set the deploy scripts would actually use that when launching masters and slaves. That is to say, local configuration was used to create command line options for remote execution! I've stayed consistent with this methodology, just using mesos-*env.sh and environment variables rather than a configuration file in order to deal with naming conflicts of options between slaves and masters.
>
> But to be clear, as Andy mentioned, by setting MESOS_CONF any local configuration will get overridden by remote hosts configuration files. I would have been happy not implementing this, but I was _trying_ to keep the old workflow.
>
> Matei Zaharia wrote:
> Got it. I think the old scripts were doing this wrong for the URL then, by assuming that the URL is the same on all the machines. The sudo was similarly a special case because of the way we launched commands (we needed to run ssh <node> sudo <command> instead of doing ssh <node> <command> and having the <command> script call sudo. I don't think this is the right way to go.
>
> Again, I think setting the --conf flag to specify local options is a bad idea because it requires the user to learn two forms of configuration.
By the way, the core reason for this is that some options, like resources and IP to bind to, have to differ across hosts. Let's make that easy. Copying other stuff, like URL, across the nodes is easy. But it would be very nonintuitive for a user to see a "mesos-slaves-env.sh" script locally, set some stuff in it, and not have it affect their process.
- Matei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6383
-----------------------------------------------------------
On 2012-03-27 04:38:04, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 04:38:04)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.template PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.template PRE-CREATION
> src/deploy/mesos-slaves-env.sh.template PRE-CREATION
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6383
-----------------------------------------------------------
README
<https://reviews.apache.org/r/4491/#comment13883>
You might want to provide an example of installing to a non-local prefix, because some users will probably not be familiar with the concept. Then refer back to this at the start of the next section.
README
<https://reviews.apache.org/r/4491/#comment13884>
Put prefix in [brackets]
README
<https://reviews.apache.org/r/4491/#comment13894>
Mention that the scripts also need Mesos to be installed to the same prefix on all machines.
README
<https://reviews.apache.org/r/4491/#comment13880>
But the slaves-env.sh file needs to be on all the nodes, right? If the intent is to ship the options automatically, that would make it hard for the user to configure, say, different numbers of CPUs and memory per host, or a different LIBPROCESS_IP, or stuff like that.
README
<https://reviews.apache.org/r/4491/#comment13886>
Why is this not called start-mesos anymore? If it's to keep it in line with the rest of the binaries we put on the PATH, I would go for something like mesos-start-cluster.sh, which seems more grammatically correct than just "start".
src/Makefile.am
<https://reviews.apache.org/r/4491/#comment13887>
Just to double-check, if someone does a make install, then modifies some config files, and then downloads a new version of Mesos and does make install again, will their directories be overwritten? Or does install protect the existing files?
src/deploy/mesos-deploy-env.sh.in
<https://reviews.apache.org/r/4491/#comment13888>
Would be nice to have a list of the variables that can be set here.
src/deploy/mesos-masters-env.sh.in
<https://reviews.apache.org/r/4491/#comment13889>
Say that you can run mesos-master --help for a list of parameters.
src/deploy/mesos-start-masters.sh.in
<https://reviews.apache.org/r/4491/#comment13892>
What does this do? If you're trying to ship the env variables from the local config files to the machine, shouldn't you do export ${var} = value? Also, gotta be careful with quoting here.
src/deploy/mesos-start-slaves.sh.in
<https://reviews.apache.org/r/4491/#comment13891>
Ditto to the similar thing in masters script.
src/tests/master_detector_tests.cpp
<https://reviews.apache.org/r/4491/#comment13895>
Was this included in the patch by mistake?
- Matei
On 2012-03-26 21:38:18, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-26 21:38:18)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.in PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.in PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.in PRE-CREATION
> src/deploy/mesos-slaves-env.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-start.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
> src/tests/master_detector_tests.cpp PRE-CREATION
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
> On 2012-03-27 07:26:28, Matei Zaharia wrote:
> >
Sorry, let's not "ship it" until we have made sure that this is usable (i.e. figured out the local vs global config thing). Again, I would go for a local config (read the scripts on each machine separately) unless there's some really good reason to go global.
- Matei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6419
-----------------------------------------------------------
On 2012-03-27 04:38:04, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 04:38:04)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.template PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.template PRE-CREATION
> src/deploy/mesos-slaves-env.sh.template PRE-CREATION
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6419
-----------------------------------------------------------
- Matei
On 2012-03-27 04:38:04, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 04:38:04)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.template PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.template PRE-CREATION
> src/deploy/mesos-slaves-env.sh.template PRE-CREATION
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6420
-----------------------------------------------------------
README
<https://reviews.apache.org/r/4491/#comment14017>
Typo here ("prefixec").
- Matei
On 2012-03-27 04:38:04, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 04:38:04)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.template PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.template PRE-CREATION
> src/deploy/mesos-slaves-env.sh.template PRE-CREATION
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Andy Konwinski <an...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6413
-----------------------------------------------------------
Ship it!
I've applied this patch and tested it. Everything seems great. Thanks for doing this Ben!
- Andy
On 2012-03-27 04:38:04, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 04:38:04)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.template PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.template PRE-CREATION
> src/deploy/mesos-slaves-env.sh.template PRE-CREATION
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
> On 2012-03-27 22:43:05, Matei Zaharia wrote:
> > This looks good, except for a few minor things:
> > 1) If you just run these out of the box, the slave will silently fail to start because its "--master" command line flag is not passed. We should tell the user to configure this in mesos.conf, or try to set it automatically if there is only one slave.
> > 2) The previous scripts attempted to automatically find the amount of cores and memory available. Should we add that back?
> > 3) It's not clear that the master and slaves are logging to anywhere (if you just run /sbin/mesos-master for example, it says "Logging to" and does not give a path). If they aren't, we should update the docs to say that you can turn on logging by setting log_dir. Ditto for the work directory (where will that be after a make install? It seems like the default is ./work).
>
> Benjamin Hindman wrote:
> 1) I've updated the README with a reminder about at least the "--master" flag.
> 2) No. I'd rather not do anything implicitly. As long as we are forcing users to update configuration files, they might as well add this information in themselves. How difficult this will be for them to do (or the lack of being able to do this programmatically) is a different story, but I don't want to tackle that now, before the release. If you have suggestions for this please create a JIRA where we can brainstorm together!
> 3) I've updated the output to say "Logging to <stderr>". I've also moved the work_dir to default to /tmp/mesos (this is okay for multiple slaves because each actual work directory for the slaves will be under /tmp/mesos/slaves/ID/...).
>
> Matei Zaharia wrote:
> Regarding 2, why not add back the old code (the one that set extra options in https://github.com/apache/mesos/blob/trunk/src/deploy/mesos-daemon)? Why break existing functionality for the release?
>
> Regarding 3, just mention that you need to set log_dir to see output when using the deploy scripts, because by default stderr gets redirected to /dev/null.
>
> Benjamin Hindman wrote:
> 2) We are breaking all sorts of functionality for these deploy scripts, for example, no more local mesos-env.sh being used to set options remotely. If the plan was to go with simplicity, and we're already breaking functionality, why add these back?
> 3) Okay, I'll add this to the README.
Well, it's a matter of judgement, but I would say that anything that saves the user's time is worth it. My own goal with the scripts was not "simplicity" for the sake of simplicity. It was to make Mesos as easy to use as possible. Anyone setting it up on a cluster will probably want to use all the CPUs and memory present on each machine, so why not do that by default?
Again, this is probably not obvious at Twitter, but Mesos is far from easy to set up and manage for people who haven't done it before. We should really strive to have a setup procedure that can be described in 4-5 shell commands. Each additional step turns people off. The Hadoop tutorial was a great example of doing that, but we need a similar thing for setting up your first cluster. It doesn't need to be the most feature-complete way to manage a cluster; advanced users might switch to monit or chef or something anyway; but it needs to save the user time and give them a reasonable running system with minimal effort.
- Matei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6455
-----------------------------------------------------------
On 2012-03-27 23:12:32, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 23:12:32)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 448410e
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/configurator/configurator.cpp 1f8134c
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh 14ad67a
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp e88315f
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
> On 2012-03-27 22:43:05, Matei Zaharia wrote:
> > This looks good, except for a few minor things:
> > 1) If you just run these out of the box, the slave will silently fail to start because its "--master" command line flag is not passed. We should tell the user to configure this in mesos.conf, or try to set it automatically if there is only one slave.
> > 2) The previous scripts attempted to automatically find the amount of cores and memory available. Should we add that back?
> > 3) It's not clear that the master and slaves are logging to anywhere (if you just run /sbin/mesos-master for example, it says "Logging to" and does not give a path). If they aren't, we should update the docs to say that you can turn on logging by setting log_dir. Ditto for the work directory (where will that be after a make install? It seems like the default is ./work).
>
> Benjamin Hindman wrote:
> 1) I've updated the README with a reminder about at least the "--master" flag.
> 2) No. I'd rather not do anything implicitly. As long as we are forcing users to update configuration files, they might as well add this information in themselves. How difficult this will be for them to do (or the lack of being able to do this programmatically) is a different story, but I don't want to tackle that now, before the release. If you have suggestions for this please create a JIRA where we can brainstorm together!
> 3) I've updated the output to say "Logging to <stderr>". I've also moved the work_dir to default to /tmp/mesos (this is okay for multiple slaves because each actual work directory for the slaves will be under /tmp/mesos/slaves/ID/...).
>
> Matei Zaharia wrote:
> Regarding 2, why not add back the old code (the one that set extra options in https://github.com/apache/mesos/blob/trunk/src/deploy/mesos-daemon)? Why break existing functionality for the release?
>
> Regarding 3, just mention that you need to set log_dir to see output when using the deploy scripts, because by default stderr gets redirected to /dev/null.
>
> Benjamin Hindman wrote:
> 2) We are breaking all sorts of functionality for these deploy scripts, for example, no more local mesos-env.sh being used to set options remotely. If the plan was to go with simplicity, and we're already breaking functionality, why add these back?
> 3) Okay, I'll add this to the README.
>
> Matei Zaharia wrote:
> Well, it's a matter of judgement, but I would say that anything that saves the user's time is worth it. My own goal with the scripts was not "simplicity" for the sake of simplicity. It was to make Mesos as easy to use as possible. Anyone setting it up on a cluster will probably want to use all the CPUs and memory present on each machine, so why not do that by default?
>
> Again, this is probably not obvious at Twitter, but Mesos is far from easy to set up and manage for people who haven't done it before. We should really strive to have a setup procedure that can be described in 4-5 shell commands. Each additional step turns people off. The Hadoop tutorial was a great example of doing that, but we need a similar thing for setting up your first cluster. It doesn't need to be the most feature-complete way to manage a cluster; advanced users might switch to monit or chef or something anyway; but it needs to save the user time and give them a reasonable running system with minimal effort.
>
> Benjamin Hindman wrote:
> I think I've got a good solution. I've updated the slave to automatically try and determine the number of cpus and amount of main memory on each machine. I'm going to commit the deploy scripts the way they are and then post that patch. This gives better defaults AND accomplishes making it easy to setup a running cluster with minimal effort without setting configuration options behind the users back. I'll post that patch shortly.
Sounds like a good solution. Let's do it.
- Matei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6455
-----------------------------------------------------------
On 2012-03-27 23:12:32, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 23:12:32)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 448410e
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/configurator/configurator.cpp 1f8134c
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh 14ad67a
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp e88315f
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
> On 2012-03-27 22:43:05, Matei Zaharia wrote:
> > This looks good, except for a few minor things:
> > 1) If you just run these out of the box, the slave will silently fail to start because its "--master" command line flag is not passed. We should tell the user to configure this in mesos.conf, or try to set it automatically if there is only one slave.
> > 2) The previous scripts attempted to automatically find the amount of cores and memory available. Should we add that back?
> > 3) It's not clear that the master and slaves are logging to anywhere (if you just run /sbin/mesos-master for example, it says "Logging to" and does not give a path). If they aren't, we should update the docs to say that you can turn on logging by setting log_dir. Ditto for the work directory (where will that be after a make install? It seems like the default is ./work).
>
> Benjamin Hindman wrote:
> 1) I've updated the README with a reminder about at least the "--master" flag.
> 2) No. I'd rather not do anything implicitly. As long as we are forcing users to update configuration files, they might as well add this information in themselves. How difficult this will be for them to do (or the lack of being able to do this programmatically) is a different story, but I don't want to tackle that now, before the release. If you have suggestions for this please create a JIRA where we can brainstorm together!
> 3) I've updated the output to say "Logging to <stderr>". I've also moved the work_dir to default to /tmp/mesos (this is okay for multiple slaves because each actual work directory for the slaves will be under /tmp/mesos/slaves/ID/...).
>
> Matei Zaharia wrote:
> Regarding 2, why not add back the old code (the one that set extra options in https://github.com/apache/mesos/blob/trunk/src/deploy/mesos-daemon)? Why break existing functionality for the release?
>
> Regarding 3, just mention that you need to set log_dir to see output when using the deploy scripts, because by default stderr gets redirected to /dev/null.
2) We are breaking all sorts of functionality for these deploy scripts, for example, no more local mesos-env.sh being used to set options remotely. If the plan was to go with simplicity, and we're already breaking functionality, why add these back?
3) Okay, I'll add this to the README.
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6455
-----------------------------------------------------------
On 2012-03-27 23:12:32, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 23:12:32)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 448410e
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/configurator/configurator.cpp 1f8134c
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh 14ad67a
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp e88315f
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
> On 2012-03-27 22:43:05, Matei Zaharia wrote:
> > This looks good, except for a few minor things:
> > 1) If you just run these out of the box, the slave will silently fail to start because its "--master" command line flag is not passed. We should tell the user to configure this in mesos.conf, or try to set it automatically if there is only one slave.
> > 2) The previous scripts attempted to automatically find the amount of cores and memory available. Should we add that back?
> > 3) It's not clear that the master and slaves are logging to anywhere (if you just run /sbin/mesos-master for example, it says "Logging to" and does not give a path). If they aren't, we should update the docs to say that you can turn on logging by setting log_dir. Ditto for the work directory (where will that be after a make install? It seems like the default is ./work).
>
> Benjamin Hindman wrote:
> 1) I've updated the README with a reminder about at least the "--master" flag.
> 2) No. I'd rather not do anything implicitly. As long as we are forcing users to update configuration files, they might as well add this information in themselves. How difficult this will be for them to do (or the lack of being able to do this programmatically) is a different story, but I don't want to tackle that now, before the release. If you have suggestions for this please create a JIRA where we can brainstorm together!
> 3) I've updated the output to say "Logging to <stderr>". I've also moved the work_dir to default to /tmp/mesos (this is okay for multiple slaves because each actual work directory for the slaves will be under /tmp/mesos/slaves/ID/...).
>
> Matei Zaharia wrote:
> Regarding 2, why not add back the old code (the one that set extra options in https://github.com/apache/mesos/blob/trunk/src/deploy/mesos-daemon)? Why break existing functionality for the release?
>
> Regarding 3, just mention that you need to set log_dir to see output when using the deploy scripts, because by default stderr gets redirected to /dev/null.
>
> Benjamin Hindman wrote:
> 2) We are breaking all sorts of functionality for these deploy scripts, for example, no more local mesos-env.sh being used to set options remotely. If the plan was to go with simplicity, and we're already breaking functionality, why add these back?
> 3) Okay, I'll add this to the README.
>
> Matei Zaharia wrote:
> Well, it's a matter of judgement, but I would say that anything that saves the user's time is worth it. My own goal with the scripts was not "simplicity" for the sake of simplicity. It was to make Mesos as easy to use as possible. Anyone setting it up on a cluster will probably want to use all the CPUs and memory present on each machine, so why not do that by default?
>
> Again, this is probably not obvious at Twitter, but Mesos is far from easy to set up and manage for people who haven't done it before. We should really strive to have a setup procedure that can be described in 4-5 shell commands. Each additional step turns people off. The Hadoop tutorial was a great example of doing that, but we need a similar thing for setting up your first cluster. It doesn't need to be the most feature-complete way to manage a cluster; advanced users might switch to monit or chef or something anyway; but it needs to save the user time and give them a reasonable running system with minimal effort.
I think I've got a good solution. I've updated the slave to automatically try and determine the number of cpus and amount of main memory on each machine. I'm going to commit the deploy scripts the way they are and then post that patch. This gives better defaults AND accomplishes making it easy to setup a running cluster with minimal effort without setting configuration options behind the users back. I'll post that patch shortly.
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6455
-----------------------------------------------------------
On 2012-03-27 23:12:32, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 23:12:32)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 448410e
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/configurator/configurator.cpp 1f8134c
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh 14ad67a
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp e88315f
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
> On 2012-03-27 22:43:05, Matei Zaharia wrote:
> > This looks good, except for a few minor things:
> > 1) If you just run these out of the box, the slave will silently fail to start because its "--master" command line flag is not passed. We should tell the user to configure this in mesos.conf, or try to set it automatically if there is only one slave.
> > 2) The previous scripts attempted to automatically find the amount of cores and memory available. Should we add that back?
> > 3) It's not clear that the master and slaves are logging to anywhere (if you just run /sbin/mesos-master for example, it says "Logging to" and does not give a path). If they aren't, we should update the docs to say that you can turn on logging by setting log_dir. Ditto for the work directory (where will that be after a make install? It seems like the default is ./work).
>
> Benjamin Hindman wrote:
> 1) I've updated the README with a reminder about at least the "--master" flag.
> 2) No. I'd rather not do anything implicitly. As long as we are forcing users to update configuration files, they might as well add this information in themselves. How difficult this will be for them to do (or the lack of being able to do this programmatically) is a different story, but I don't want to tackle that now, before the release. If you have suggestions for this please create a JIRA where we can brainstorm together!
> 3) I've updated the output to say "Logging to <stderr>". I've also moved the work_dir to default to /tmp/mesos (this is okay for multiple slaves because each actual work directory for the slaves will be under /tmp/mesos/slaves/ID/...).
Regarding 2, why not add back the old code (the one that set extra options in https://github.com/apache/mesos/blob/trunk/src/deploy/mesos-daemon)? Why break existing functionality for the release?
Regarding 3, just mention that you need to set log_dir to see output when using the deploy scripts, because by default stderr gets redirected to /dev/null.
- Matei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6455
-----------------------------------------------------------
On 2012-03-27 23:12:32, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 23:12:32)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 448410e
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/configurator/configurator.cpp 1f8134c
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh 14ad67a
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp e88315f
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
> On 2012-03-27 22:43:05, Matei Zaharia wrote:
> > This looks good, except for a few minor things:
> > 1) If you just run these out of the box, the slave will silently fail to start because its "--master" command line flag is not passed. We should tell the user to configure this in mesos.conf, or try to set it automatically if there is only one slave.
> > 2) The previous scripts attempted to automatically find the amount of cores and memory available. Should we add that back?
> > 3) It's not clear that the master and slaves are logging to anywhere (if you just run /sbin/mesos-master for example, it says "Logging to" and does not give a path). If they aren't, we should update the docs to say that you can turn on logging by setting log_dir. Ditto for the work directory (where will that be after a make install? It seems like the default is ./work).
1) I've updated the README with a reminder about at least the "--master" flag.
2) No. I'd rather not do anything implicitly. As long as we are forcing users to update configuration files, they might as well add this information in themselves. How difficult this will be for them to do (or the lack of being able to do this programmatically) is a different story, but I don't want to tackle that now, before the release. If you have suggestions for this please create a JIRA where we can brainstorm together!
3) I've updated the output to say "Logging to <stderr>". I've also moved the work_dir to default to /tmp/mesos (this is okay for multiple slaves because each actual work directory for the slaves will be under /tmp/mesos/slaves/ID/...).
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6455
-----------------------------------------------------------
On 2012-03-27 21:15:39, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 21:15:39)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 448410e
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/configurator/configurator.cpp 1f8134c
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh 14ad67a
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp e88315f
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6455
-----------------------------------------------------------
This looks good, except for a few minor things:
1) If you just run these out of the box, the slave will silently fail to start because its "--master" command line flag is not passed. We should tell the user to configure this in mesos.conf, or try to set it automatically if there is only one slave.
2) The previous scripts attempted to automatically find the amount of cores and memory available. Should we add that back?
3) It's not clear that the master and slaves are logging to anywhere (if you just run /sbin/mesos-master for example, it says "Logging to" and does not give a path). If they aren't, we should update the docs to say that you can turn on logging by setting log_dir. Ditto for the work directory (where will that be after a make install? It seems like the default is ./work).
- Matei
On 2012-03-27 21:15:39, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 21:15:39)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 448410e
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/configurator/configurator.cpp 1f8134c
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh 14ad67a
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp e88315f
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Andy Konwinski <an...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6456
-----------------------------------------------------------
Ship it!
I downloaded the newest patch, and tested it. Everything seems to work. Let's get this committed, and then release!!
- Andy
On 2012-03-27 21:15:39, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 21:15:39)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 448410e
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/configurator/configurator.cpp 1f8134c
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh 14ad67a
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp e88315f
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
> On 2012-03-28 02:57:51, Matei Zaharia wrote:
> > Looks good to me with the auto-detection of resources in the slave. Thanks for spending the time on this!
Awesome, submitted. Patch for the auto-detection here: https://reviews.apache.org/r/4532.
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6465
-----------------------------------------------------------
On 2012-03-27 23:12:32, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 23:12:32)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 448410e
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/configurator/configurator.cpp 1f8134c
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh 14ad67a
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp e88315f
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Matei Zaharia <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6465
-----------------------------------------------------------
Ship it!
Looks good to me with the auto-detection of resources in the slave. Thanks for spending the time on this!
- Matei
On 2012-03-27 23:12:32, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 23:12:32)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 448410e
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/configurator/configurator.cpp 1f8134c
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh 14ad67a
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp e88315f
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/
-----------------------------------------------------------
(Updated 2012-03-27 23:12:32.655005)
Review request for mesos, Andy Konwinski and Matei Zaharia.
Changes
-------
Review comments.
Summary
-------
Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
Diffs (updated)
-----
README 8166b04
configure.ac 7a0d89f
src/Makefile.am 448410e
src/conf/deploy-env.sh.template a4565e2
src/configurator/configurator.hpp ecd1365
src/configurator/configurator.cpp 1f8134c
src/deploy/deploy-to-slaves 0e10486
src/deploy/mesos-daemon b3144c0
src/deploy/mesos-daemon.sh.in PRE-CREATION
src/deploy/mesos-deploy-env.sh.template PRE-CREATION
src/deploy/mesos-env.sh 14ad67a
src/deploy/mesos-start-cluster.sh.in PRE-CREATION
src/deploy/mesos-start-masters.sh.in PRE-CREATION
src/deploy/mesos-start-slaves.sh.in PRE-CREATION
src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
src/deploy/mesos-stop-masters.sh.in PRE-CREATION
src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
src/deploy/start-masters a450ee0
src/deploy/start-mesos 9e5b922
src/deploy/start-slaves b3b931f
src/deploy/stop-masters f2aa9e2
src/deploy/stop-mesos 679a09e
src/deploy/stop-slaves 141acd4
src/master/main.cpp e88315f
Diff: https://reviews.apache.org/r/4491/diff
Testing
-------
N/A
Thanks,
Benjamin
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/
-----------------------------------------------------------
(Updated 2012-03-27 21:15:39.505425)
Review request for mesos, Andy Konwinski and Matei Zaharia.
Changes
-------
s/make-conf/make.conf (for real this time)
Summary
-------
Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
Diffs (updated)
-----
README 8166b04
configure.ac 7a0d89f
src/Makefile.am 448410e
src/conf/deploy-env.sh.template a4565e2
src/configurator/configurator.hpp ecd1365
src/configurator/configurator.cpp 1f8134c
src/deploy/deploy-to-slaves 0e10486
src/deploy/mesos-daemon b3144c0
src/deploy/mesos-daemon.sh.in PRE-CREATION
src/deploy/mesos-deploy-env.sh.template PRE-CREATION
src/deploy/mesos-env.sh 14ad67a
src/deploy/mesos-start-cluster.sh.in PRE-CREATION
src/deploy/mesos-start-masters.sh.in PRE-CREATION
src/deploy/mesos-start-slaves.sh.in PRE-CREATION
src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
src/deploy/mesos-stop-masters.sh.in PRE-CREATION
src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
src/deploy/start-masters a450ee0
src/deploy/start-mesos 9e5b922
src/deploy/start-slaves b3b931f
src/deploy/stop-masters f2aa9e2
src/deploy/stop-mesos 679a09e
src/deploy/stop-slaves 141acd4
src/master/main.cpp e88315f
Diff: https://reviews.apache.org/r/4491/diff
Testing
-------
N/A
Thanks,
Benjamin
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/
-----------------------------------------------------------
(Updated 2012-03-27 20:49:32.409803)
Review request for mesos, Andy Konwinski and Matei Zaharia.
Changes
-------
s/mesos-conf/mesos.conf
Summary
-------
Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
Diffs (updated)
-----
README 8166b04
configure.ac 7a0d89f
src/Makefile.am 448410e
src/conf/deploy-env.sh.template a4565e2
src/configurator/configurator.hpp ecd1365
src/configurator/configurator.cpp 1f8134c
src/deploy/deploy-to-slaves 0e10486
src/deploy/mesos-daemon b3144c0
src/deploy/mesos-daemon.sh.in PRE-CREATION
src/deploy/mesos-deploy-env.sh.template PRE-CREATION
src/deploy/mesos-env.sh 14ad67a
src/deploy/mesos-start-cluster.sh.in PRE-CREATION
src/deploy/mesos-start-masters.sh.in PRE-CREATION
src/deploy/mesos-start-slaves.sh.in PRE-CREATION
src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
src/deploy/mesos-stop-masters.sh.in PRE-CREATION
src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
src/deploy/start-masters a450ee0
src/deploy/start-mesos 9e5b922
src/deploy/start-slaves b3b931f
src/deploy/stop-masters f2aa9e2
src/deploy/stop-mesos 679a09e
src/deploy/stop-slaves 141acd4
src/master/main.cpp e88315f
Diff: https://reviews.apache.org/r/4491/diff
Testing
-------
N/A
Thanks,
Benjamin
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/
-----------------------------------------------------------
(Updated 2012-03-27 19:17:54.732360)
Review request for mesos, Andy Konwinski and Matei Zaharia.
Changes
-------
Minor bug fix.
Summary
-------
Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
Diffs (updated)
-----
README 8166b04
configure.ac 7a0d89f
src/Makefile.am 448410e
src/conf/deploy-env.sh.template a4565e2
src/configurator/configurator.hpp ecd1365
src/configurator/configurator.cpp 1f8134c
src/deploy/deploy-to-slaves 0e10486
src/deploy/mesos-daemon b3144c0
src/deploy/mesos-daemon.sh.in PRE-CREATION
src/deploy/mesos-deploy-env.sh.template PRE-CREATION
src/deploy/mesos-env.sh 14ad67a
src/deploy/mesos-start-cluster.sh.in PRE-CREATION
src/deploy/mesos-start-masters.sh.in PRE-CREATION
src/deploy/mesos-start-slaves.sh.in PRE-CREATION
src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
src/deploy/mesos-stop-masters.sh.in PRE-CREATION
src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
src/deploy/start-masters a450ee0
src/deploy/start-mesos 9e5b922
src/deploy/start-slaves b3b931f
src/deploy/stop-masters f2aa9e2
src/deploy/stop-mesos 679a09e
src/deploy/stop-slaves 141acd4
src/master/main.cpp e88315f
Diff: https://reviews.apache.org/r/4491/diff
Testing
-------
N/A
Thanks,
Benjamin
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/
-----------------------------------------------------------
(Updated 2012-03-27 19:14:19.288931)
Review request for mesos, Andy Konwinski and Matei Zaharia.
Changes
-------
Simplified deploy scripts as per review comments.
Summary
-------
Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
Diffs (updated)
-----
README 8166b04
configure.ac 7a0d89f
src/Makefile.am 448410e
src/conf/deploy-env.sh.template a4565e2
src/configurator/configurator.hpp ecd1365
src/configurator/configurator.cpp 1f8134c
src/deploy/deploy-to-slaves 0e10486
src/deploy/mesos-daemon b3144c0
src/deploy/mesos-daemon.sh.in PRE-CREATION
src/deploy/mesos-deploy-env.sh.template PRE-CREATION
src/deploy/mesos-env.sh 14ad67a
src/deploy/mesos-start-cluster.sh.in PRE-CREATION
src/deploy/mesos-start-masters.sh.in PRE-CREATION
src/deploy/mesos-start-slaves.sh.in PRE-CREATION
src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
src/deploy/mesos-stop-masters.sh.in PRE-CREATION
src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
src/deploy/start-masters a450ee0
src/deploy/start-mesos 9e5b922
src/deploy/start-slaves b3b931f
src/deploy/stop-masters f2aa9e2
src/deploy/stop-mesos 679a09e
src/deploy/stop-slaves 141acd4
src/master/main.cpp e88315f
Diff: https://reviews.apache.org/r/4491/diff
Testing
-------
N/A
Thanks,
Benjamin
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/
-----------------------------------------------------------
(Updated 2012-03-27 04:38:04.062133)
Review request for mesos, Andy Konwinski and Matei Zaharia.
Changes
-------
Review comments.
Summary
-------
Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
Diffs (updated)
-----
README 8166b04
configure.ac 7a0d89f
src/Makefile.am 090c07a
src/conf/deploy-env.sh.template a4565e2
src/configurator/configurator.hpp ecd1365
src/deploy/deploy-to-slaves 0e10486
src/deploy/mesos-common-env.sh.template PRE-CREATION
src/deploy/mesos-daemon b3144c0
src/deploy/mesos-daemon.sh.in PRE-CREATION
src/deploy/mesos-deploy-env.sh.template PRE-CREATION
src/deploy/mesos-env.sh c3dfda2
src/deploy/mesos-masters-env.sh.template PRE-CREATION
src/deploy/mesos-slaves-env.sh.template PRE-CREATION
src/deploy/mesos-start-cluster.sh.in PRE-CREATION
src/deploy/mesos-start-masters.sh.in PRE-CREATION
src/deploy/mesos-start-slaves.sh.in PRE-CREATION
src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
src/deploy/mesos-stop-masters.sh.in PRE-CREATION
src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
src/deploy/start-masters a450ee0
src/deploy/start-mesos 9e5b922
src/deploy/start-slaves b3b931f
src/deploy/stop-masters f2aa9e2
src/deploy/stop-mesos 679a09e
src/deploy/stop-slaves 141acd4
src/master/main.cpp f31212b
Diff: https://reviews.apache.org/r/4491/diff
Testing
-------
N/A
Thanks,
Benjamin
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
> On 2012-03-27 04:30:40, Charles Reiss wrote:
> > src/Makefile.am, line 365
> > <https://reviews.apache.org/r/4491/diff/4/?file=96669#file96669line365>
> >
> > .in files should be added to the distribution automatically in the root Makefile, so you don't need this.
Oh, yeah! Thanks!
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6401
-----------------------------------------------------------
On 2012-03-27 04:21:16, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 04:21:16)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.template PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.template PRE-CREATION
> src/deploy/mesos-slaves-env.sh.template PRE-CREATION
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Charles Reiss <wo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/#review6401
-----------------------------------------------------------
src/Makefile.am
<https://reviews.apache.org/r/4491/#comment13988>
.in files should be added to the distribution automatically in the root Makefile, so you don't need this.
- Charles
On 2012-03-27 04:21:16, Benjamin Hindman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4491/
> -----------------------------------------------------------
>
> (Updated 2012-03-27 04:21:16)
>
>
> Review request for mesos, Andy Konwinski and Matei Zaharia.
>
>
> Summary
> -------
>
> Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
>
>
> Diffs
> -----
>
> README 8166b04
> configure.ac 7a0d89f
> src/Makefile.am 090c07a
> src/conf/deploy-env.sh.template a4565e2
> src/configurator/configurator.hpp ecd1365
> src/deploy/deploy-to-slaves 0e10486
> src/deploy/mesos-common-env.sh.template PRE-CREATION
> src/deploy/mesos-daemon b3144c0
> src/deploy/mesos-daemon.sh.in PRE-CREATION
> src/deploy/mesos-deploy-env.sh.template PRE-CREATION
> src/deploy/mesos-env.sh c3dfda2
> src/deploy/mesos-masters-env.sh.template PRE-CREATION
> src/deploy/mesos-slaves-env.sh.template PRE-CREATION
> src/deploy/mesos-start-cluster.sh.in PRE-CREATION
> src/deploy/mesos-start-masters.sh.in PRE-CREATION
> src/deploy/mesos-start-slaves.sh.in PRE-CREATION
> src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
> src/deploy/mesos-stop-masters.sh.in PRE-CREATION
> src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
> src/deploy/start-masters a450ee0
> src/deploy/start-mesos 9e5b922
> src/deploy/start-slaves b3b931f
> src/deploy/stop-masters f2aa9e2
> src/deploy/stop-mesos 679a09e
> src/deploy/stop-slaves 141acd4
> src/master/main.cpp f31212b
>
> Diff: https://reviews.apache.org/r/4491/diff
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Benjamin
>
>
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/
-----------------------------------------------------------
(Updated 2012-03-27 04:21:16.310065)
Review request for mesos, Andy Konwinski and Matei Zaharia.
Changes
-------
Even better quoting.
Summary
-------
Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
Diffs (updated)
-----
README 8166b04
configure.ac 7a0d89f
src/Makefile.am 090c07a
src/conf/deploy-env.sh.template a4565e2
src/configurator/configurator.hpp ecd1365
src/deploy/deploy-to-slaves 0e10486
src/deploy/mesos-common-env.sh.template PRE-CREATION
src/deploy/mesos-daemon b3144c0
src/deploy/mesos-daemon.sh.in PRE-CREATION
src/deploy/mesos-deploy-env.sh.template PRE-CREATION
src/deploy/mesos-env.sh c3dfda2
src/deploy/mesos-masters-env.sh.template PRE-CREATION
src/deploy/mesos-slaves-env.sh.template PRE-CREATION
src/deploy/mesos-start-cluster.sh.in PRE-CREATION
src/deploy/mesos-start-masters.sh.in PRE-CREATION
src/deploy/mesos-start-slaves.sh.in PRE-CREATION
src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
src/deploy/mesos-stop-masters.sh.in PRE-CREATION
src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
src/deploy/start-masters a450ee0
src/deploy/start-mesos 9e5b922
src/deploy/start-slaves b3b931f
src/deploy/stop-masters f2aa9e2
src/deploy/stop-mesos 679a09e
src/deploy/stop-slaves 141acd4
src/master/main.cpp f31212b
Diff: https://reviews.apache.org/r/4491/diff
Testing
-------
N/A
Thanks,
Benjamin
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/
-----------------------------------------------------------
(Updated 2012-03-27 04:16:50.781819)
Review request for mesos, Andy Konwinski and Matei Zaharia.
Changes
-------
Manual testing, found a bug.
Summary
-------
Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
Diffs (updated)
-----
README 8166b04
configure.ac 7a0d89f
src/Makefile.am 090c07a
src/conf/deploy-env.sh.template a4565e2
src/configurator/configurator.hpp ecd1365
src/deploy/deploy-to-slaves 0e10486
src/deploy/mesos-common-env.sh.template PRE-CREATION
src/deploy/mesos-daemon b3144c0
src/deploy/mesos-daemon.sh.in PRE-CREATION
src/deploy/mesos-deploy-env.sh.template PRE-CREATION
src/deploy/mesos-env.sh c3dfda2
src/deploy/mesos-masters-env.sh.template PRE-CREATION
src/deploy/mesos-slaves-env.sh.template PRE-CREATION
src/deploy/mesos-start-cluster.sh.in PRE-CREATION
src/deploy/mesos-start-masters.sh.in PRE-CREATION
src/deploy/mesos-start-slaves.sh.in PRE-CREATION
src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
src/deploy/mesos-stop-masters.sh.in PRE-CREATION
src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
src/deploy/start-masters a450ee0
src/deploy/start-mesos 9e5b922
src/deploy/start-slaves b3b931f
src/deploy/stop-masters f2aa9e2
src/deploy/stop-mesos 679a09e
src/deploy/stop-slaves 141acd4
src/master/main.cpp f31212b
Diff: https://reviews.apache.org/r/4491/diff
Testing
-------
N/A
Thanks,
Benjamin
Re: Review Request: Updated deploy scripts (see description).
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4491/
-----------------------------------------------------------
(Updated 2012-03-27 04:01:46.186251)
Review request for mesos, Andy Konwinski and Matei Zaharia.
Changes
-------
Review comments.
Summary
-------
Updated deploy scripts to work with latest version of Mesos. Eliminated 'deploy-to-slaves' functionality, instead we assume the binaries and libraries will get installed via other means. Updated the semantics of where configuration options are gathered from since a single configuration file was insufficient given conflicting configuration options between binaries (e.g., mesos-master and mesos-slave). Added a section to the README describing these scripts.
Diffs (updated)
-----
README 8166b04
configure.ac 7a0d89f
src/Makefile.am 090c07a
src/conf/deploy-env.sh.template a4565e2
src/configurator/configurator.hpp ecd1365
src/deploy/deploy-to-slaves 0e10486
src/deploy/mesos-common-env.sh.template PRE-CREATION
src/deploy/mesos-daemon b3144c0
src/deploy/mesos-daemon.sh.in PRE-CREATION
src/deploy/mesos-deploy-env.sh.in PRE-CREATION
src/deploy/mesos-env.sh c3dfda2
src/deploy/mesos-masters-env.sh.template PRE-CREATION
src/deploy/mesos-slaves-env.sh.template PRE-CREATION
src/deploy/mesos-start-cluster.sh.in PRE-CREATION
src/deploy/mesos-start-masters.sh.in PRE-CREATION
src/deploy/mesos-start-slaves.sh.in PRE-CREATION
src/deploy/mesos-stop-cluster.sh.in PRE-CREATION
src/deploy/mesos-stop-masters.sh.in PRE-CREATION
src/deploy/mesos-stop-slaves.sh.in PRE-CREATION
src/deploy/start-masters a450ee0
src/deploy/start-mesos 9e5b922
src/deploy/start-slaves b3b931f
src/deploy/stop-masters f2aa9e2
src/deploy/stop-mesos 679a09e
src/deploy/stop-slaves 141acd4
src/master/main.cpp f31212b
Diff: https://reviews.apache.org/r/4491/diff
Testing
-------
N/A
Thanks,
Benjamin