You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Olivier Sallou <ol...@irisa.fr> on 2015/12/08 14:49:49 UTC
GenOuest makes use of Mesos
Hi,
the GenOuest (http://www.genouest.org) academic lab is now using Mesos
in production in its core facility to manage scientists computing tasks
(for bioinformatics)
To do so, we have developed a new mesos framework, GoDocker
(http://www.genouest.org/godocker) to submit batch computing scripts on
premises. It mounts users home directory or other shared resources to
execute jobs in Docker containers, using Mesos as main scheduler.
GoDocker schedules the jobs according to user/groups priorities and
quotas and provides a CLI, a REST web interface and a partial DRMAA
library support. Framewok is open source.
Thanks for adding us to the Mesos fellows ;-)
Regards
Olivier (GenOuest core developer member)
--
Olivier Sallou
IRISA / University of Rennes 1
Campus de Beaulieu, 35000 RENNES - FRANCE
Tel: 02.99.84.71.95
gpg key id: 4096R/326D8438 (keyring.debian.org)
Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
Re: GenOuest makes use of Mesos
Posted by tommy xiao <xi...@gmail.com>.
Cool.
2015-12-08 21:49 GMT+08:00 Olivier Sallou <ol...@irisa.fr>:
> Hi,
> the GenOuest (http://www.genouest.org) academic lab is now using Mesos
> in production in its core facility to manage scientists computing tasks
> (for bioinformatics)
> To do so, we have developed a new mesos framework, GoDocker
> (http://www.genouest.org/godocker) to submit batch computing scripts on
> premises. It mounts users home directory or other shared resources to
> execute jobs in Docker containers, using Mesos as main scheduler.
> GoDocker schedules the jobs according to user/groups priorities and
> quotas and provides a CLI, a REST web interface and a partial DRMAA
> library support. Framewok is open source.
>
> Thanks for adding us to the Mesos fellows ;-)
>
> Regards
>
> Olivier (GenOuest core developer member)
>
> --
> Olivier Sallou
> IRISA / University of Rennes 1
> Campus de Beaulieu, 35000 RENNES - FRANCE
> Tel: 02.99.84.71.95
>
> gpg key id: 4096R/326D8438 (keyring.debian.org)
> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
>
>
>
--
Deshi Xiao
Twitter: xds2000
E-mail: xiaods(AT)gmail.com
Re: GenOuest makes use of Mesos
Posted by Olivier Sallou <ol...@irisa.fr>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On 12/09/2015 03:41 PM, Joao Ribeiro wrote:
> Hi Oliver,
>
> GoDocker looks very promising. I have been looking for a a framework
to substitute AWS Lambda for batch jobs scheduling based on docker for
flexibility. That said, I would like to know your thought on this to
better understand if GoDocker could do the job.
> A couple of things I would need to know better:
>
> - How fast can GoDocker launch a job? Can GoDocker pre populate all
nodes with the container and files needed for the job to achieve <500ms
launches? I realize this depends a lot on the Job size, just looking for
a generic view before getting into full testing.
Fast... depends on number of pending jobs. There is at first a scheduler
based on user/project previous usage, priorities and quota checks to
reorder the jobs before assigning them to nodes. The more you have
pending jobs, the more it will take for this step. If you select a FIFO
scheduler instead of Fair share scheduler it is of course quite more
faster. Then it depends also on Mesos offers (we wait for a mesos offer
to assign jobs to nodes).
We cannot prepopulate containers to exec scripts in containers
afterwards. We always start a new container which has a dedicated
directory to write some output files (besides optional home directory
etc...). Once container is started on node, we cannot access anymore the
container to "submit" tasks in the container, though I think this could
be managed with a special Executor in Mesos instead of using the default
Executor.
The logic is:
get pending jobs
reorder according to prio/quota etc...
get mesos offer and assign jobs to mesos nodes
one (or many) watcher checks job status
if job over, update job status and meta-data
one possiblity of course would be to execute in the container a program
that fetches job commands or files to use from a db/queue available
elsewhere, and exit when queue is empty. This speeds up the process of
many files, but you won't get details "per job" .
> - Is it possible to extend GoDocker to enable usage metrics reports? This could be achieved by
simply logging a few metrics relating to the user and job.
For job metrics, GoDocker makes use of cAdvisor for live monitoring.
cAdvisor also has a prometheus endpoint (that we use in our premises) to
get live and past monitoring of containers.
GoDocker provides itself a prometheus endpoint for global statistics. We
also store in job meta data its start/end time, requested cpu/ram, and
real job duration.
Olivier
>
>
> { name: “*João Ribeiro*”,
> email: “jonnybgod@gmail.com <ma...@gmail.com>”,
> phone: “+351 916 002 675 <tel:+351916002675>”,
> skype: “jmn_ribeiro <skype:jmn_ribeiro?add>”,
> linkedin: “https://pt.linkedin.com/in/jmnribeiro”
> github: “https://github.com/JonnyBGod” }
>
>> On 09 Dec 2015, at 10:22, Elouan Keryell-Even
<elouan.keryell@gmail.com <ma...@gmail.com>> wrote:
>>
>> Thanks for the insight, very interesting indeed.
>>
>> Elouan Keryell-Even
>>
>> Software Engineer @ Atos Integration
>>
>> Toulouse, France
>>
>>
>> 2015-12-09 11:08 GMT+01:00 Olivier Sallou <olivier.sallou@irisa.fr
<ma...@irisa.fr>>:
>>
>>
>>
>> -------------------------
>>
>> *De: *"Elouan Keryell-Even" <elouan.keryell@gmail.com
<ma...@gmail.com>>
>> *À: *user@mesos.apache.org <ma...@mesos.apache.org>
>> *Envoyé: *Mercredi 9 Décembre 2015 10:48:10
>> *Objet: *Re: GenOuest makes use of Mesos
>>
>> Hi Olivier,
>>
>> I've just read the presentation on your project's webpage,
and it seems cool! I'm curious about the following mentioned feature:
"Optional mount of user home or other shared directories in container".
Does your framework take care of remotely copying the home directory
onto the node where the container is going to run, or does the directory
have to be already available (as a shared directory for example).
>>
>> We focus to "replace" classical computing cluster frameworks like
Sun Grid Engine. So home directories etc... are shared among nodes (nfs,
etc...), like in a usual cluster environment.
>> The mount of "user home dir" is based on an Auth/ACL
plugin/configuration. Admin specifies the available mounts for all
users, or on a project basis. Then the Auth/ACL plugin maps those
volumes to real available directories (if using the LDAP auth plugin,
then get homeDirectory from LDAP and add as a volume to container).
>>
>> Otherwise, we are currently using framework Chronos for our
Batch oriented containers, and your framework seems to fit the same
spot. If you have some experience with Chronos, I'd be interested in a
brief comparison of both frameworks. From what I understand, GODocker
offers scheduling policy's customization, which I don't think Chronos does.
>>
>> There is indeed scheduling algorithms plugin mechanism. An
interesting one is the fair-share policy imlpementation, where tasks are
scheduler (ordered) based on previous user/project consumption.
>>
>> Difference with Chronos, is Chronos is made for cron like jobs.
GODocker is a batch submission tool. User specifies the computing script
he wants to execute and it is executed "immediatly". The scheduling is a
matter of putting priorities when executing jobs (if users submits 1000
jobs but I have only 100 slots available). We focus on users submitting
"many" jobs or many users submitting jobs, each job could be a computing
task of a few seconds or several days.
>>
>>
>>
>> Olivier
>>
>>
>> Thanks for your attention,
>>
>> Elouan Keryell-Even
>>
>> Software Engineer @ Atos Integration
>>
>> Toulouse, France
>>
>>
>> 2015-12-08 14:49 GMT+01:00 Olivier Sallou
<olivier.sallou@irisa.fr <ma...@irisa.fr>>:
>>
>> Hi,
>> the GenOuest (http://www.genouest.org
<http://www.genouest.org/>) academic lab is now using Mesos
>> in production in its core facility to manage scientists
computing tasks
>> (for bioinformatics)
>> To do so, we have developed a new mesos framework, GoDocker
>> (http://www.genouest.org/godocker) to submit batch
computing scripts on
>> premises. It mounts users home directory or other shared
resources to
>> execute jobs in Docker containers, using Mesos as main
scheduler.
>> GoDocker schedules the jobs according to user/groups
priorities and
>> quotas and provides a CLI, a REST web interface and a
partial DRMAA
>> library support. Framewok is open source.
>>
>> Thanks for adding us to the Mesos fellows ;-)
>>
>> Regards
>>
>> Olivier (GenOuest core developer member)
>>
>> --
>> Olivier Sallou
>> IRISA / University of Rennes 1
>> Campus de Beaulieu, 35000 RENNES - FRANCE
>> Tel: 02.99.84.71.95 <tel:02.99.84.71.95>
>>
>> gpg key id: 4096R/326D8438 (keyring.debian.org
<http://keyring.debian.org/>)
>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC
68DB 326D 8438
>>
>>
>>
>>
>>
>
- --
Olivier Sallou
IRISA / University of Rennes 1
Campus de Beaulieu, 35000 RENNES - FRANCE
Tel: 02.99.84.71.95
gpg key id: 4096R/326D8438 (keyring.debian.org)
Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iQIcBAEBCAAGBQJWaV0hAAoJEHjcaNsybYQ4WMIP+wbL9AbiYJzNKbLVT7NJvHPn
f2/gZoxefjZxt1qLhBGtu9LglhptAyliZ3lYe7g8LGdUNzsHcmBXSDwk2lMNjYdj
IT/FbWLXU7KeKcUixM2SG7uw3nrtrkwBOpYa2lPEDwPsSKZlbTsyaKsSKDJlO7gb
EQBXKyRVtiGZOnt/WbgCNT1+EfwzN07DfYd9wQnVXt2o5ouIuYc9JRNMYBYYjRnP
rBFjkE4YoquPC0W7MV62I7C/C4HsEiorlvm1tbPGv0k20Ep7+Nv6LeY/05a+n0Gv
mImBNvzjELt3KNYhaZb22o5aGDdSSKeOYt3hs83T7AoSkLThHNWwDdRmB4d/BwxP
ieFWTtprTl+gPf3zV4ERvlGQKap+agNY9BIgZnrEV9XBm+qR/tgo0jQ8cV9zKSJT
Di7mmDSDozdEci7TFbpIl6WrUjet84Cfjrfr7rM7wmgXWmYNLNZoeXNS03w426Cz
Zwy4b8CJuB5g1IIsaqOMLoUVB73AFtRH9OTEnz07dgio3C/PwwkZPaMt8nunZKoV
jA7cCOufQjBLpX9kwscasYplAmwwkFjrR6chEPu4rFBPvuR31ce8gTmkbvo45m9N
nsIZ9TNZB+aXFNiSa9GSnCtNAfMxmh02bvGbSJ2G3ILKUQqLeQBpXk0OL5Cz9G4O
z3nxqyzo438A2mDK4ihK
=m7/b
-----END PGP SIGNATURE-----
Re: GenOuest makes use of Mesos
Posted by Joao Ribeiro <jo...@gmail.com>.
Hi Oliver,
GoDocker looks very promising. I have been looking for a a framework to substitute AWS Lambda for batch jobs scheduling based on docker for flexibility. That said, I would like to know your thought on this to better understand if GoDocker could do the job.
A couple of things I would need to know better:
- How fast can GoDocker launch a job? Can GoDocker pre populate all nodes with the container and files needed for the job to achieve <500ms launches? I realize this depends a lot on the Job size, just looking for a generic view before getting into full testing.
- Is it possible to extend GoDocker to enable usage metrics reports? This could be achieved by simply logging a few metrics relating to the user and job.
{ name: “João Ribeiro”,
email: “jonnybgod@gmail.com <ma...@gmail.com>”,
phone: “+351 916 002 675 <tel:+351916002675>”,
skype: “jmn_ribeiro <skype:jmn_ribeiro?add>”,
linkedin: “https://pt.linkedin.com/in/jmnribeiro <https://pt.linkedin.com/in/jmnribeiro>”
github: “https://github.com/JonnyBGod <https://github.com/JonnyBGod>” }
> On 09 Dec 2015, at 10:22, Elouan Keryell-Even <el...@gmail.com> wrote:
>
> Thanks for the insight, very interesting indeed.
>
> Elouan Keryell-Even
> Software Engineer @ Atos Integration
>
> Toulouse, France
>
>
> 2015-12-09 11:08 GMT+01:00 Olivier Sallou <olivier.sallou@irisa.fr <ma...@irisa.fr>>:
>
>
> De: "Elouan Keryell-Even" <elouan.keryell@gmail.com <ma...@gmail.com>>
> À: user@mesos.apache.org <ma...@mesos.apache.org>
> Envoyé: Mercredi 9 Décembre 2015 10:48:10
> Objet: Re: GenOuest makes use of Mesos
>
> Hi Olivier,
>
> I've just read the presentation on your project's webpage, and it seems cool! I'm curious about the following mentioned feature: "Optional mount of user home or other shared directories in container". Does your framework take care of remotely copying the home directory onto the node where the container is going to run, or does the directory have to be already available (as a shared directory for example).
> We focus to "replace" classical computing cluster frameworks like Sun Grid Engine. So home directories etc... are shared among nodes (nfs, etc...), like in a usual cluster environment.
> The mount of "user home dir" is based on an Auth/ACL plugin/configuration. Admin specifies the available mounts for all users, or on a project basis. Then the Auth/ACL plugin maps those volumes to real available directories (if using the LDAP auth plugin, then get homeDirectory from LDAP and add as a volume to container).
> Otherwise, we are currently using framework Chronos for our Batch oriented containers, and your framework seems to fit the same spot. If you have some experience with Chronos, I'd be interested in a brief comparison of both frameworks. From what I understand, GODocker offers scheduling policy's customization, which I don't think Chronos does.
> There is indeed scheduling algorithms plugin mechanism. An interesting one is the fair-share policy imlpementation, where tasks are scheduler (ordered) based on previous user/project consumption.
>
> Difference with Chronos, is Chronos is made for cron like jobs. GODocker is a batch submission tool. User specifies the computing script he wants to execute and it is executed "immediatly". The scheduling is a matter of putting priorities when executing jobs (if users submits 1000 jobs but I have only 100 slots available). We focus on users submitting "many" jobs or many users submitting jobs, each job could be a computing task of a few seconds or several days.
>
>
>
> Olivier
>
>
> Thanks for your attention,
>
> Elouan Keryell-Even
> Software Engineer @ Atos Integration
>
> Toulouse, France
>
>
> 2015-12-08 14:49 GMT+01:00 Olivier Sallou <olivier.sallou@irisa.fr <ma...@irisa.fr>>:
> Hi,
> the GenOuest (http://www.genouest.org <http://www.genouest.org/>) academic lab is now using Mesos
> in production in its core facility to manage scientists computing tasks
> (for bioinformatics)
> To do so, we have developed a new mesos framework, GoDocker
> (http://www.genouest.org/godocker <http://www.genouest.org/godocker>) to submit batch computing scripts on
> premises. It mounts users home directory or other shared resources to
> execute jobs in Docker containers, using Mesos as main scheduler.
> GoDocker schedules the jobs according to user/groups priorities and
> quotas and provides a CLI, a REST web interface and a partial DRMAA
> library support. Framewok is open source.
>
> Thanks for adding us to the Mesos fellows ;-)
>
> Regards
>
> Olivier (GenOuest core developer member)
>
> --
> Olivier Sallou
> IRISA / University of Rennes 1
> Campus de Beaulieu, 35000 RENNES - FRANCE
> Tel: 02.99.84.71.95 <tel:02.99.84.71.95>
>
> gpg key id: 4096R/326D8438 (keyring.debian.org <http://keyring.debian.org/>)
> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
>
>
>
>
>
Re: GenOuest makes use of Mesos
Posted by Elouan Keryell-Even <el...@gmail.com>.
Thanks for the insight, very interesting indeed.
Elouan Keryell-Even
Software Engineer @ Atos Integration
Toulouse, France
2015-12-09 11:08 GMT+01:00 Olivier Sallou <ol...@irisa.fr>:
>
>
> ------------------------------
>
> *De: *"Elouan Keryell-Even" <el...@gmail.com>
> *À: *user@mesos.apache.org
> *Envoyé: *Mercredi 9 Décembre 2015 10:48:10
> *Objet: *Re: GenOuest makes use of Mesos
>
> Hi Olivier,
>
> I've just read the presentation on your project's webpage, and it seems
> cool! I'm curious about the following mentioned feature: "Optional mount of
> user home or other shared directories in container". Does your framework
> take care of remotely copying the home directory onto the node where the
> container is going to run, or does the directory have to be already
> available (as a shared directory for example).
>
> We focus to "replace" classical computing cluster frameworks like Sun Grid
> Engine. So home directories etc... are shared among nodes (nfs, etc...),
> like in a usual cluster environment.
> The mount of "user home dir" is based on an Auth/ACL plugin/configuration.
> Admin specifies the available mounts for all users, or on a project basis.
> Then the Auth/ACL plugin maps those volumes to real available directories
> (if using the LDAP auth plugin, then get homeDirectory from LDAP and add as
> a volume to container).
>
> Otherwise, we are currently using framework Chronos for our Batch oriented
> containers, and your framework seems to fit the same spot. If you have some
> experience with Chronos, I'd be interested in a brief comparison of both
> frameworks. From what I understand, GODocker offers scheduling policy's
> customization, which I don't think Chronos does.
>
> There is indeed scheduling algorithms plugin mechanism. An interesting one
> is the fair-share policy imlpementation, where tasks are scheduler
> (ordered) based on previous user/project consumption.
>
> Difference with Chronos, is Chronos is made for cron like jobs. GODocker
> is a batch submission tool. User specifies the computing script he wants to
> execute and it is executed "immediatly". The scheduling is a matter of
> putting priorities when executing jobs (if users submits 1000 jobs but I
> have only 100 slots available). We focus on users submitting "many" jobs or
> many users submitting jobs, each job could be a computing task of a few
> seconds or several days.
>
>
>
> Olivier
>
>
> Thanks for your attention,
>
> Elouan Keryell-Even
>
> Software Engineer @ Atos Integration
>
> Toulouse, France
>
> 2015-12-08 14:49 GMT+01:00 Olivier Sallou <ol...@irisa.fr>:
>
>> Hi,
>> the GenOuest (http://www.genouest.org) academic lab is now using Mesos
>> in production in its core facility to manage scientists computing tasks
>> (for bioinformatics)
>> To do so, we have developed a new mesos framework, GoDocker
>> (http://www.genouest.org/godocker) to submit batch computing scripts on
>> premises. It mounts users home directory or other shared resources to
>> execute jobs in Docker containers, using Mesos as main scheduler.
>> GoDocker schedules the jobs according to user/groups priorities and
>> quotas and provides a CLI, a REST web interface and a partial DRMAA
>> library support. Framewok is open source.
>>
>> Thanks for adding us to the Mesos fellows ;-)
>>
>> Regards
>>
>> Olivier (GenOuest core developer member)
>>
>> --
>> Olivier Sallou
>> IRISA / University of Rennes 1
>> Campus de Beaulieu, 35000 RENNES - FRANCE
>> Tel: 02.99.84.71.95
>>
>> gpg key id: 4096R/326D8438 (keyring.debian.org)
>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
>>
>>
>>
>
>
Re: GenOuest makes use of Mesos
Posted by Olivier Sallou <ol...@irisa.fr>.
----- Mail original -----
> De: "Elouan Keryell-Even" <el...@gmail.com>
> À: user@mesos.apache.org
> Envoyé: Mercredi 9 Décembre 2015 10:48:10
> Objet: Re: GenOuest makes use of Mesos
> Hi Olivier,
> I've just read the presentation on your project's webpage, and it seems cool!
> I'm curious about the following mentioned feature: "Optional mount of user
> home or other shared directories in container". Does your framework take
> care of remotely copying the home directory onto the node where the
> container is going to run, or does the directory have to be already
> available (as a shared directory for example).
We focus to "replace" classical computing cluster frameworks like Sun Grid Engine. So home directories etc... are shared among nodes (nfs, etc...), like in a usual cluster environment.
The mount of "user home dir" is based on an Auth/ACL plugin/configuration. Admin specifies the available mounts for all users, or on a project basis. Then the Auth/ACL plugin maps those volumes to real available directories (if using the LDAP auth plugin, then get homeDirectory from LDAP and add as a volume to container).
> Otherwise, we are currently using framework Chronos for our Batch oriented
> containers, and your framework seems to fit the same spot. If you have some
> experience with Chronos, I'd be interested in a brief comparison of both
> frameworks. From what I understand, GODocker offers scheduling policy's
> customization, which I don't think Chronos does.
There is indeed scheduling algorithms plugin mechanism. An interesting one is the fair-share policy imlpementation, where tasks are scheduler (ordered) based on previous user/project consumption.
Difference with Chronos, is Chronos is made for cron like jobs. GODocker is a batch submission tool. User specifies the computing script he wants to execute and it is executed "immediatly". The scheduling is a matter of putting priorities when executing jobs (if users submits 1000 jobs but I have only 100 slots available). We focus on users submitting "many" jobs or many users submitting jobs, each job could be a computing task of a few seconds or several days.
Olivier
> Thanks for your attention,
> Elouan Keryell-Even
> Software Engineer @ Atos Integration
> Toulouse, France
> 2015-12-08 14:49 GMT+01:00 Olivier Sallou < olivier.sallou@irisa.fr > :
> > Hi,
>
> > the GenOuest ( http://www.genouest.org ) academic lab is now using Mesos
>
> > in production in its core facility to manage scientists computing tasks
>
> > (for bioinformatics)
>
> > To do so, we have developed a new mesos framework, GoDocker
>
> > ( http://www.genouest.org/godocker ) to submit batch computing scripts on
>
> > premises. It mounts users home directory or other shared resources to
>
> > execute jobs in Docker containers, using Mesos as main scheduler.
>
> > GoDocker schedules the jobs according to user/groups priorities and
>
> > quotas and provides a CLI, a REST web interface and a partial DRMAA
>
> > library support. Framewok is open source.
>
> > Thanks for adding us to the Mesos fellows ;-)
>
> > Regards
>
> > Olivier (GenOuest core developer member)
>
> > --
>
> > Olivier Sallou
>
> > IRISA / University of Rennes 1
>
> > Campus de Beaulieu, 35000 RENNES - FRANCE
>
> > Tel: 02.99.84.71.95
>
> > gpg key id: 4096R/326D8438 ( keyring.debian.org )
>
> > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
>
Re: GenOuest makes use of Mesos
Posted by Elouan Keryell-Even <el...@gmail.com>.
Hi Olivier,
I've just read the presentation on your project's webpage, and it seems
cool! I'm curious about the following mentioned feature: "Optional mount of
user home or other shared directories in container". Does your framework
take care of remotely copying the home directory onto the node where the
container is going to run, or does the directory have to be already
available (as a shared directory for example).
Otherwise, we are currently using framework Chronos for our Batch oriented
containers, and your framework seems to fit the same spot. If you have some
experience with Chronos, I'd be interested in a brief comparison of both
frameworks. From what I understand, GODocker offers scheduling policy's
customization, which I don't think Chronos does.
Thanks for your attention,
Elouan Keryell-Even
Software Engineer @ Atos Integration
Toulouse, France
2015-12-08 14:49 GMT+01:00 Olivier Sallou <ol...@irisa.fr>:
> Hi,
> the GenOuest (http://www.genouest.org) academic lab is now using Mesos
> in production in its core facility to manage scientists computing tasks
> (for bioinformatics)
> To do so, we have developed a new mesos framework, GoDocker
> (http://www.genouest.org/godocker) to submit batch computing scripts on
> premises. It mounts users home directory or other shared resources to
> execute jobs in Docker containers, using Mesos as main scheduler.
> GoDocker schedules the jobs according to user/groups priorities and
> quotas and provides a CLI, a REST web interface and a partial DRMAA
> library support. Framewok is open source.
>
> Thanks for adding us to the Mesos fellows ;-)
>
> Regards
>
> Olivier (GenOuest core developer member)
>
> --
> Olivier Sallou
> IRISA / University of Rennes 1
> Campus de Beaulieu, 35000 RENNES - FRANCE
> Tel: 02.99.84.71.95
>
> gpg key id: 4096R/326D8438 (keyring.debian.org)
> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
>
>
>
Re: GenOuest makes use of Mesos
Posted by Benjamin Mahler <bm...@apache.org>.
Pushed these changes, thanks Olivier.
On Tue, Jan 5, 2016 at 4:35 AM, Olivier Sallou <ol...@irisa.fr>
wrote:
>
>
> On 01/05/2016 02:48 AM, Benjamin Mahler wrote:
>
> Up to you, would you mind sending a pull request (easier for you since
> this is a small change), or a reviewboard request to get yourself added?
>
> Sure, I have made the pull request, thanks
>
>
> On Wed, Dec 9, 2015 at 12:16 AM, Olivier Sallou <ol...@irisa.fr>
> wrote:
>
>>
>>
>> ------------------------------
>>
>> *De: *"Benjamin Mahler" <be...@gmail.com>
>> *À: *user@mesos.apache.org
>> *Envoyé: *Mardi 8 Décembre 2015 22:33:40
>> *Objet: *Re: GenOuest makes use of Mesos
>>
>> Great to hear Olivier, would you like to be added to the powered by mesos
>> list?
>>
>> https://github.com/apache/mesos/blob/master/docs/powered-by-mesos.md
>>
>> Sure
>>
>> Should the framework be also added to
>> <https://github.com/apache/mesos/blob/master/docs/frameworks.md>
>> https://github.com/apache/mesos/blob/master/docs/frameworks.md ?
>>
>> Olivier
>>
>>
>>
>> On Tue, Dec 8, 2015 at 1:04 PM, Arunabha Ghosh < <ar...@gmail.com>
>> arunabha.gh@gmail.com> wrote:
>>
>>> Welcome to the community, Oliver.
>>>
>>> On Tue, Dec 8, 2015 at 5:49 AM, Olivier Sallou <
>>> <ol...@irisa.fr> wrote:
>>>
>>>> Hi,
>>>> the GenOuest ( <http://www.genouest.org>http://www.genouest.org)
>>>> academic lab is now using Mesos
>>>> in production in its core facility to manage scientists computing tasks
>>>> (for bioinformatics)
>>>> To do so, we have developed a new mesos framework, GoDocker
>>>> (http://www.genouest.org/godocker) to submit batch computing scripts on
>>>> premises. It mounts users home directory or other shared resources to
>>>> execute jobs in Docker containers, using Mesos as main scheduler.
>>>> GoDocker schedules the jobs according to user/groups priorities and
>>>> quotas and provides a CLI, a REST web interface and a partial DRMAA
>>>> library support. Framewok is open source.
>>>>
>>>> Thanks for adding us to the Mesos fellows ;-)
>>>>
>>>> Regards
>>>>
>>>> Olivier (GenOuest core developer member)
>>>>
>>>> --
>>>> Olivier Sallou
>>>> IRISA / University of Rennes 1
>>>> Campus de Beaulieu, 35000 RENNES - FRANCE
>>>> Tel: 02.99.84.71.95
>>>>
>>>> gpg key id: 4096R/326D8438 (keyring.debian.org)
>>>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
>>>>
>>>>
>>>>
>>>
>>
>>
>
> --
> Olivier Sallou
> IRISA / University of Rennes 1
> Campus de Beaulieu, 35000 RENNES - FRANCE
> Tel: 02.99.84.71.95
>
> gpg key id: 4096R/326D8438 (keyring.debian.org)
> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
>
>
>
Re: GenOuest makes use of Mesos
Posted by Olivier Sallou <ol...@irisa.fr>.
On 01/05/2016 02:48 AM, Benjamin Mahler wrote:
> Up to you, would you mind sending a pull request (easier for you since
> this is a small change), or a reviewboard request to get yourself added?
Sure, I have made the pull request, thanks
>
> On Wed, Dec 9, 2015 at 12:16 AM, Olivier Sallou
> <olivier.sallou@irisa.fr <ma...@irisa.fr>> wrote:
>
>
>
> ------------------------------------------------------------------------
>
> *De: *"Benjamin Mahler" <benjamin.mahler@gmail.com
> <ma...@gmail.com>>
> *À: *user@mesos.apache.org <ma...@mesos.apache.org>
> *Envoyé: *Mardi 8 Décembre 2015 22:33:40
> *Objet: *Re: GenOuest makes use of Mesos
>
> Great to hear Olivier, would you like to be added to the
> powered by mesos list?
>
> https://github.com/apache/mesos/blob/master/docs/powered-by-mesos.md
>
> Sure
>
> Should the framework be also added
> to https://github.com/apache/mesos/blob/master/docs/frameworks.md ?
>
> Olivier
>
>
>
> On Tue, Dec 8, 2015 at 1:04 PM, Arunabha Ghosh
> <arunabha.gh@gmail.com <ma...@gmail.com>> wrote:
>
> Welcome to the community, Oliver.
>
> On Tue, Dec 8, 2015 at 5:49 AM, Olivier Sallou
> <olivier.sallou@irisa.fr <ma...@irisa.fr>>
> wrote:
>
> Hi,
> the GenOuest (http://www.genouest.org) academic lab is
> now using Mesos
> in production in its core facility to manage
> scientists computing tasks
> (for bioinformatics)
> To do so, we have developed a new mesos framework,
> GoDocker
> (http://www.genouest.org/godocker) to submit batch
> computing scripts on
> premises. It mounts users home directory or other
> shared resources to
> execute jobs in Docker containers, using Mesos as main
> scheduler.
> GoDocker schedules the jobs according to user/groups
> priorities and
> quotas and provides a CLI, a REST web interface and a
> partial DRMAA
> library support. Framewok is open source.
>
> Thanks for adding us to the Mesos fellows ;-)
>
> Regards
>
> Olivier (GenOuest core developer member)
>
> --
> Olivier Sallou
> IRISA / University of Rennes 1
> Campus de Beaulieu, 35000 RENNES - FRANCE
> Tel: 02.99.84.71.95
>
> gpg key id: 4096R/326D8438 (keyring.debian.org
> <http://keyring.debian.org>)
> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC
> 68DB 326D 8438
>
>
>
>
>
>
--
Olivier Sallou
IRISA / University of Rennes 1
Campus de Beaulieu, 35000 RENNES - FRANCE
Tel: 02.99.84.71.95
gpg key id: 4096R/326D8438 (keyring.debian.org)
Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
Re: GenOuest makes use of Mesos
Posted by Benjamin Mahler <bm...@apache.org>.
Up to you, would you mind sending a pull request (easier for you since this
is a small change), or a reviewboard request to get yourself added?
On Wed, Dec 9, 2015 at 12:16 AM, Olivier Sallou <ol...@irisa.fr>
wrote:
>
>
> ------------------------------
>
> *De: *"Benjamin Mahler" <be...@gmail.com>
> *À: *user@mesos.apache.org
> *Envoyé: *Mardi 8 Décembre 2015 22:33:40
> *Objet: *Re: GenOuest makes use of Mesos
>
> Great to hear Olivier, would you like to be added to the powered by mesos
> list?
>
> https://github.com/apache/mesos/blob/master/docs/powered-by-mesos.md
>
> Sure
>
> Should the framework be also added to
> https://github.com/apache/mesos/blob/master/docs/frameworks.md ?
>
> Olivier
>
>
>
> On Tue, Dec 8, 2015 at 1:04 PM, Arunabha Ghosh <ar...@gmail.com>
> wrote:
>
>> Welcome to the community, Oliver.
>>
>> On Tue, Dec 8, 2015 at 5:49 AM, Olivier Sallou <ol...@irisa.fr>
>> wrote:
>>
>>> Hi,
>>> the GenOuest (http://www.genouest.org) academic lab is now using Mesos
>>> in production in its core facility to manage scientists computing tasks
>>> (for bioinformatics)
>>> To do so, we have developed a new mesos framework, GoDocker
>>> (http://www.genouest.org/godocker) to submit batch computing scripts on
>>> premises. It mounts users home directory or other shared resources to
>>> execute jobs in Docker containers, using Mesos as main scheduler.
>>> GoDocker schedules the jobs according to user/groups priorities and
>>> quotas and provides a CLI, a REST web interface and a partial DRMAA
>>> library support. Framewok is open source.
>>>
>>> Thanks for adding us to the Mesos fellows ;-)
>>>
>>> Regards
>>>
>>> Olivier (GenOuest core developer member)
>>>
>>> --
>>> Olivier Sallou
>>> IRISA / University of Rennes 1
>>> Campus de Beaulieu, 35000 RENNES - FRANCE
>>> Tel: 02.99.84.71.95
>>>
>>> gpg key id: 4096R/326D8438 (keyring.debian.org)
>>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
>>>
>>>
>>>
>>
>
>
Re: GenOuest makes use of Mesos
Posted by Olivier Sallou <ol...@irisa.fr>.
----- Mail original -----
> De: "Benjamin Mahler" <be...@gmail.com>
> À: user@mesos.apache.org
> Envoyé: Mardi 8 Décembre 2015 22:33:40
> Objet: Re: GenOuest makes use of Mesos
> Great to hear Olivier, would you like to be added to the powered by mesos
> list?
> https://github.com/apache/mesos/blob/master/docs/powered-by-mesos.md
Sure
Should the framework be also added to https://github.com/apache/mesos/blob/master/docs/frameworks.md ?
Olivier
> On Tue, Dec 8, 2015 at 1:04 PM, Arunabha Ghosh < arunabha.gh@gmail.com >
> wrote:
> > Welcome to the community, Oliver.
>
> > On Tue, Dec 8, 2015 at 5:49 AM, Olivier Sallou < olivier.sallou@irisa.fr >
> > wrote:
>
> > > Hi,
> >
>
> > > the GenOuest ( http://www.genouest.org ) academic lab is now using Mesos
> >
>
> > > in production in its core facility to manage scientists computing tasks
> >
>
> > > (for bioinformatics)
> >
>
> > > To do so, we have developed a new mesos framework, GoDocker
> >
>
> > > ( http://www.genouest.org/godocker ) to submit batch computing scripts on
> >
>
> > > premises. It mounts users home directory or other shared resources to
> >
>
> > > execute jobs in Docker containers, using Mesos as main scheduler.
> >
>
> > > GoDocker schedules the jobs according to user/groups priorities and
> >
>
> > > quotas and provides a CLI, a REST web interface and a partial DRMAA
> >
>
> > > library support. Framewok is open source.
> >
>
> > > Thanks for adding us to the Mesos fellows ;-)
> >
>
> > > Regards
> >
>
> > > Olivier (GenOuest core developer member)
> >
>
> > > --
> >
>
> > > Olivier Sallou
> >
>
> > > IRISA / University of Rennes 1
> >
>
> > > Campus de Beaulieu, 35000 RENNES - FRANCE
> >
>
> > > Tel: 02.99.84.71.95
> >
>
> > > gpg key id: 4096R/326D8438 ( keyring.debian.org )
> >
>
> > > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
> >
>
Re: GenOuest makes use of Mesos
Posted by Benjamin Mahler <be...@gmail.com>.
Great to hear Olivier, would you like to be added to the powered by mesos
list?
https://github.com/apache/mesos/blob/master/docs/powered-by-mesos.md
On Tue, Dec 8, 2015 at 1:04 PM, Arunabha Ghosh <ar...@gmail.com>
wrote:
> Welcome to the community, Oliver.
>
> On Tue, Dec 8, 2015 at 5:49 AM, Olivier Sallou <ol...@irisa.fr>
> wrote:
>
>> Hi,
>> the GenOuest (http://www.genouest.org) academic lab is now using Mesos
>> in production in its core facility to manage scientists computing tasks
>> (for bioinformatics)
>> To do so, we have developed a new mesos framework, GoDocker
>> (http://www.genouest.org/godocker) to submit batch computing scripts on
>> premises. It mounts users home directory or other shared resources to
>> execute jobs in Docker containers, using Mesos as main scheduler.
>> GoDocker schedules the jobs according to user/groups priorities and
>> quotas and provides a CLI, a REST web interface and a partial DRMAA
>> library support. Framewok is open source.
>>
>> Thanks for adding us to the Mesos fellows ;-)
>>
>> Regards
>>
>> Olivier (GenOuest core developer member)
>>
>> --
>> Olivier Sallou
>> IRISA / University of Rennes 1
>> Campus de Beaulieu, 35000 RENNES - FRANCE
>> Tel: 02.99.84.71.95
>>
>> gpg key id: 4096R/326D8438 (keyring.debian.org)
>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
>>
>>
>>
>
Re: GenOuest makes use of Mesos
Posted by Arunabha Ghosh <ar...@gmail.com>.
Welcome to the community, Oliver.
On Tue, Dec 8, 2015 at 5:49 AM, Olivier Sallou <ol...@irisa.fr>
wrote:
> Hi,
> the GenOuest (http://www.genouest.org) academic lab is now using Mesos
> in production in its core facility to manage scientists computing tasks
> (for bioinformatics)
> To do so, we have developed a new mesos framework, GoDocker
> (http://www.genouest.org/godocker) to submit batch computing scripts on
> premises. It mounts users home directory or other shared resources to
> execute jobs in Docker containers, using Mesos as main scheduler.
> GoDocker schedules the jobs according to user/groups priorities and
> quotas and provides a CLI, a REST web interface and a partial DRMAA
> library support. Framewok is open source.
>
> Thanks for adding us to the Mesos fellows ;-)
>
> Regards
>
> Olivier (GenOuest core developer member)
>
> --
> Olivier Sallou
> IRISA / University of Rennes 1
> Campus de Beaulieu, 35000 RENNES - FRANCE
> Tel: 02.99.84.71.95
>
> gpg key id: 4096R/326D8438 (keyring.debian.org)
> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
>
>
>