You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by DhilipKumar Sankaranarayanan <s....@gmail.com> on 2016/07/03 15:44:52 UTC
A Redis Framework for Apache Mesos
Hello All,
We have built a framework for provisioning redis-servers in Apache Mesos
enabled infrastructure. It would be awesome to get communities
feedback. While mesos ecosystem is strengthening its capability in storage
layer, redis could be an addition to the lineup. This is primarily
intended for providers who would like to host redis as a service in their
infrastructure.
There is an elaborate README about the project which should help in setting
it up: https://github.com/mesos/mr-redis. Please let us know if there is
anything missing in the documentation by raising a PR or opening an issue
or even writing to us.
This project is also packaged as mr-redis with DCOS so should be pretty
straight forward to install it via DCOS CLI or DCOS GUI.
(A Step by Step guide is provided in the README for your convenience)
Salient Features of this project include:
1) Create multiple redis clusters (Master-Slave Cluster) with ease
2) Redis instances recover in seconds and not in minutes
3) If a Master fails a slaves is automatically promoted as the New
master, all the old slaves now replicate from the newly master plus a new
slave is added to the cluster. All this without using redis-sentinel in
your datacenter and all these happens in a couple of seconds.
4) A CLI to perform basic operations such as create / status / delete
redis instances on the fly. CLI is cross compiled for Windows and Darwin
users too.
5) Scheduler is a HTTP REST server which also can respond to simple
Angular UI we have built to get started with. Instructions on how to setup
the UI is here. https://github.com/mesos/mr-redis/tree/master/ui/app
We also had an opportunity to talk about this during the recent MesosCon
2016:
https://www.youtube.com/watch?v=xe-Gom5tOl0&index=32&list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC
Future Work:
· Implement a proxy technique to expose one single endpoint for the
redis instance. (Work in progress)
· Implement Memory Cgroups per redis PROCS
· Add support for Redis 3.0 cluster instances (Adding shards to a
running redis instance)
· Implement integration test suite and benchmarking suite to the
framework.
Special thanks to Adobe.io team who expressed interest in collaborating in
the development of this product. I’m sure it’s going to be great working
with all of you folks.
Advanced happy Independence day America and happy week ahead rest of the
world.
Looking forward to hear from you all,
Dhilip
Re: A Redis Framework for Apache Mesos
Posted by DhilipKumar Sankaranarayanan <s....@gmail.com>.
Thanks Avinash, It is done now.
On Mon, Jul 4, 2016 at 12:01 PM, Avinash Sridharan <av...@mesosphere.io>
wrote:
>
>
> On Mon, Jul 4, 2016 at 12:11 AM, DhilipKumar Sankaranarayanan <
> s.dhilipkumar@gmail.com> wrote:
>
>> *"*
>> *Would be nice to get on the frameworks page:*
>>
>> *http://mesos.apache.org/documentation/latest/frameworks/
>> <http://mesos.apache.org/documentation/latest/frameworks/> *
>> *?*
>> *"*
>> Sure i would love to do that, where should i raise a PR?
>>
> I think you can raise a PR on github. The preferred route though is to
> generate an RB request on Apache review board (
> https://reviews.apache.org/r/) . One of the Shepherds could help with
> this.
> +Vinod ^^
>
>> *"Also is it already part of the DC/OS universe ?"*
>>
>> Yes we have reasonable documentation
>> <https://github.com/mesos/mr-redis#dcos>with regards to DCOS
>> integration, Please give it a try and it will be awesome to get your
>> feedback on that.
>>
>> Regards,
>> Dhilip
>>
>> On Sun, Jul 3, 2016 at 6:09 PM, Avinash Sridharan <av...@mesosphere.io>
>> wrote:
>>
>>> Would be nice to get on the frameworks page:
>>> http://mesos.apache.org/documentation/latest/frameworks/
>>> ?
>>>
>>> Also is it already part of the DC/OS universe ?
>>>
>>> On Sun, Jul 3, 2016 at 2:29 PM, Christoph Heer <ch...@thelabmill.de>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> it looks really cool. Can you maybe explain why do you use etcd for
>>>> leader election instead of the normally already existing Zookeeper cluster?
>>>> Do you use some special etcd features?
>>>>
>>>> Best regards
>>>> Christoph
>>>>
>>>> > On 03 Jul 2016, at 17:49, tommy xiao <xi...@gmail.com> wrote:
>>>> >
>>>> > Cool. thanks for your sharing.
>>>> >
>>>> > 2016-07-03 23:44 GMT+08:00 DhilipKumar Sankaranarayanan <
>>>> s.dhilipkumar@gmail.com>:
>>>> > Hello All,
>>>> >
>>>> >
>>>> >
>>>> > We have built a framework for provisioning redis-servers in Apache
>>>> Mesos enabled infrastructure. It would be awesome to get communities
>>>> feedback. While mesos ecosystem is strengthening its capability in storage
>>>> layer, redis could be an addition to the lineup. This is primarily
>>>> intended for providers who would like to host redis as a service in their
>>>> infrastructure.
>>>> >
>>>> >
>>>> >
>>>> > There is an elaborate README about the project which should help in
>>>> setting it up: https://github.com/mesos/mr-redis. Please let us know
>>>> if there is anything missing in the documentation by raising a PR or
>>>> opening an issue or even writing to us.
>>>> >
>>>> >
>>>> >
>>>> > This project is also packaged as mr-redis with DCOS so should be
>>>> pretty straight forward to install it via DCOS CLI or DCOS GUI.
>>>> >
>>>> >
>>>> >
>>>> > (A Step by Step guide is provided in the README for your convenience)
>>>> >
>>>> >
>>>> >
>>>> > Salient Features of this project include:
>>>> >
>>>> > 1) Create multiple redis clusters (Master-Slave Cluster) with
>>>> ease
>>>> >
>>>> > 2) Redis instances recover in seconds and not in minutes
>>>> >
>>>> > 3) If a Master fails a slaves is automatically promoted as the
>>>> New master, all the old slaves now replicate from the newly master plus a
>>>> new slave is added to the cluster. All this without using redis-sentinel
>>>> in your datacenter and all these happens in a couple of seconds.
>>>> >
>>>> > 4) A CLI to perform basic operations such as create / status /
>>>> delete redis instances on the fly. CLI is cross compiled for Windows and
>>>> Darwin users too.
>>>> >
>>>> > 5) Scheduler is a HTTP REST server which also can respond to
>>>> simple Angular UI we have built to get started with. Instructions on how
>>>> to setup the UI is here.
>>>> https://github.com/mesos/mr-redis/tree/master/ui/app
>>>> >
>>>> >
>>>> >
>>>> > We also had an opportunity to talk about this during the recent
>>>> MesosCon 2016:
>>>> https://www.youtube.com/watch?v=xe-Gom5tOl0&index=32&list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC
>>>> >
>>>> >
>>>> >
>>>> > Future Work:
>>>> >
>>>> > · Implement a proxy technique to expose one single endpoint
>>>> for the redis instance. (Work in progress)
>>>> >
>>>> > · Implement Memory Cgroups per redis PROCS
>>>> >
>>>> > · Add support for Redis 3.0 cluster instances (Adding shards
>>>> to a running redis instance)
>>>> >
>>>> > · Implement integration test suite and benchmarking suite to
>>>> the framework.
>>>> >
>>>> >
>>>> >
>>>> > Special thanks to Adobe.io team who expressed interest in
>>>> collaborating in the development of this product. I’m sure it’s going to be
>>>> great working with all of you folks.
>>>> >
>>>> >
>>>> >
>>>> > Advanced happy Independence day America and happy week ahead rest of
>>>> the world.
>>>> >
>>>> >
>>>> >
>>>> > Looking forward to hear from you all,
>>>> >
>>>> > Dhilip
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Deshi Xiao
>>>> > Twitter: xds2000
>>>> > E-mail: xiaods(AT)gmail.com
>>>>
>>>>
>>>
>>>
>>> --
>>> Avinash Sridharan, Mesosphere
>>> +1 (323) 702 5245
>>>
>>
>>
>
>
> --
> Avinash Sridharan, Mesosphere
> +1 (323) 702 5245
>
Re: A Redis Framework for Apache Mesos
Posted by Avinash Sridharan <av...@mesosphere.io>.
On Mon, Jul 4, 2016 at 12:11 AM, DhilipKumar Sankaranarayanan <
s.dhilipkumar@gmail.com> wrote:
> *"*
> *Would be nice to get on the frameworks page:*
>
> *http://mesos.apache.org/documentation/latest/frameworks/
> <http://mesos.apache.org/documentation/latest/frameworks/> *
> *?*
> *"*
> Sure i would love to do that, where should i raise a PR?
>
I think you can raise a PR on github. The preferred route though is to
generate an RB request on Apache review board (https://reviews.apache.org/r/)
. One of the Shepherds could help with this.
+Vinod ^^
> *"Also is it already part of the DC/OS universe ?"*
>
> Yes we have reasonable documentation
> <https://github.com/mesos/mr-redis#dcos>with regards to DCOS integration,
> Please give it a try and it will be awesome to get your feedback on that.
>
> Regards,
> Dhilip
>
> On Sun, Jul 3, 2016 at 6:09 PM, Avinash Sridharan <av...@mesosphere.io>
> wrote:
>
>> Would be nice to get on the frameworks page:
>> http://mesos.apache.org/documentation/latest/frameworks/
>> ?
>>
>> Also is it already part of the DC/OS universe ?
>>
>> On Sun, Jul 3, 2016 at 2:29 PM, Christoph Heer <ch...@thelabmill.de>
>> wrote:
>>
>>> Hi,
>>>
>>> it looks really cool. Can you maybe explain why do you use etcd for
>>> leader election instead of the normally already existing Zookeeper cluster?
>>> Do you use some special etcd features?
>>>
>>> Best regards
>>> Christoph
>>>
>>> > On 03 Jul 2016, at 17:49, tommy xiao <xi...@gmail.com> wrote:
>>> >
>>> > Cool. thanks for your sharing.
>>> >
>>> > 2016-07-03 23:44 GMT+08:00 DhilipKumar Sankaranarayanan <
>>> s.dhilipkumar@gmail.com>:
>>> > Hello All,
>>> >
>>> >
>>> >
>>> > We have built a framework for provisioning redis-servers in Apache
>>> Mesos enabled infrastructure. It would be awesome to get communities
>>> feedback. While mesos ecosystem is strengthening its capability in storage
>>> layer, redis could be an addition to the lineup. This is primarily
>>> intended for providers who would like to host redis as a service in their
>>> infrastructure.
>>> >
>>> >
>>> >
>>> > There is an elaborate README about the project which should help in
>>> setting it up: https://github.com/mesos/mr-redis. Please let us know if
>>> there is anything missing in the documentation by raising a PR or opening
>>> an issue or even writing to us.
>>> >
>>> >
>>> >
>>> > This project is also packaged as mr-redis with DCOS so should be
>>> pretty straight forward to install it via DCOS CLI or DCOS GUI.
>>> >
>>> >
>>> >
>>> > (A Step by Step guide is provided in the README for your convenience)
>>> >
>>> >
>>> >
>>> > Salient Features of this project include:
>>> >
>>> > 1) Create multiple redis clusters (Master-Slave Cluster) with ease
>>> >
>>> > 2) Redis instances recover in seconds and not in minutes
>>> >
>>> > 3) If a Master fails a slaves is automatically promoted as the
>>> New master, all the old slaves now replicate from the newly master plus a
>>> new slave is added to the cluster. All this without using redis-sentinel
>>> in your datacenter and all these happens in a couple of seconds.
>>> >
>>> > 4) A CLI to perform basic operations such as create / status /
>>> delete redis instances on the fly. CLI is cross compiled for Windows and
>>> Darwin users too.
>>> >
>>> > 5) Scheduler is a HTTP REST server which also can respond to
>>> simple Angular UI we have built to get started with. Instructions on how
>>> to setup the UI is here.
>>> https://github.com/mesos/mr-redis/tree/master/ui/app
>>> >
>>> >
>>> >
>>> > We also had an opportunity to talk about this during the recent
>>> MesosCon 2016:
>>> https://www.youtube.com/watch?v=xe-Gom5tOl0&index=32&list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC
>>> >
>>> >
>>> >
>>> > Future Work:
>>> >
>>> > · Implement a proxy technique to expose one single endpoint
>>> for the redis instance. (Work in progress)
>>> >
>>> > · Implement Memory Cgroups per redis PROCS
>>> >
>>> > · Add support for Redis 3.0 cluster instances (Adding shards
>>> to a running redis instance)
>>> >
>>> > · Implement integration test suite and benchmarking suite to
>>> the framework.
>>> >
>>> >
>>> >
>>> > Special thanks to Adobe.io team who expressed interest in
>>> collaborating in the development of this product. I’m sure it’s going to be
>>> great working with all of you folks.
>>> >
>>> >
>>> >
>>> > Advanced happy Independence day America and happy week ahead rest of
>>> the world.
>>> >
>>> >
>>> >
>>> > Looking forward to hear from you all,
>>> >
>>> > Dhilip
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Deshi Xiao
>>> > Twitter: xds2000
>>> > E-mail: xiaods(AT)gmail.com
>>>
>>>
>>
>>
>> --
>> Avinash Sridharan, Mesosphere
>> +1 (323) 702 5245
>>
>
>
--
Avinash Sridharan, Mesosphere
+1 (323) 702 5245
Re: A Redis Framework for Apache Mesos
Posted by DhilipKumar Sankaranarayanan <s....@gmail.com>.
*"*
*Would be nice to get on the frameworks page:*
*http://mesos.apache.org/documentation/latest/frameworks/
<http://mesos.apache.org/documentation/latest/frameworks/> *
*?*
*"*
Sure i would love to do that, where should i raise a PR?
*"Also is it already part of the DC/OS universe ?"*
Yes we have reasonable documentation
<https://github.com/mesos/mr-redis#dcos>with regards to DCOS integration,
Please give it a try and it will be awesome to get your feedback on that.
Regards,
Dhilip
On Sun, Jul 3, 2016 at 6:09 PM, Avinash Sridharan <av...@mesosphere.io>
wrote:
> Would be nice to get on the frameworks page:
> http://mesos.apache.org/documentation/latest/frameworks/
> ?
>
> Also is it already part of the DC/OS universe ?
>
> On Sun, Jul 3, 2016 at 2:29 PM, Christoph Heer <ch...@thelabmill.de>
> wrote:
>
>> Hi,
>>
>> it looks really cool. Can you maybe explain why do you use etcd for
>> leader election instead of the normally already existing Zookeeper cluster?
>> Do you use some special etcd features?
>>
>> Best regards
>> Christoph
>>
>> > On 03 Jul 2016, at 17:49, tommy xiao <xi...@gmail.com> wrote:
>> >
>> > Cool. thanks for your sharing.
>> >
>> > 2016-07-03 23:44 GMT+08:00 DhilipKumar Sankaranarayanan <
>> s.dhilipkumar@gmail.com>:
>> > Hello All,
>> >
>> >
>> >
>> > We have built a framework for provisioning redis-servers in Apache
>> Mesos enabled infrastructure. It would be awesome to get communities
>> feedback. While mesos ecosystem is strengthening its capability in storage
>> layer, redis could be an addition to the lineup. This is primarily
>> intended for providers who would like to host redis as a service in their
>> infrastructure.
>> >
>> >
>> >
>> > There is an elaborate README about the project which should help in
>> setting it up: https://github.com/mesos/mr-redis. Please let us know if
>> there is anything missing in the documentation by raising a PR or opening
>> an issue or even writing to us.
>> >
>> >
>> >
>> > This project is also packaged as mr-redis with DCOS so should be
>> pretty straight forward to install it via DCOS CLI or DCOS GUI.
>> >
>> >
>> >
>> > (A Step by Step guide is provided in the README for your convenience)
>> >
>> >
>> >
>> > Salient Features of this project include:
>> >
>> > 1) Create multiple redis clusters (Master-Slave Cluster) with ease
>> >
>> > 2) Redis instances recover in seconds and not in minutes
>> >
>> > 3) If a Master fails a slaves is automatically promoted as the New
>> master, all the old slaves now replicate from the newly master plus a new
>> slave is added to the cluster. All this without using redis-sentinel in
>> your datacenter and all these happens in a couple of seconds.
>> >
>> > 4) A CLI to perform basic operations such as create / status /
>> delete redis instances on the fly. CLI is cross compiled for Windows and
>> Darwin users too.
>> >
>> > 5) Scheduler is a HTTP REST server which also can respond to
>> simple Angular UI we have built to get started with. Instructions on how
>> to setup the UI is here.
>> https://github.com/mesos/mr-redis/tree/master/ui/app
>> >
>> >
>> >
>> > We also had an opportunity to talk about this during the recent
>> MesosCon 2016:
>> https://www.youtube.com/watch?v=xe-Gom5tOl0&index=32&list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC
>> >
>> >
>> >
>> > Future Work:
>> >
>> > · Implement a proxy technique to expose one single endpoint for
>> the redis instance. (Work in progress)
>> >
>> > · Implement Memory Cgroups per redis PROCS
>> >
>> > · Add support for Redis 3.0 cluster instances (Adding shards to
>> a running redis instance)
>> >
>> > · Implement integration test suite and benchmarking suite to
>> the framework.
>> >
>> >
>> >
>> > Special thanks to Adobe.io team who expressed interest in collaborating
>> in the development of this product. I’m sure it’s going to be great working
>> with all of you folks.
>> >
>> >
>> >
>> > Advanced happy Independence day America and happy week ahead rest of
>> the world.
>> >
>> >
>> >
>> > Looking forward to hear from you all,
>> >
>> > Dhilip
>> >
>> >
>> >
>> >
>> >
>> > --
>> > Deshi Xiao
>> > Twitter: xds2000
>> > E-mail: xiaods(AT)gmail.com
>>
>>
>
>
> --
> Avinash Sridharan, Mesosphere
> +1 (323) 702 5245
>
Re: A Redis Framework for Apache Mesos
Posted by Avinash Sridharan <av...@mesosphere.io>.
Would be nice to get on the frameworks page:
http://mesos.apache.org/documentation/latest/frameworks/
?
Also is it already part of the DC/OS universe ?
On Sun, Jul 3, 2016 at 2:29 PM, Christoph Heer <ch...@thelabmill.de>
wrote:
> Hi,
>
> it looks really cool. Can you maybe explain why do you use etcd for leader
> election instead of the normally already existing Zookeeper cluster? Do you
> use some special etcd features?
>
> Best regards
> Christoph
>
> > On 03 Jul 2016, at 17:49, tommy xiao <xi...@gmail.com> wrote:
> >
> > Cool. thanks for your sharing.
> >
> > 2016-07-03 23:44 GMT+08:00 DhilipKumar Sankaranarayanan <
> s.dhilipkumar@gmail.com>:
> > Hello All,
> >
> >
> >
> > We have built a framework for provisioning redis-servers in Apache Mesos
> enabled infrastructure. It would be awesome to get communities feedback.
> While mesos ecosystem is strengthening its capability in storage layer,
> redis could be an addition to the lineup. This is primarily intended for
> providers who would like to host redis as a service in their infrastructure.
> >
> >
> >
> > There is an elaborate README about the project which should help in
> setting it up: https://github.com/mesos/mr-redis. Please let us know if
> there is anything missing in the documentation by raising a PR or opening
> an issue or even writing to us.
> >
> >
> >
> > This project is also packaged as mr-redis with DCOS so should be pretty
> straight forward to install it via DCOS CLI or DCOS GUI.
> >
> >
> >
> > (A Step by Step guide is provided in the README for your convenience)
> >
> >
> >
> > Salient Features of this project include:
> >
> > 1) Create multiple redis clusters (Master-Slave Cluster) with ease
> >
> > 2) Redis instances recover in seconds and not in minutes
> >
> > 3) If a Master fails a slaves is automatically promoted as the New
> master, all the old slaves now replicate from the newly master plus a new
> slave is added to the cluster. All this without using redis-sentinel in
> your datacenter and all these happens in a couple of seconds.
> >
> > 4) A CLI to perform basic operations such as create / status /
> delete redis instances on the fly. CLI is cross compiled for Windows and
> Darwin users too.
> >
> > 5) Scheduler is a HTTP REST server which also can respond to simple
> Angular UI we have built to get started with. Instructions on how to setup
> the UI is here. https://github.com/mesos/mr-redis/tree/master/ui/app
> >
> >
> >
> > We also had an opportunity to talk about this during the recent MesosCon
> 2016:
> https://www.youtube.com/watch?v=xe-Gom5tOl0&index=32&list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC
> >
> >
> >
> > Future Work:
> >
> > · Implement a proxy technique to expose one single endpoint for
> the redis instance. (Work in progress)
> >
> > · Implement Memory Cgroups per redis PROCS
> >
> > · Add support for Redis 3.0 cluster instances (Adding shards to
> a running redis instance)
> >
> > · Implement integration test suite and benchmarking suite to the
> framework.
> >
> >
> >
> > Special thanks to Adobe.io team who expressed interest in collaborating
> in the development of this product. I’m sure it’s going to be great working
> with all of you folks.
> >
> >
> >
> > Advanced happy Independence day America and happy week ahead rest of the
> world.
> >
> >
> >
> > Looking forward to hear from you all,
> >
> > Dhilip
> >
> >
> >
> >
> >
> > --
> > Deshi Xiao
> > Twitter: xds2000
> > E-mail: xiaods(AT)gmail.com
>
>
--
Avinash Sridharan, Mesosphere
+1 (323) 702 5245
Re: A Redis Framework for Apache Mesos
Posted by DhilipKumar Sankaranarayanan <s....@gmail.com>.
We use ETCD to store the state of the scheduler (mr-redis) not to perform
master election like "Mesos Master". The Scheduler itself is stateless it
stores plenty of information with respect to the redis instances in etcd.
we already have a request <https://github.com/mesos/mr-redis/issues/17>from
a user to start supporting zookeeper from the community. I haven't started
working on that yet. Any volunteers are welcome.
Plus etcd is not very hard to setup either, probably few commands
<https://github.com/coreos/etcd#getting-started> and you have a running
instance of etcd.
Hope this makes sense,
Dhilip
On Sun, Jul 3, 2016 at 2:29 PM, Christoph Heer <ch...@thelabmill.de>
wrote:
> Hi,
>
> it looks really cool. Can you maybe explain why do you use etcd for leader
> election instead of the normally already existing Zookeeper cluster? Do you
> use some special etcd features?
>
> Best regards
> Christoph
>
> > On 03 Jul 2016, at 17:49, tommy xiao <xi...@gmail.com> wrote:
> >
> > Cool. thanks for your sharing.
> >
> > 2016-07-03 23:44 GMT+08:00 DhilipKumar Sankaranarayanan <
> s.dhilipkumar@gmail.com>:
> > Hello All,
> >
> >
> >
> > We have built a framework for provisioning redis-servers in Apache Mesos
> enabled infrastructure. It would be awesome to get communities feedback.
> While mesos ecosystem is strengthening its capability in storage layer,
> redis could be an addition to the lineup. This is primarily intended for
> providers who would like to host redis as a service in their infrastructure.
> >
> >
> >
> > There is an elaborate README about the project which should help in
> setting it up: https://github.com/mesos/mr-redis. Please let us know if
> there is anything missing in the documentation by raising a PR or opening
> an issue or even writing to us.
> >
> >
> >
> > This project is also packaged as mr-redis with DCOS so should be pretty
> straight forward to install it via DCOS CLI or DCOS GUI.
> >
> >
> >
> > (A Step by Step guide is provided in the README for your convenience)
> >
> >
> >
> > Salient Features of this project include:
> >
> > 1) Create multiple redis clusters (Master-Slave Cluster) with ease
> >
> > 2) Redis instances recover in seconds and not in minutes
> >
> > 3) If a Master fails a slaves is automatically promoted as the New
> master, all the old slaves now replicate from the newly master plus a new
> slave is added to the cluster. All this without using redis-sentinel in
> your datacenter and all these happens in a couple of seconds.
> >
> > 4) A CLI to perform basic operations such as create / status /
> delete redis instances on the fly. CLI is cross compiled for Windows and
> Darwin users too.
> >
> > 5) Scheduler is a HTTP REST server which also can respond to simple
> Angular UI we have built to get started with. Instructions on how to setup
> the UI is here. https://github.com/mesos/mr-redis/tree/master/ui/app
> >
> >
> >
> > We also had an opportunity to talk about this during the recent MesosCon
> 2016:
> https://www.youtube.com/watch?v=xe-Gom5tOl0&index=32&list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC
> >
> >
> >
> > Future Work:
> >
> > · Implement a proxy technique to expose one single endpoint for
> the redis instance. (Work in progress)
> >
> > · Implement Memory Cgroups per redis PROCS
> >
> > · Add support for Redis 3.0 cluster instances (Adding shards to
> a running redis instance)
> >
> > · Implement integration test suite and benchmarking suite to the
> framework.
> >
> >
> >
> > Special thanks to Adobe.io team who expressed interest in collaborating
> in the development of this product. I’m sure it’s going to be great working
> with all of you folks.
> >
> >
> >
> > Advanced happy Independence day America and happy week ahead rest of the
> world.
> >
> >
> >
> > Looking forward to hear from you all,
> >
> > Dhilip
> >
> >
> >
> >
> >
> > --
> > Deshi Xiao
> > Twitter: xds2000
> > E-mail: xiaods(AT)gmail.com
>
>
Re: A Redis Framework for Apache Mesos
Posted by Christoph Heer <ch...@thelabmill.de>.
Hi,
it looks really cool. Can you maybe explain why do you use etcd for leader election instead of the normally already existing Zookeeper cluster? Do you use some special etcd features?
Best regards
Christoph
> On 03 Jul 2016, at 17:49, tommy xiao <xi...@gmail.com> wrote:
>
> Cool. thanks for your sharing.
>
> 2016-07-03 23:44 GMT+08:00 DhilipKumar Sankaranarayanan <s....@gmail.com>:
> Hello All,
>
>
>
> We have built a framework for provisioning redis-servers in Apache Mesos enabled infrastructure. It would be awesome to get communities feedback. While mesos ecosystem is strengthening its capability in storage layer, redis could be an addition to the lineup. This is primarily intended for providers who would like to host redis as a service in their infrastructure.
>
>
>
> There is an elaborate README about the project which should help in setting it up: https://github.com/mesos/mr-redis. Please let us know if there is anything missing in the documentation by raising a PR or opening an issue or even writing to us.
>
>
>
> This project is also packaged as mr-redis with DCOS so should be pretty straight forward to install it via DCOS CLI or DCOS GUI.
>
>
>
> (A Step by Step guide is provided in the README for your convenience)
>
>
>
> Salient Features of this project include:
>
> 1) Create multiple redis clusters (Master-Slave Cluster) with ease
>
> 2) Redis instances recover in seconds and not in minutes
>
> 3) If a Master fails a slaves is automatically promoted as the New master, all the old slaves now replicate from the newly master plus a new slave is added to the cluster. All this without using redis-sentinel in your datacenter and all these happens in a couple of seconds.
>
> 4) A CLI to perform basic operations such as create / status / delete redis instances on the fly. CLI is cross compiled for Windows and Darwin users too.
>
> 5) Scheduler is a HTTP REST server which also can respond to simple Angular UI we have built to get started with. Instructions on how to setup the UI is here. https://github.com/mesos/mr-redis/tree/master/ui/app
>
>
>
> We also had an opportunity to talk about this during the recent MesosCon 2016: https://www.youtube.com/watch?v=xe-Gom5tOl0&index=32&list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC
>
>
>
> Future Work:
>
> · Implement a proxy technique to expose one single endpoint for the redis instance. (Work in progress)
>
> · Implement Memory Cgroups per redis PROCS
>
> · Add support for Redis 3.0 cluster instances (Adding shards to a running redis instance)
>
> · Implement integration test suite and benchmarking suite to the framework.
>
>
>
> Special thanks to Adobe.io team who expressed interest in collaborating in the development of this product. I’m sure it’s going to be great working with all of you folks.
>
>
>
> Advanced happy Independence day America and happy week ahead rest of the world.
>
>
>
> Looking forward to hear from you all,
>
> Dhilip
>
>
>
>
>
> --
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com
Re: A Redis Framework for Apache Mesos
Posted by tommy xiao <xi...@gmail.com>.
Cool. thanks for your sharing.
2016-07-03 23:44 GMT+08:00 DhilipKumar Sankaranarayanan <
s.dhilipkumar@gmail.com>:
> Hello All,
>
>
>
> We have built a framework for provisioning redis-servers in Apache Mesos
> enabled infrastructure. It would be awesome to get communities
> feedback. While mesos ecosystem is strengthening its capability in storage
> layer, redis could be an addition to the lineup. This is primarily
> intended for providers who would like to host redis as a service in their
> infrastructure.
>
>
>
> There is an elaborate README about the project which should help in
> setting it up: https://github.com/mesos/mr-redis. Please let us know if
> there is anything missing in the documentation by raising a PR or opening
> an issue or even writing to us.
>
>
>
> This project is also packaged as mr-redis with DCOS so should be pretty
> straight forward to install it via DCOS CLI or DCOS GUI.
>
>
>
> (A Step by Step guide is provided in the README for your convenience)
>
>
>
> Salient Features of this project include:
>
> 1) Create multiple redis clusters (Master-Slave Cluster) with ease
>
> 2) Redis instances recover in seconds and not in minutes
>
> 3) If a Master fails a slaves is automatically promoted as the New
> master, all the old slaves now replicate from the newly master plus a new
> slave is added to the cluster. All this without using redis-sentinel in
> your datacenter and all these happens in a couple of seconds.
>
> 4) A CLI to perform basic operations such as create / status /
> delete redis instances on the fly. CLI is cross compiled for Windows and
> Darwin users too.
>
> 5) Scheduler is a HTTP REST server which also can respond to simple
> Angular UI we have built to get started with. Instructions on how to setup
> the UI is here. https://github.com/mesos/mr-redis/tree/master/ui/app
>
>
>
> We also had an opportunity to talk about this during the recent MesosCon
> 2016:
> https://www.youtube.com/watch?v=xe-Gom5tOl0&index=32&list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC
>
>
>
> Future Work:
>
> · Implement a proxy technique to expose one single endpoint for
> the redis instance. (Work in progress)
>
> · Implement Memory Cgroups per redis PROCS
>
> · Add support for Redis 3.0 cluster instances (Adding shards to a
> running redis instance)
>
> · Implement integration test suite and benchmarking suite to the
> framework.
>
>
>
> Special thanks to Adobe.io team who expressed interest in collaborating in
> the development of this product. I’m sure it’s going to be great working
> with all of you folks.
>
>
>
> Advanced happy Independence day America and happy week ahead rest of the
> world.
>
>
>
> Looking forward to hear from you all,
>
> Dhilip
>
>
--
Deshi Xiao
Twitter: xds2000
E-mail: xiaods(AT)gmail.com