You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Rodrick Brown <ro...@orchard-app.com> on 2016/12/02 20:04:44 UTC
Mesos cluster across a wide area network
How feasible is it to run Mesos across a private WAN.
I have two data centers and private fiber between each location latency is about 80ms between both locations.
My thinking is to run 5 masters Three in NY and 2 in London same setup for Zookeeper.
I have about 120 nodes in NYC and 80 nodes in London.
Or should I just create two silo clusters of 3 masters in each region.
Thanks.
--
RB
--
*NOTICE TO RECIPIENTS*: This communication is confidential and intended for
the use of the addressee only. If you are not an intended recipient of this
communication, please delete it immediately and notify the sender by return
email. Unauthorized reading, dissemination, distribution or copying of this
communication is prohibited. This communication does not constitute an
offer to sell or a solicitation of an indication of interest to purchase
any loan, security or any other financial product or instrument, nor is it
an offer to sell or a solicitation of an indication of interest to purchase
any products or services to any persons who are prohibited from receiving
such information under applicable law. The contents of this communication
may not be accurate or complete and are subject to change without notice.
As such, Orchard App, Inc. (including its subsidiaries and affiliates,
"Orchard") makes no representation regarding the accuracy or completeness
of the information contained herein. The intended recipient is advised to
consult its own professional advisors, including those specializing in
legal, tax and accounting matters. Orchard does not provide legal, tax or
accounting advice.
Re: Mesos cluster across a wide area network
Posted by Rodrick Brown <ro...@orchard-app.com>.
> On Dec 2, 2016, at 3:28 PM, Avinash Sridharan <av...@mesosphere.io> wrote:
>
> There has been some discussion on federated clusters on this list before. Probably @bmahler can add some more thoughts here. Apart from the issue of stabilizing the Master quorum there is also the issue of scheduling, i.e., how do frameworks decide which set of nodes, NY or Longond, to schedule a job on? Or put another way which resources to accept (from nodes in NY or London) for which tasks
I was planning on using dedicated roles and or attributes to determine where services ran am I missing something obvious here?
I’ve ran small regional Mesos clusters before and I didn’t think this would change much on the global scale but maybe I’m not seeing something.
> On Fri, Dec 2, 2016 at 12:12 PM, Alexander Gallego <agallego@concord.io <ma...@concord.io>> wrote:
>
>
> On Fri, Dec 2, 2016 at 3:04 PM, Rodrick Brown <rodrick@orchard-app.com <ma...@orchard-app.com>> wrote:
> How feasible is it to run Mesos across a private WAN.
>
> I have two data centers and private fiber between each location latency is about 80ms between both locations.
> My thinking is to run 5 masters Three in NY and 2 in London same setup for Zookeeper.
> I have about 120 nodes in NYC and 80 nodes in London.
>
> Or should I just create two silo clusters of 3 masters in each region.
>
> If you do, you miss the opportunity of doing oversubscriptions. i.e.: say NY is overloaded, you might want to move computations to London on a pre-emptable schedule.
>
> iff you can get zookeeper to be stable then you can do this.
>
> if you only have 2 locations might not be a good idea as the leader election is about consistency, so your cluster might not be able to launch new work during a partition.
>
>
> Thanks.
>
> --
>
> RB
>
> NOTICE TO RECIPIENTS: This communication is confidential and intended for the use of the addressee only. If you are not an intended recipient of this communication, please delete it immediately and notify the sender by return email. Unauthorized reading, dissemination, distribution or copying of this communication is prohibited. This communication does not constitute an offer to sell or a solicitation of an indication of interest to purchase any loan, security or any other financial product or instrument, nor is it an offer to sell or a solicitation of an indication of interest to purchase any products or services to any persons who are prohibited from receiving such information under applicable law. The contents of this communication may not be accurate or complete and are subject to change without notice. As such, Orchard App, Inc. (including its subsidiaries and affiliates, "Orchard") makes no representation regarding the accuracy or completeness of the information contained herein. The intended recipient is advised to consult its own professional advisors, including those specializing in legal, tax and accounting matters. Orchard does not provide legal, tax or accounting advice.
>
>
>
> --
>
>
>
>
>
> Alexander Gallego
> Co-Founder & CTO
>
>
>
> --
> Avinash Sridharan, Mesosphere
> +1 (323) 702 5245
--
*NOTICE TO RECIPIENTS*: This communication is confidential and intended for
the use of the addressee only. If you are not an intended recipient of this
communication, please delete it immediately and notify the sender by return
email. Unauthorized reading, dissemination, distribution or copying of this
communication is prohibited. This communication does not constitute an
offer to sell or a solicitation of an indication of interest to purchase
any loan, security or any other financial product or instrument, nor is it
an offer to sell or a solicitation of an indication of interest to purchase
any products or services to any persons who are prohibited from receiving
such information under applicable law. The contents of this communication
may not be accurate or complete and are subject to change without notice.
As such, Orchard App, Inc. (including its subsidiaries and affiliates,
"Orchard") makes no representation regarding the accuracy or completeness
of the information contained herein. The intended recipient is advised to
consult its own professional advisors, including those specializing in
legal, tax and accounting matters. Orchard does not provide legal, tax or
accounting advice.
Re: Mesos cluster across a wide area network
Posted by Avinash Sridharan <av...@mesosphere.io>.
There has been some discussion on federated clusters on this list before.
Probably @bmahler can add some more thoughts here. Apart from the issue of
stabilizing the Master quorum there is also the issue of scheduling, i.e.,
how do frameworks decide which set of nodes, NY or Longond, to schedule a
job on? Or put another way which resources to accept (from nodes in NY or
London) for which tasks?
On Fri, Dec 2, 2016 at 12:12 PM, Alexander Gallego <ag...@concord.io>
wrote:
>
>
> On Fri, Dec 2, 2016 at 3:04 PM, Rodrick Brown <ro...@orchard-app.com>
> wrote:
>
>> How feasible is it to run Mesos across a private WAN.
>>
>> I have two data centers and private fiber between each location latency
>> is about 80ms between both locations.
>> My thinking is to run 5 masters Three in NY and 2 in London same setup
>> for Zookeeper.
>> I have about 120 nodes in NYC and 80 nodes in London.
>>
>> Or should I just create two silo clusters of 3 masters in each region.
>>
>
> If you do, you miss the opportunity of doing oversubscriptions. i.e.: say
> NY is overloaded, you might want to move computations to London on a
> pre-emptable schedule.
>
> iff you can get zookeeper to be stable then you can do this.
>
> if you only have 2 locations might not be a good idea as the leader
> election is about consistency, so your cluster might not be able to launch
> new work during a partition.
>
>
>>
>> Thanks.
>>
>> --
>>
>> RB
>>
>> *NOTICE TO RECIPIENTS*: This communication is confidential and intended
>> for the use of the addressee only. If you are not an intended recipient of
>> this communication, please delete it immediately and notify the sender
>> by return email. Unauthorized reading, dissemination, distribution or
>> copying of this communication is prohibited. This communication does not constitute
>> an offer to sell or a solicitation of an indication of interest to purchase
>> any loan, security or any other financial product or instrument, nor is it
>> an offer to sell or a solicitation of an indication of interest to purchase
>> any products or services to any persons who are prohibited from receiving
>> such information under applicable law. The contents of this communication
>> may not be accurate or complete and are subject to change without notice.
>> As such, Orchard App, Inc. (including its subsidiaries and affiliates,
>> "Orchard") makes no representation regarding the accuracy or
>> completeness of the information contained herein. The intended recipient is
>> advised to consult its own professional advisors, including those
>> specializing in legal, tax and accounting matters. Orchard does not
>> provide legal, tax or accounting advice.
>>
>
>
>
> --
>
>
>
>
>
> Alexander Gallego
> Co-Founder & CTO
>
--
Avinash Sridharan, Mesosphere
+1 (323) 702 5245
Re: Mesos cluster across a wide area network
Posted by Alexander Gallego <ag...@concord.io>.
On Fri, Dec 2, 2016 at 3:04 PM, Rodrick Brown <ro...@orchard-app.com>
wrote:
> How feasible is it to run Mesos across a private WAN.
>
> I have two data centers and private fiber between each location latency is
> about 80ms between both locations.
> My thinking is to run 5 masters Three in NY and 2 in London same setup for
> Zookeeper.
> I have about 120 nodes in NYC and 80 nodes in London.
>
> Or should I just create two silo clusters of 3 masters in each region.
>
If you do, you miss the opportunity of doing oversubscriptions. i.e.: say
NY is overloaded, you might want to move computations to London on a
pre-emptable schedule.
iff you can get zookeeper to be stable then you can do this.
if you only have 2 locations might not be a good idea as the leader
election is about consistency, so your cluster might not be able to launch
new work during a partition.
>
> Thanks.
>
> --
>
> RB
>
> *NOTICE TO RECIPIENTS*: This communication is confidential and intended
> for the use of the addressee only. If you are not an intended recipient of
> this communication, please delete it immediately and notify the sender by
> return email. Unauthorized reading, dissemination, distribution or copying
> of this communication is prohibited. This communication does not constitute
> an offer to sell or a solicitation of an indication of interest to purchase
> any loan, security or any other financial product or instrument, nor is it
> an offer to sell or a solicitation of an indication of interest to purchase
> any products or services to any persons who are prohibited from receiving
> such information under applicable law. The contents of this communication
> may not be accurate or complete and are subject to change without notice.
> As such, Orchard App, Inc. (including its subsidiaries and affiliates,
> "Orchard") makes no representation regarding the accuracy or completeness
> of the information contained herein. The intended recipient is advised to
> consult its own professional advisors, including those specializing in
> legal, tax and accounting matters. Orchard does not provide legal, tax or
> accounting advice.
>
--
Alexander Gallego
Co-Founder & CTO