You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Marco Massenzio <ma...@mesosphere.io> on 2015/09/25 10:13:45 UTC
Fwd: [Breaking Change 0.24 & Upgrade path] ZooKeeper MasterInfo change.
Folks:
as a reminder, please be aware that as of Mesos 0.24.0, as announced back
in June, Mesos Master will write its information (`MasterInfo`) to
ZooKeeper in JSON format (see below for details).
If your framework relied on parsing the info (either de-serializing the
Protocol Buffer or just looking for an "IP-like" string) this change will
be a breaking change.
Just to confirm (see also Vinod's comments below) any rolling upgrades
(i.e., clusters with 0.22+0.23 and 0.23+0.24) of Mesos will just work.
This was in conjunction with the HTTP API release and removing the need for
non-C++ developers to have to link with libmesos and have to deal with
Protocol Buffers.
An example of how to access the new format in Python can be found in [0]
and we're happy to help with other languages too.
Any questions, please just ask.
[0] http://github.com/massenz/zk-mesos
Marco Massenzio
*Distributed Systems Engineerhttp://codetrips.com <http://codetrips.com>*
---------- Forwarded message ----------
From: Vinod Kone <vi...@gmail.com>
Date: Wed, Jun 24, 2015 at 4:17 PM
Subject: Re: [Breaking Change 0.24 & Upgrade path] ZooKeeper MasterInfo
change.
To: dev <de...@mesos.apache.org>
Just to clarify, any frameworks that are using the Mesos provided bindings
(aka libmesos.so) should not worry, as long as the version of the bindings
and version of the mesos master are not separated by more than 1 version.
In other words, you should be able to live upgrade a cluster from 0.23.0 to
0.24.0.
For framework schedulers that don't use the bindings (pesos, jesos etc), it
is prudent to add support for JSON formatted ZNODE to their master
detection code.
Thanks,
On Wed, Jun 24, 2015 at 4:10 PM, Marco Massenzio <ma...@mesosphere.io>
wrote:
> Folks,
>
> as heads-up, we are planning to convert the format of the MasterInfo
> information stored in ZooKeeper from the Protocol Buffer binary format to
> JSON - this is in conjunction with the HTTP API development, to allow
> frameworks *not* to depend on libmesos and other binary dependencies to
> interact with Mesos Master nodes.
>
> *NOTE* - there is no change in 0.23 (so any Master/Slave/Framework that is
> currently working in 0.22 *will continue to work* in 0.23 too) but as of
> Mesos 0.24, frameworks and other clients relying on the binary format will
> break.
>
> The details of the design are in this Google Doc:
>
>
https://docs.google.com/document/d/1i2pWJaIjnFYhuR-000NG-AC1rFKKrRh3Wn47Y2G6lRE/edit
>
> the actual work is detailed in MESOS-2340:
> https://issues.apache.org/jira/browse/MESOS-2340
>
> and the patch (and associated test) are here:
> https://reviews.apache.org/r/35571/
> https://reviews.apache.org/r/35815/
>
> *Marco Massenzio*
> *Distributed Systems Engineer*
>
Re: Fwd: [Breaking Change 0.24 & Upgrade path] ZooKeeper MasterInfo
change.
Posted by CCAAT <cc...@tampabay.rr.com>.
On 09/25/2015 04:34 PM, Raúl Gutiérrez Segalés wrote:
> On Sep 25, 2015 9:08 AM, "Marco Massenzio" <marco@mesosphere.io
> <ma...@mesosphere.io>> wrote:
> >
> > +1 to what Alex says.
> >
> > As far as we know, the functionality we use (ephemeral sequential
> nodes and writing simple data to a znode) is part of the "base API"
> offered by ZooKeeper and every version would support it.
> > (then again, not a ZK expert here - if anyone knows better, please
> feel free to correct me).
> >
>
> Yup, not of that is changing in the upcoming ZooKeeper releases (3.4.7 &
> 3.5.2) and probably never will.
>
> -rgs
That simplifies the related dependencies for zookeeper.
Thanks,
James
Re: Fwd: [Breaking Change 0.24 & Upgrade path] ZooKeeper MasterInfo change.
Posted by Raúl Gutiérrez Segalés <rg...@itevenworks.net>.
On Sep 25, 2015 9:08 AM, "Marco Massenzio" <ma...@mesosphere.io> wrote:
>
> +1 to what Alex says.
>
> As far as we know, the functionality we use (ephemeral sequential nodes
and writing simple data to a znode) is part of the "base API" offered by
ZooKeeper and every version would support it.
> (then again, not a ZK expert here - if anyone knows better, please feel
free to correct me).
>
Yup, not of that is changing in the upcoming ZooKeeper releases (3.4.7 &
3.5.2) and probably never will.
-rgs
Re: Fwd: [Breaking Change 0.24 & Upgrade path] ZooKeeper MasterInfo change.
Posted by Marco Massenzio <ma...@mesosphere.io>.
+1 to what Alex says.
As far as we know, the functionality we use (ephemeral sequential nodes and
writing simple data to a znode) is part of the "base API" offered by
ZooKeeper and every version would support it.
(then again, not a ZK expert here - if anyone knows better, please feel
free to correct me).
*Marco Massenzio*
*Distributed Systems Engineerhttp://codetrips.com <http://codetrips.com>*
On Fri, Sep 25, 2015 at 6:24 AM, Alex Rukletsov <al...@mesosphere.com> wrote:
> James—
>
> Marco will correct me if I'm wrong, but my understanding is that this
> change does *not* impact what ZooKeeper version you can use with Mesos. We
> have changed the format of the message stored in ZK from protobuf to JSON.
> This message is needed by frameworks for mesos master leader detection.
>
> HTH,
> Alex
>
> On Fri, Sep 25, 2015 at 11:12 AM, CCAAT <cc...@tampabay.rr.com> wrote:
>
>> On 09/25/2015 08:13 AM, Marco Massenzio wrote:
>>
>>> Folks:
>>>
>>> as a reminder, please be aware that as of Mesos 0.24.0, as announced
>>> back in June, Mesos Master will write its information (`MasterInfo`) to
>>> ZooKeeper in JSON format (see below for details).
>>>
>>
>>
>> What versions of Zookeeper are supported by this change? That is, what
>> is the oldest version of Zookeeper known to work or not work with this
>> change in Mesos?
>>
>>
>> James
>>
>>
>>
>>
>>
>>> If your framework relied on parsing the info (either de-serializing the
>>> Protocol Buffer or just looking for an "IP-like" string) this change
>>> will be a breaking change.
>>>
>>> Just to confirm (see also Vinod's comments below) any rolling upgrades
>>> (i.e., clusters with 0.22+0.23 and 0.23+0.24) of Mesos will just work.
>>>
>>> This was in conjunction with the HTTP API release and removing the need
>>> for non-C++ developers to have to link with libmesos and have to deal
>>> with Protocol Buffers.
>>>
>>> An example of how to access the new format in Python can be found in [0]
>>> and we're happy to help with other languages too.
>>> Any questions, please just ask.
>>>
>>> [0] http://github.com/massenz/zk-mesos
>>>
>>> Marco Massenzio
>>> /Distributed Systems Engineer
>>> http://codetrips.com/
>>>
>>> ---------- Forwarded message ----------
>>> From: *Vinod Kone* <vinodkone@gmail.com <ma...@gmail.com>>
>>> Date: Wed, Jun 24, 2015 at 4:17 PM
>>> Subject: Re: [Breaking Change 0.24 & Upgrade path] ZooKeeper MasterInfo
>>> change.
>>> To: dev <dev@mesos.apache.org <ma...@mesos.apache.org>>
>>>
>>>
>>> Just to clarify, any frameworks that are using the Mesos provided
>>> bindings
>>> (aka libmesos.so) should not worry, as long as the version of the
>>> bindings
>>> and version of the mesos master are not separated by more than 1 version.
>>> In other words, you should be able to live upgrade a cluster from 0.23.0
>>> to
>>> 0.24.0.
>>>
>>> For framework schedulers that don't use the bindings (pesos, jesos etc),
>>> it
>>> is prudent to add support for JSON formatted ZNODE to their master
>>> detection code.
>>>
>>> Thanks,
>>>
>>> On Wed, Jun 24, 2015 at 4:10 PM, Marco Massenzio <marco@mesosphere.io
>>> <ma...@mesosphere.io>>
>>> wrote:
>>>
>>> Folks,
>>>>
>>>> as heads-up, we are planning to convert the format of the MasterInfo
>>>> information stored in ZooKeeper from the Protocol Buffer binary format
>>>> to
>>>> JSON - this is in conjunction with the HTTP API development, to allow
>>>> frameworks *not* to depend on libmesos and other binary dependencies to
>>>> interact with Mesos Master nodes.
>>>>
>>>> > *NOTE* - there is no change in 0.23 (so any Master/Slave/Framework
>>> that is
>>> > currently working in 0.22 *will continue to work* in 0.23 too) but as
>>> of
>>>
>>>> Mesos 0.24, frameworks and other clients relying on the binary format
>>>> will
>>>> break.
>>>>
>>>> The details of the design are in this Google Doc:
>>>>
>>>>
>>>> https://docs.google.com/document/d/1i2pWJaIjnFYhuR-000NG-AC1rFKKrRh3Wn47Y2G6lRE/edit
>>>>
>>>> the actual work is detailed in MESOS-2340:
>>>> https://issues.apache.org/jira/browse/MESOS-2340
>>>>
>>>> and the patch (and associated test) are here:
>>>> https://reviews.apache.org/r/35571/
>>>> https://reviews.apache.org/r/35815/
>>>>
>>>> > *Marco Massenzio*
>>> > *Distributed Systems Engineer*
>>> >
>>>
>>>
>>
>
Re: Fwd: [Breaking Change 0.24 & Upgrade path] ZooKeeper MasterInfo change.
Posted by Alex Rukletsov <al...@mesosphere.com>.
James—
Marco will correct me if I'm wrong, but my understanding is that this
change does *not* impact what ZooKeeper version you can use with Mesos. We
have changed the format of the message stored in ZK from protobuf to JSON.
This message is needed by frameworks for mesos master leader detection.
HTH,
Alex
On Fri, Sep 25, 2015 at 11:12 AM, CCAAT <cc...@tampabay.rr.com> wrote:
> On 09/25/2015 08:13 AM, Marco Massenzio wrote:
>
>> Folks:
>>
>> as a reminder, please be aware that as of Mesos 0.24.0, as announced
>> back in June, Mesos Master will write its information (`MasterInfo`) to
>> ZooKeeper in JSON format (see below for details).
>>
>
>
> What versions of Zookeeper are supported by this change? That is, what
> is the oldest version of Zookeeper known to work or not work with this
> change in Mesos?
>
>
> James
>
>
>
>
>
>> If your framework relied on parsing the info (either de-serializing the
>> Protocol Buffer or just looking for an "IP-like" string) this change
>> will be a breaking change.
>>
>> Just to confirm (see also Vinod's comments below) any rolling upgrades
>> (i.e., clusters with 0.22+0.23 and 0.23+0.24) of Mesos will just work.
>>
>> This was in conjunction with the HTTP API release and removing the need
>> for non-C++ developers to have to link with libmesos and have to deal
>> with Protocol Buffers.
>>
>> An example of how to access the new format in Python can be found in [0]
>> and we're happy to help with other languages too.
>> Any questions, please just ask.
>>
>> [0] http://github.com/massenz/zk-mesos
>>
>> Marco Massenzio
>> /Distributed Systems Engineer
>> http://codetrips.com/
>>
>> ---------- Forwarded message ----------
>> From: *Vinod Kone* <vinodkone@gmail.com <ma...@gmail.com>>
>> Date: Wed, Jun 24, 2015 at 4:17 PM
>> Subject: Re: [Breaking Change 0.24 & Upgrade path] ZooKeeper MasterInfo
>> change.
>> To: dev <dev@mesos.apache.org <ma...@mesos.apache.org>>
>>
>>
>> Just to clarify, any frameworks that are using the Mesos provided bindings
>> (aka libmesos.so) should not worry, as long as the version of the bindings
>> and version of the mesos master are not separated by more than 1 version.
>> In other words, you should be able to live upgrade a cluster from 0.23.0
>> to
>> 0.24.0.
>>
>> For framework schedulers that don't use the bindings (pesos, jesos etc),
>> it
>> is prudent to add support for JSON formatted ZNODE to their master
>> detection code.
>>
>> Thanks,
>>
>> On Wed, Jun 24, 2015 at 4:10 PM, Marco Massenzio <marco@mesosphere.io
>> <ma...@mesosphere.io>>
>> wrote:
>>
>> Folks,
>>>
>>> as heads-up, we are planning to convert the format of the MasterInfo
>>> information stored in ZooKeeper from the Protocol Buffer binary format to
>>> JSON - this is in conjunction with the HTTP API development, to allow
>>> frameworks *not* to depend on libmesos and other binary dependencies to
>>> interact with Mesos Master nodes.
>>>
>>> > *NOTE* - there is no change in 0.23 (so any Master/Slave/Framework
>> that is
>> > currently working in 0.22 *will continue to work* in 0.23 too) but as
>> of
>>
>>> Mesos 0.24, frameworks and other clients relying on the binary format
>>> will
>>> break.
>>>
>>> The details of the design are in this Google Doc:
>>>
>>>
>>> https://docs.google.com/document/d/1i2pWJaIjnFYhuR-000NG-AC1rFKKrRh3Wn47Y2G6lRE/edit
>>>
>>> the actual work is detailed in MESOS-2340:
>>> https://issues.apache.org/jira/browse/MESOS-2340
>>>
>>> and the patch (and associated test) are here:
>>> https://reviews.apache.org/r/35571/
>>> https://reviews.apache.org/r/35815/
>>>
>>> > *Marco Massenzio*
>> > *Distributed Systems Engineer*
>> >
>>
>>
>
Re: Fwd: [Breaking Change 0.24 & Upgrade path] ZooKeeper MasterInfo
change.
Posted by CCAAT <cc...@tampabay.rr.com>.
On 09/25/2015 08:13 AM, Marco Massenzio wrote:
> Folks:
>
> as a reminder, please be aware that as of Mesos 0.24.0, as announced
> back in June, Mesos Master will write its information (`MasterInfo`) to
> ZooKeeper in JSON format (see below for details).
What versions of Zookeeper are supported by this change? That is, what
is the oldest version of Zookeeper known to work or not work with this
change in Mesos?
James
>
> If your framework relied on parsing the info (either de-serializing the
> Protocol Buffer or just looking for an "IP-like" string) this change
> will be a breaking change.
>
> Just to confirm (see also Vinod's comments below) any rolling upgrades
> (i.e., clusters with 0.22+0.23 and 0.23+0.24) of Mesos will just work.
>
> This was in conjunction with the HTTP API release and removing the need
> for non-C++ developers to have to link with libmesos and have to deal
> with Protocol Buffers.
>
> An example of how to access the new format in Python can be found in [0]
> and we're happy to help with other languages too.
> Any questions, please just ask.
>
> [0] http://github.com/massenz/zk-mesos
>
> Marco Massenzio
> /Distributed Systems Engineer
> http://codetrips.com/
>
> ---------- Forwarded message ----------
> From: *Vinod Kone* <vinodkone@gmail.com <ma...@gmail.com>>
> Date: Wed, Jun 24, 2015 at 4:17 PM
> Subject: Re: [Breaking Change 0.24 & Upgrade path] ZooKeeper MasterInfo
> change.
> To: dev <dev@mesos.apache.org <ma...@mesos.apache.org>>
>
>
> Just to clarify, any frameworks that are using the Mesos provided bindings
> (aka libmesos.so) should not worry, as long as the version of the bindings
> and version of the mesos master are not separated by more than 1 version.
> In other words, you should be able to live upgrade a cluster from 0.23.0 to
> 0.24.0.
>
> For framework schedulers that don't use the bindings (pesos, jesos etc), it
> is prudent to add support for JSON formatted ZNODE to their master
> detection code.
>
> Thanks,
>
> On Wed, Jun 24, 2015 at 4:10 PM, Marco Massenzio <marco@mesosphere.io
> <ma...@mesosphere.io>>
> wrote:
>
>> Folks,
>>
>> as heads-up, we are planning to convert the format of the MasterInfo
>> information stored in ZooKeeper from the Protocol Buffer binary format to
>> JSON - this is in conjunction with the HTTP API development, to allow
>> frameworks *not* to depend on libmesos and other binary dependencies to
>> interact with Mesos Master nodes.
>>
> > *NOTE* - there is no change in 0.23 (so any Master/Slave/Framework
> that is
> > currently working in 0.22 *will continue to work* in 0.23 too) but as of
>> Mesos 0.24, frameworks and other clients relying on the binary format will
>> break.
>>
>> The details of the design are in this Google Doc:
>>
>>https://docs.google.com/document/d/1i2pWJaIjnFYhuR-000NG-AC1rFKKrRh3Wn47Y2G6lRE/edit
>>
>> the actual work is detailed in MESOS-2340:
>>https://issues.apache.org/jira/browse/MESOS-2340
>>
>> and the patch (and associated test) are here:
>>https://reviews.apache.org/r/35571/
>>https://reviews.apache.org/r/35815/
>>
> > *Marco Massenzio*
> > *Distributed Systems Engineer*
> >
>