You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Thomas Rampelberg <th...@saunter.org> on 2014/08/01 18:37:22 UTC
Python bindings are changing!
- What problem are we trying to solve?
Currently, the python bindings group protobufs, stub implementations
and compiled code into a single python package that cannot be
distributed easily. This forces python projects using mesos to copy
protobufs around and forces a onerous dependency on anyone who would
like to do a pure python binding.
- How was this problem solved?
The current python package has been split into two separate packages:
- mesos.interface (stub implementations and protobufs)
- mesos.native (old _mesos module)
These are python meta-packages and can be installed as separate
pieces. The `mesos.interface` package will be hosted on pypi and can
be installed via. easy_install and pip.
See https://issues.apache.org/jira/browse/MESOS-857 and
https://reviews.apache.org/r/23224/.
- Why should I care?
These changes are not backwards compatible. With 0.20.0 you will need
to change how you use the python bindings. Here's a quick overview:
mesos.Scheduler -> mesos.interface.Scheduler
mesos.mesos_pb2 -> mesos.interface.mesos_pb2
mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
For more details, you can take a look at the examples in `src/examples/python".
Re: Python bindings are changing!
Posted by Tom Arnfeld <to...@duedil.com>.
Woah, this is really awesome Thomas! Especially the pip install ;-)
Looking forward to bringing pesos up to speed with this.
On 1 August 2014 21:30, Jie Yu <yu...@gmail.com> wrote:
> Thomas,
>
> Thank you for the heads-up. One question: what if mesos and python binding
> have different versions? For example, is it ok to use a 0.19.0 python
> binding and having a 0.20.0 mesos? Same question for the reverse.
>
> - Jie
>
>
> On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org>
> wrote:
>
>> - What problem are we trying to solve?
>>
>> Currently, the python bindings group protobufs, stub implementations
>> and compiled code into a single python package that cannot be
>> distributed easily. This forces python projects using mesos to copy
>> protobufs around and forces a onerous dependency on anyone who would
>> like to do a pure python binding.
>>
>> - How was this problem solved?
>>
>> The current python package has been split into two separate packages:
>>
>> - mesos.interface (stub implementations and protobufs)
>> - mesos.native (old _mesos module)
>>
>> These are python meta-packages and can be installed as separate
>> pieces. The `mesos.interface` package will be hosted on pypi and can
>> be installed via. easy_install and pip.
>>
>> See https://issues.apache.org/jira/browse/MESOS-857 and
>> https://reviews.apache.org/r/23224/.
>>
>> - Why should I care?
>>
>> These changes are not backwards compatible. With 0.20.0 you will need
>> to change how you use the python bindings. Here's a quick overview:
>>
>> mesos.Scheduler -> mesos.interface.Scheduler
>> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
>> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
>>
>> For more details, you can take a look at the examples in
>> `src/examples/python".
>>
>
>
Re: Python bindings are changing!
Posted by Thomas Rampelberg <th...@saunter.org>.
There is one important step to do during the upgrade:
- Upgrade your python framework and executor. You'll be able to either
import using the new configuration or the old. Replace the existing
imports with this:
try:
from mesos.native import MesosExecutorDriver, MesosSchedulerDriver
from mesos.interface import Executor, Scheduler
from mesos.interface import mesos_pb2
except ImportError:
from mesos import Executor, MesosExecutorDriver,
MesosSchedulerDriver, Scheduler
import mesos_pb2
Then, you'll be able to follow the normal upgrade instructions.
Following the ones from the 0.18.x -> 0.19.x instructions:
- Install the new master binaries and restart the masters.
- Install the new slave binaries and restart the slaves.
- Upgrade the schedulers by installing the latest python egg.
- Restart the schedulers.
- Upgrade the executors by installing the latest python egg.
On Wed, Aug 13, 2014 at 10:19 AM, Jie Yu <yu...@gmail.com> wrote:
> Thomas,
>
> We are about to cut the 0.20.0 release. I am updating the docs/upgrades.md.
> Regarding the python binding change, can you let us know the exact deploy
> procedure if someone is using the old python binding?
>
> More specifically, say all masters/slaves/executors/schedulers are using
> 0.19.1. What should we upgrade first? For example
> 1) If we upgrade the slaves first (libmesos.so will be updated), what
> happens if the new slave launches an old executor? (here, you actually have
> two cases: a) the executor bundles libmesos.so; b) the execute does not
> bundle libmesos.so)
> 2) Same question for framework, which one should we upgrade first (scheduler
> or master)?
>
> Also, would you please test all these scenarios so that we can provide
> correct guidance to the community. Thanks a lot!
>
> - Jie
>
>
> On Sun, Aug 3, 2014 at 11:18 PM, Benjamin Mahler <be...@gmail.com>
> wrote:
>>>
>>> It might work to use
>>> 0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
>>
>>
>> Is there a deprecation cycle? How should folks be upgrading Python
>> schedulers
>> and executors to 0.20.0 if they are not statically bundling libmesos?
>> Is there an upgrade order required?
>>
>> We will need to document this carefully in the upgrade procedures for
>> 0.20.0.
>>
>> Thanks for cleaning this up Thomas! Appreciate it.
>>
>> On Sat, Aug 2, 2014 at 12:12 PM, Thomas Rampelberg <th...@saunter.org>
>> wrote:
>>>
>>> You'll want to match the python bindings to your mesos version as the
>>> functionality is coming from libmesos itself. It might work to use
>>> 0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
>>>
>>>
>>> On Fri, Aug 1, 2014 at 1:30 PM, Jie Yu <yu...@gmail.com> wrote:
>>> > Thomas,
>>> >
>>> > Thank you for the heads-up. One question: what if mesos and python
>>> > binding
>>> > have different versions? For example, is it ok to use a 0.19.0 python
>>> > binding and having a 0.20.0 mesos? Same question for the reverse.
>>> >
>>> > - Jie
>>> >
>>> >
>>> > On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org>
>>> > wrote:
>>> >
>>> >> - What problem are we trying to solve?
>>> >>
>>> >> Currently, the python bindings group protobufs, stub implementations
>>> >> and compiled code into a single python package that cannot be
>>> >> distributed easily. This forces python projects using mesos to copy
>>> >> protobufs around and forces a onerous dependency on anyone who would
>>> >> like to do a pure python binding.
>>> >>
>>> >> - How was this problem solved?
>>> >>
>>> >> The current python package has been split into two separate packages:
>>> >>
>>> >> - mesos.interface (stub implementations and protobufs)
>>> >> - mesos.native (old _mesos module)
>>> >>
>>> >> These are python meta-packages and can be installed as separate
>>> >> pieces. The `mesos.interface` package will be hosted on pypi and can
>>> >> be installed via. easy_install and pip.
>>> >>
>>> >> See https://issues.apache.org/jira/browse/MESOS-857 and
>>> >> https://reviews.apache.org/r/23224/.
>>> >>
>>> >> - Why should I care?
>>> >>
>>> >> These changes are not backwards compatible. With 0.20.0 you will need
>>> >> to change how you use the python bindings. Here's a quick overview:
>>> >>
>>> >> mesos.Scheduler -> mesos.interface.Scheduler
>>> >> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
>>> >> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
>>> >>
>>> >> For more details, you can take a look at the examples in
>>> >> `src/examples/python".
>>> >>
>>
>>
>
Re: Python bindings are changing!
Posted by Benjamin Mahler <be...@gmail.com>.
Hey Thomas,
Could you share the scripts you're using to publish to pypi? It's not part
of the release process as of yet:
http://mesos.apache.org/documentation/latest/release-guide/
The 0.21.1 eggs were never published:
https://issues.apache.org/jira/browse/MESOS-2310
On Thu, Aug 14, 2014 at 4:50 PM, Thomas Rampelberg <th...@saunter.org>
wrote:
> > More specifically, say all masters/slaves/executors/schedulers are using
> > 0.19.1. What should we upgrade first? For example
> > 1) If we upgrade the slaves first (libmesos.so will be updated), what
> > happens if the new slave launches an old executor? (here, you actually
> have
> > two cases: a) the executor bundles libmesos.so; b) the execute does not
> > bundle libmesos.so)
>
> The python egg statically links libmesos.so. If the new slave launches
> an executor running the old egg, your slave will be running 0.20 and
> the executor will be on 0.19.
>
> > 2) Same question for framework, which one should we upgrade first
> (scheduler
> > or master)?
>
> There's nothing different here wrt. python.
>
Re: Python bindings are changing!
Posted by Benjamin Mahler <be...@gmail.com>.
Hey Thomas,
Could you share the scripts you're using to publish to pypi? It's not part
of the release process as of yet:
http://mesos.apache.org/documentation/latest/release-guide/
The 0.21.1 eggs were never published:
https://issues.apache.org/jira/browse/MESOS-2310
On Thu, Aug 14, 2014 at 4:50 PM, Thomas Rampelberg <th...@saunter.org>
wrote:
> > More specifically, say all masters/slaves/executors/schedulers are using
> > 0.19.1. What should we upgrade first? For example
> > 1) If we upgrade the slaves first (libmesos.so will be updated), what
> > happens if the new slave launches an old executor? (here, you actually
> have
> > two cases: a) the executor bundles libmesos.so; b) the execute does not
> > bundle libmesos.so)
>
> The python egg statically links libmesos.so. If the new slave launches
> an executor running the old egg, your slave will be running 0.20 and
> the executor will be on 0.19.
>
> > 2) Same question for framework, which one should we upgrade first
> (scheduler
> > or master)?
>
> There's nothing different here wrt. python.
>
Re: Python bindings are changing!
Posted by Thomas Rampelberg <th...@saunter.org>.
> More specifically, say all masters/slaves/executors/schedulers are using
> 0.19.1. What should we upgrade first? For example
> 1) If we upgrade the slaves first (libmesos.so will be updated), what
> happens if the new slave launches an old executor? (here, you actually have
> two cases: a) the executor bundles libmesos.so; b) the execute does not
> bundle libmesos.so)
The python egg statically links libmesos.so. If the new slave launches
an executor running the old egg, your slave will be running 0.20 and
the executor will be on 0.19.
> 2) Same question for framework, which one should we upgrade first (scheduler
> or master)?
There's nothing different here wrt. python.
Re: Python bindings are changing!
Posted by Thomas Rampelberg <th...@saunter.org>.
There is one important step to do during the upgrade:
- Upgrade your python framework and executor. You'll be able to either
import using the new configuration or the old. Replace the existing
imports with this:
try:
from mesos.native import MesosExecutorDriver, MesosSchedulerDriver
from mesos.interface import Executor, Scheduler
from mesos.interface import mesos_pb2
except ImportError:
from mesos import Executor, MesosExecutorDriver,
MesosSchedulerDriver, Scheduler
import mesos_pb2
Then, you'll be able to follow the normal upgrade instructions.
Following the ones from the 0.18.x -> 0.19.x instructions:
- Install the new master binaries and restart the masters.
- Install the new slave binaries and restart the slaves.
- Upgrade the schedulers by installing the latest python egg.
- Restart the schedulers.
- Upgrade the executors by installing the latest python egg.
On Wed, Aug 13, 2014 at 10:19 AM, Jie Yu <yu...@gmail.com> wrote:
> Thomas,
>
> We are about to cut the 0.20.0 release. I am updating the docs/upgrades.md.
> Regarding the python binding change, can you let us know the exact deploy
> procedure if someone is using the old python binding?
>
> More specifically, say all masters/slaves/executors/schedulers are using
> 0.19.1. What should we upgrade first? For example
> 1) If we upgrade the slaves first (libmesos.so will be updated), what
> happens if the new slave launches an old executor? (here, you actually have
> two cases: a) the executor bundles libmesos.so; b) the execute does not
> bundle libmesos.so)
> 2) Same question for framework, which one should we upgrade first (scheduler
> or master)?
>
> Also, would you please test all these scenarios so that we can provide
> correct guidance to the community. Thanks a lot!
>
> - Jie
>
>
> On Sun, Aug 3, 2014 at 11:18 PM, Benjamin Mahler <be...@gmail.com>
> wrote:
>>>
>>> It might work to use
>>> 0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
>>
>>
>> Is there a deprecation cycle? How should folks be upgrading Python
>> schedulers
>> and executors to 0.20.0 if they are not statically bundling libmesos?
>> Is there an upgrade order required?
>>
>> We will need to document this carefully in the upgrade procedures for
>> 0.20.0.
>>
>> Thanks for cleaning this up Thomas! Appreciate it.
>>
>> On Sat, Aug 2, 2014 at 12:12 PM, Thomas Rampelberg <th...@saunter.org>
>> wrote:
>>>
>>> You'll want to match the python bindings to your mesos version as the
>>> functionality is coming from libmesos itself. It might work to use
>>> 0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
>>>
>>>
>>> On Fri, Aug 1, 2014 at 1:30 PM, Jie Yu <yu...@gmail.com> wrote:
>>> > Thomas,
>>> >
>>> > Thank you for the heads-up. One question: what if mesos and python
>>> > binding
>>> > have different versions? For example, is it ok to use a 0.19.0 python
>>> > binding and having a 0.20.0 mesos? Same question for the reverse.
>>> >
>>> > - Jie
>>> >
>>> >
>>> > On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org>
>>> > wrote:
>>> >
>>> >> - What problem are we trying to solve?
>>> >>
>>> >> Currently, the python bindings group protobufs, stub implementations
>>> >> and compiled code into a single python package that cannot be
>>> >> distributed easily. This forces python projects using mesos to copy
>>> >> protobufs around and forces a onerous dependency on anyone who would
>>> >> like to do a pure python binding.
>>> >>
>>> >> - How was this problem solved?
>>> >>
>>> >> The current python package has been split into two separate packages:
>>> >>
>>> >> - mesos.interface (stub implementations and protobufs)
>>> >> - mesos.native (old _mesos module)
>>> >>
>>> >> These are python meta-packages and can be installed as separate
>>> >> pieces. The `mesos.interface` package will be hosted on pypi and can
>>> >> be installed via. easy_install and pip.
>>> >>
>>> >> See https://issues.apache.org/jira/browse/MESOS-857 and
>>> >> https://reviews.apache.org/r/23224/.
>>> >>
>>> >> - Why should I care?
>>> >>
>>> >> These changes are not backwards compatible. With 0.20.0 you will need
>>> >> to change how you use the python bindings. Here's a quick overview:
>>> >>
>>> >> mesos.Scheduler -> mesos.interface.Scheduler
>>> >> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
>>> >> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
>>> >>
>>> >> For more details, you can take a look at the examples in
>>> >> `src/examples/python".
>>> >>
>>
>>
>
Re: Python bindings are changing!
Posted by Thomas Rampelberg <th...@saunter.org>.
> More specifically, say all masters/slaves/executors/schedulers are using
> 0.19.1. What should we upgrade first? For example
> 1) If we upgrade the slaves first (libmesos.so will be updated), what
> happens if the new slave launches an old executor? (here, you actually have
> two cases: a) the executor bundles libmesos.so; b) the execute does not
> bundle libmesos.so)
The python egg statically links libmesos.so. If the new slave launches
an executor running the old egg, your slave will be running 0.20 and
the executor will be on 0.19.
> 2) Same question for framework, which one should we upgrade first (scheduler
> or master)?
There's nothing different here wrt. python.
Re: Python bindings are changing!
Posted by Jie Yu <yu...@gmail.com>.
Thomas,
We are about to cut the 0.20.0 release. I am updating the docs/upgrades.md.
Regarding the python binding change, can you let us know the exact deploy
procedure if someone is using the old python binding?
More specifically, say all masters/slaves/executors/schedulers are using
0.19.1. What should we upgrade first? For example
1) If we upgrade the slaves first (libmesos.so will be updated), what
happens if the new slave launches an old executor? (here, you actually have
two cases: a) the executor bundles libmesos.so; b) the execute does not
bundle libmesos.so)
2) Same question for framework, which one should we upgrade first
(scheduler or master)?
Also, would you please test all these scenarios so that we can provide
correct guidance to the community. Thanks a lot!
- Jie
On Sun, Aug 3, 2014 at 11:18 PM, Benjamin Mahler <be...@gmail.com>
wrote:
> It might work to use
>> 0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
>>
>
> Is there a deprecation cycle? How should folks be upgrading Python
> schedulers
> and executors to 0.20.0 if they are not statically bundling libmesos?
> Is there an upgrade order required?
>
> We will need to document this carefully in the upgrade procedures for
> 0.20.0.
>
> Thanks for cleaning this up Thomas! Appreciate it.
>
> On Sat, Aug 2, 2014 at 12:12 PM, Thomas Rampelberg <th...@saunter.org>
> wrote:
>
>> You'll want to match the python bindings to your mesos version as the
>> functionality is coming from libmesos itself. It might work to use
>> 0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
>>
>>
>> On Fri, Aug 1, 2014 at 1:30 PM, Jie Yu <yu...@gmail.com> wrote:
>> > Thomas,
>> >
>> > Thank you for the heads-up. One question: what if mesos and python
>> binding
>> > have different versions? For example, is it ok to use a 0.19.0 python
>> > binding and having a 0.20.0 mesos? Same question for the reverse.
>> >
>> > - Jie
>> >
>> >
>> > On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org>
>> > wrote:
>> >
>> >> - What problem are we trying to solve?
>> >>
>> >> Currently, the python bindings group protobufs, stub implementations
>> >> and compiled code into a single python package that cannot be
>> >> distributed easily. This forces python projects using mesos to copy
>> >> protobufs around and forces a onerous dependency on anyone who would
>> >> like to do a pure python binding.
>> >>
>> >> - How was this problem solved?
>> >>
>> >> The current python package has been split into two separate packages:
>> >>
>> >> - mesos.interface (stub implementations and protobufs)
>> >> - mesos.native (old _mesos module)
>> >>
>> >> These are python meta-packages and can be installed as separate
>> >> pieces. The `mesos.interface` package will be hosted on pypi and can
>> >> be installed via. easy_install and pip.
>> >>
>> >> See https://issues.apache.org/jira/browse/MESOS-857 and
>> >> https://reviews.apache.org/r/23224/.
>> >>
>> >> - Why should I care?
>> >>
>> >> These changes are not backwards compatible. With 0.20.0 you will need
>> >> to change how you use the python bindings. Here's a quick overview:
>> >>
>> >> mesos.Scheduler -> mesos.interface.Scheduler
>> >> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
>> >> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
>> >>
>> >> For more details, you can take a look at the examples in
>> >> `src/examples/python".
>> >>
>>
>
>
Re: Python bindings are changing!
Posted by Jie Yu <yu...@gmail.com>.
Thomas,
We are about to cut the 0.20.0 release. I am updating the docs/upgrades.md.
Regarding the python binding change, can you let us know the exact deploy
procedure if someone is using the old python binding?
More specifically, say all masters/slaves/executors/schedulers are using
0.19.1. What should we upgrade first? For example
1) If we upgrade the slaves first (libmesos.so will be updated), what
happens if the new slave launches an old executor? (here, you actually have
two cases: a) the executor bundles libmesos.so; b) the execute does not
bundle libmesos.so)
2) Same question for framework, which one should we upgrade first
(scheduler or master)?
Also, would you please test all these scenarios so that we can provide
correct guidance to the community. Thanks a lot!
- Jie
On Sun, Aug 3, 2014 at 11:18 PM, Benjamin Mahler <be...@gmail.com>
wrote:
> It might work to use
>> 0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
>>
>
> Is there a deprecation cycle? How should folks be upgrading Python
> schedulers
> and executors to 0.20.0 if they are not statically bundling libmesos?
> Is there an upgrade order required?
>
> We will need to document this carefully in the upgrade procedures for
> 0.20.0.
>
> Thanks for cleaning this up Thomas! Appreciate it.
>
> On Sat, Aug 2, 2014 at 12:12 PM, Thomas Rampelberg <th...@saunter.org>
> wrote:
>
>> You'll want to match the python bindings to your mesos version as the
>> functionality is coming from libmesos itself. It might work to use
>> 0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
>>
>>
>> On Fri, Aug 1, 2014 at 1:30 PM, Jie Yu <yu...@gmail.com> wrote:
>> > Thomas,
>> >
>> > Thank you for the heads-up. One question: what if mesos and python
>> binding
>> > have different versions? For example, is it ok to use a 0.19.0 python
>> > binding and having a 0.20.0 mesos? Same question for the reverse.
>> >
>> > - Jie
>> >
>> >
>> > On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org>
>> > wrote:
>> >
>> >> - What problem are we trying to solve?
>> >>
>> >> Currently, the python bindings group protobufs, stub implementations
>> >> and compiled code into a single python package that cannot be
>> >> distributed easily. This forces python projects using mesos to copy
>> >> protobufs around and forces a onerous dependency on anyone who would
>> >> like to do a pure python binding.
>> >>
>> >> - How was this problem solved?
>> >>
>> >> The current python package has been split into two separate packages:
>> >>
>> >> - mesos.interface (stub implementations and protobufs)
>> >> - mesos.native (old _mesos module)
>> >>
>> >> These are python meta-packages and can be installed as separate
>> >> pieces. The `mesos.interface` package will be hosted on pypi and can
>> >> be installed via. easy_install and pip.
>> >>
>> >> See https://issues.apache.org/jira/browse/MESOS-857 and
>> >> https://reviews.apache.org/r/23224/.
>> >>
>> >> - Why should I care?
>> >>
>> >> These changes are not backwards compatible. With 0.20.0 you will need
>> >> to change how you use the python bindings. Here's a quick overview:
>> >>
>> >> mesos.Scheduler -> mesos.interface.Scheduler
>> >> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
>> >> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
>> >>
>> >> For more details, you can take a look at the examples in
>> >> `src/examples/python".
>> >>
>>
>
>
Re: Python bindings are changing!
Posted by Benjamin Mahler <be...@gmail.com>.
>
> It might work to use
> 0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
>
Is there a deprecation cycle? How should folks be upgrading Python
schedulers
and executors to 0.20.0 if they are not statically bundling libmesos?
Is there an upgrade order required?
We will need to document this carefully in the upgrade procedures for
0.20.0.
Thanks for cleaning this up Thomas! Appreciate it.
On Sat, Aug 2, 2014 at 12:12 PM, Thomas Rampelberg <th...@saunter.org>
wrote:
> You'll want to match the python bindings to your mesos version as the
> functionality is coming from libmesos itself. It might work to use
> 0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
>
>
> On Fri, Aug 1, 2014 at 1:30 PM, Jie Yu <yu...@gmail.com> wrote:
> > Thomas,
> >
> > Thank you for the heads-up. One question: what if mesos and python
> binding
> > have different versions? For example, is it ok to use a 0.19.0 python
> > binding and having a 0.20.0 mesos? Same question for the reverse.
> >
> > - Jie
> >
> >
> > On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org>
> > wrote:
> >
> >> - What problem are we trying to solve?
> >>
> >> Currently, the python bindings group protobufs, stub implementations
> >> and compiled code into a single python package that cannot be
> >> distributed easily. This forces python projects using mesos to copy
> >> protobufs around and forces a onerous dependency on anyone who would
> >> like to do a pure python binding.
> >>
> >> - How was this problem solved?
> >>
> >> The current python package has been split into two separate packages:
> >>
> >> - mesos.interface (stub implementations and protobufs)
> >> - mesos.native (old _mesos module)
> >>
> >> These are python meta-packages and can be installed as separate
> >> pieces. The `mesos.interface` package will be hosted on pypi and can
> >> be installed via. easy_install and pip.
> >>
> >> See https://issues.apache.org/jira/browse/MESOS-857 and
> >> https://reviews.apache.org/r/23224/.
> >>
> >> - Why should I care?
> >>
> >> These changes are not backwards compatible. With 0.20.0 you will need
> >> to change how you use the python bindings. Here's a quick overview:
> >>
> >> mesos.Scheduler -> mesos.interface.Scheduler
> >> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
> >> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
> >>
> >> For more details, you can take a look at the examples in
> >> `src/examples/python".
> >>
>
Re: Python bindings are changing!
Posted by Benjamin Mahler <be...@gmail.com>.
>
> It might work to use
> 0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
>
Is there a deprecation cycle? How should folks be upgrading Python
schedulers
and executors to 0.20.0 if they are not statically bundling libmesos?
Is there an upgrade order required?
We will need to document this carefully in the upgrade procedures for
0.20.0.
Thanks for cleaning this up Thomas! Appreciate it.
On Sat, Aug 2, 2014 at 12:12 PM, Thomas Rampelberg <th...@saunter.org>
wrote:
> You'll want to match the python bindings to your mesos version as the
> functionality is coming from libmesos itself. It might work to use
> 0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
>
>
> On Fri, Aug 1, 2014 at 1:30 PM, Jie Yu <yu...@gmail.com> wrote:
> > Thomas,
> >
> > Thank you for the heads-up. One question: what if mesos and python
> binding
> > have different versions? For example, is it ok to use a 0.19.0 python
> > binding and having a 0.20.0 mesos? Same question for the reverse.
> >
> > - Jie
> >
> >
> > On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org>
> > wrote:
> >
> >> - What problem are we trying to solve?
> >>
> >> Currently, the python bindings group protobufs, stub implementations
> >> and compiled code into a single python package that cannot be
> >> distributed easily. This forces python projects using mesos to copy
> >> protobufs around and forces a onerous dependency on anyone who would
> >> like to do a pure python binding.
> >>
> >> - How was this problem solved?
> >>
> >> The current python package has been split into two separate packages:
> >>
> >> - mesos.interface (stub implementations and protobufs)
> >> - mesos.native (old _mesos module)
> >>
> >> These are python meta-packages and can be installed as separate
> >> pieces. The `mesos.interface` package will be hosted on pypi and can
> >> be installed via. easy_install and pip.
> >>
> >> See https://issues.apache.org/jira/browse/MESOS-857 and
> >> https://reviews.apache.org/r/23224/.
> >>
> >> - Why should I care?
> >>
> >> These changes are not backwards compatible. With 0.20.0 you will need
> >> to change how you use the python bindings. Here's a quick overview:
> >>
> >> mesos.Scheduler -> mesos.interface.Scheduler
> >> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
> >> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
> >>
> >> For more details, you can take a look at the examples in
> >> `src/examples/python".
> >>
>
Re: Python bindings are changing!
Posted by Thomas Rampelberg <th...@saunter.org>.
You'll want to match the python bindings to your mesos version as the
functionality is coming from libmesos itself. It might work to use
0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
On Fri, Aug 1, 2014 at 1:30 PM, Jie Yu <yu...@gmail.com> wrote:
> Thomas,
>
> Thank you for the heads-up. One question: what if mesos and python binding
> have different versions? For example, is it ok to use a 0.19.0 python
> binding and having a 0.20.0 mesos? Same question for the reverse.
>
> - Jie
>
>
> On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org>
> wrote:
>
>> - What problem are we trying to solve?
>>
>> Currently, the python bindings group protobufs, stub implementations
>> and compiled code into a single python package that cannot be
>> distributed easily. This forces python projects using mesos to copy
>> protobufs around and forces a onerous dependency on anyone who would
>> like to do a pure python binding.
>>
>> - How was this problem solved?
>>
>> The current python package has been split into two separate packages:
>>
>> - mesos.interface (stub implementations and protobufs)
>> - mesos.native (old _mesos module)
>>
>> These are python meta-packages and can be installed as separate
>> pieces. The `mesos.interface` package will be hosted on pypi and can
>> be installed via. easy_install and pip.
>>
>> See https://issues.apache.org/jira/browse/MESOS-857 and
>> https://reviews.apache.org/r/23224/.
>>
>> - Why should I care?
>>
>> These changes are not backwards compatible. With 0.20.0 you will need
>> to change how you use the python bindings. Here's a quick overview:
>>
>> mesos.Scheduler -> mesos.interface.Scheduler
>> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
>> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
>>
>> For more details, you can take a look at the examples in
>> `src/examples/python".
>>
Re: Python bindings are changing!
Posted by Tom Arnfeld <to...@duedil.com>.
Woah, this is really awesome Thomas! Especially the pip install ;-)
Looking forward to bringing pesos up to speed with this.
On 1 August 2014 21:30, Jie Yu <yu...@gmail.com> wrote:
> Thomas,
>
> Thank you for the heads-up. One question: what if mesos and python binding
> have different versions? For example, is it ok to use a 0.19.0 python
> binding and having a 0.20.0 mesos? Same question for the reverse.
>
> - Jie
>
>
> On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org>
> wrote:
>
>> - What problem are we trying to solve?
>>
>> Currently, the python bindings group protobufs, stub implementations
>> and compiled code into a single python package that cannot be
>> distributed easily. This forces python projects using mesos to copy
>> protobufs around and forces a onerous dependency on anyone who would
>> like to do a pure python binding.
>>
>> - How was this problem solved?
>>
>> The current python package has been split into two separate packages:
>>
>> - mesos.interface (stub implementations and protobufs)
>> - mesos.native (old _mesos module)
>>
>> These are python meta-packages and can be installed as separate
>> pieces. The `mesos.interface` package will be hosted on pypi and can
>> be installed via. easy_install and pip.
>>
>> See https://issues.apache.org/jira/browse/MESOS-857 and
>> https://reviews.apache.org/r/23224/.
>>
>> - Why should I care?
>>
>> These changes are not backwards compatible. With 0.20.0 you will need
>> to change how you use the python bindings. Here's a quick overview:
>>
>> mesos.Scheduler -> mesos.interface.Scheduler
>> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
>> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
>>
>> For more details, you can take a look at the examples in
>> `src/examples/python".
>>
>
>
Re: Python bindings are changing!
Posted by Thomas Rampelberg <th...@saunter.org>.
You'll want to match the python bindings to your mesos version as the
functionality is coming from libmesos itself. It might work to use
0.19 with a 0.20 mesos (or visa versa), but there be dragons =)
On Fri, Aug 1, 2014 at 1:30 PM, Jie Yu <yu...@gmail.com> wrote:
> Thomas,
>
> Thank you for the heads-up. One question: what if mesos and python binding
> have different versions? For example, is it ok to use a 0.19.0 python
> binding and having a 0.20.0 mesos? Same question for the reverse.
>
> - Jie
>
>
> On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org>
> wrote:
>
>> - What problem are we trying to solve?
>>
>> Currently, the python bindings group protobufs, stub implementations
>> and compiled code into a single python package that cannot be
>> distributed easily. This forces python projects using mesos to copy
>> protobufs around and forces a onerous dependency on anyone who would
>> like to do a pure python binding.
>>
>> - How was this problem solved?
>>
>> The current python package has been split into two separate packages:
>>
>> - mesos.interface (stub implementations and protobufs)
>> - mesos.native (old _mesos module)
>>
>> These are python meta-packages and can be installed as separate
>> pieces. The `mesos.interface` package will be hosted on pypi and can
>> be installed via. easy_install and pip.
>>
>> See https://issues.apache.org/jira/browse/MESOS-857 and
>> https://reviews.apache.org/r/23224/.
>>
>> - Why should I care?
>>
>> These changes are not backwards compatible. With 0.20.0 you will need
>> to change how you use the python bindings. Here's a quick overview:
>>
>> mesos.Scheduler -> mesos.interface.Scheduler
>> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
>> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
>>
>> For more details, you can take a look at the examples in
>> `src/examples/python".
>>
Re: Python bindings are changing!
Posted by Jie Yu <yu...@gmail.com>.
Thomas,
Thank you for the heads-up. One question: what if mesos and python binding
have different versions? For example, is it ok to use a 0.19.0 python
binding and having a 0.20.0 mesos? Same question for the reverse.
- Jie
On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org>
wrote:
> - What problem are we trying to solve?
>
> Currently, the python bindings group protobufs, stub implementations
> and compiled code into a single python package that cannot be
> distributed easily. This forces python projects using mesos to copy
> protobufs around and forces a onerous dependency on anyone who would
> like to do a pure python binding.
>
> - How was this problem solved?
>
> The current python package has been split into two separate packages:
>
> - mesos.interface (stub implementations and protobufs)
> - mesos.native (old _mesos module)
>
> These are python meta-packages and can be installed as separate
> pieces. The `mesos.interface` package will be hosted on pypi and can
> be installed via. easy_install and pip.
>
> See https://issues.apache.org/jira/browse/MESOS-857 and
> https://reviews.apache.org/r/23224/.
>
> - Why should I care?
>
> These changes are not backwards compatible. With 0.20.0 you will need
> to change how you use the python bindings. Here's a quick overview:
>
> mesos.Scheduler -> mesos.interface.Scheduler
> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
>
> For more details, you can take a look at the examples in
> `src/examples/python".
>
Re: Python bindings are changing!
Posted by Thomas Rampelberg <th...@saunter.org>.
`pip install mesos.interface` now works =)
On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org> wrote:
> - What problem are we trying to solve?
>
> Currently, the python bindings group protobufs, stub implementations
> and compiled code into a single python package that cannot be
> distributed easily. This forces python projects using mesos to copy
> protobufs around and forces a onerous dependency on anyone who would
> like to do a pure python binding.
>
> - How was this problem solved?
>
> The current python package has been split into two separate packages:
>
> - mesos.interface (stub implementations and protobufs)
> - mesos.native (old _mesos module)
>
> These are python meta-packages and can be installed as separate
> pieces. The `mesos.interface` package will be hosted on pypi and can
> be installed via. easy_install and pip.
>
> See https://issues.apache.org/jira/browse/MESOS-857 and
> https://reviews.apache.org/r/23224/.
>
> - Why should I care?
>
> These changes are not backwards compatible. With 0.20.0 you will need
> to change how you use the python bindings. Here's a quick overview:
>
> mesos.Scheduler -> mesos.interface.Scheduler
> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
>
> For more details, you can take a look at the examples in `src/examples/python".
Re: Python bindings are changing!
Posted by Jie Yu <yu...@gmail.com>.
Thomas,
Thank you for the heads-up. One question: what if mesos and python binding
have different versions? For example, is it ok to use a 0.19.0 python
binding and having a 0.20.0 mesos? Same question for the reverse.
- Jie
On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org>
wrote:
> - What problem are we trying to solve?
>
> Currently, the python bindings group protobufs, stub implementations
> and compiled code into a single python package that cannot be
> distributed easily. This forces python projects using mesos to copy
> protobufs around and forces a onerous dependency on anyone who would
> like to do a pure python binding.
>
> - How was this problem solved?
>
> The current python package has been split into two separate packages:
>
> - mesos.interface (stub implementations and protobufs)
> - mesos.native (old _mesos module)
>
> These are python meta-packages and can be installed as separate
> pieces. The `mesos.interface` package will be hosted on pypi and can
> be installed via. easy_install and pip.
>
> See https://issues.apache.org/jira/browse/MESOS-857 and
> https://reviews.apache.org/r/23224/.
>
> - Why should I care?
>
> These changes are not backwards compatible. With 0.20.0 you will need
> to change how you use the python bindings. Here's a quick overview:
>
> mesos.Scheduler -> mesos.interface.Scheduler
> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
>
> For more details, you can take a look at the examples in
> `src/examples/python".
>
Re: Python bindings are changing!
Posted by Thomas Rampelberg <th...@saunter.org>.
`pip install mesos.interface` now works =)
On Fri, Aug 1, 2014 at 9:37 AM, Thomas Rampelberg <th...@saunter.org> wrote:
> - What problem are we trying to solve?
>
> Currently, the python bindings group protobufs, stub implementations
> and compiled code into a single python package that cannot be
> distributed easily. This forces python projects using mesos to copy
> protobufs around and forces a onerous dependency on anyone who would
> like to do a pure python binding.
>
> - How was this problem solved?
>
> The current python package has been split into two separate packages:
>
> - mesos.interface (stub implementations and protobufs)
> - mesos.native (old _mesos module)
>
> These are python meta-packages and can be installed as separate
> pieces. The `mesos.interface` package will be hosted on pypi and can
> be installed via. easy_install and pip.
>
> See https://issues.apache.org/jira/browse/MESOS-857 and
> https://reviews.apache.org/r/23224/.
>
> - Why should I care?
>
> These changes are not backwards compatible. With 0.20.0 you will need
> to change how you use the python bindings. Here's a quick overview:
>
> mesos.Scheduler -> mesos.interface.Scheduler
> mesos.mesos_pb2 -> mesos.interface.mesos_pb2
> mesos.MesosSchedulerDriver -> mesos.native.MesosSchedulerDriver
>
> For more details, you can take a look at the examples in `src/examples/python".