You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Thodoris Zois <zo...@ics.forth.gr> on 2018/05/25 11:29:36 UTC
Spark version for Mesos 0.27.0
Hello,
Could you please tell me which version of Spark works with Apache Mesos
version 0.27.0? (I cannot find anything on docs at github)
Thank you very much,
Thodoris Zois
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Spark version for Mesos 0.27.0
Posted by Thodoris Zois <zo...@ics.forth.gr>.
Ok! Thank you very much!
- Thodoris
On Thu, 2018-05-31 at 11:30 +0200, Szuromi Tamás wrote:
> I see it in the Serenity docs. Anyway, I guess you are able to use
> the newest version of Spark with the Mesos 0.27 without any issues so
> don't have to dispense newer Spark features and fixes.
> Thodoris Zois <zo...@ics.forth.gr> ezt írta (időpont: 2018. máj. 31.,
> Cs, 11:22):
> > Hello,
> > The reason is that I want to make some tests and use the
> > oversubscription feature of Mesos along with Spark. Intel and
> > Mesosphere have built a project, called Serenity that actually
> > measures the usage slack on each Mesos agent and returns resources
> > to the cluster.
> > Unfortunately, Serenity is not compatible with newer versions of
> > Mesos...
> > - Thodoris
> >
> > On Thu, 2018-05-31 at 11:08 +0200, Szuromi Tamás wrote:
> > > Hey,
> > > I'm sure we used Spark 1.6 on Mesos 0.27 as well but at that time
> > > we used with fine-grained scheduling and not dynamic allocation.
> > > Also, newer Spark versions should work with older mesos versions
> > > like 0.27.
> > > Why do you have Mesos 0.27 btw?
> > >
> > > cheers,
> > > Tamas
> > > Thodoris Zois <zo...@ics.forth.gr> ezt írta (időpont: 2018. máj.
> > > 31., Cs, 1:56):
> > > > Hello,
> > > > I need Mesos 0.27 for specific purposes and unfortunately I
> > > > can’t use a newer version. Did you find anything? Could it be
> > > > Spark 1.6?
> > > >
> > > > Except that, from which version Spark supports dynamic
> > > > allocation on Mesos?
> > > >
> > > > - Thodoris
> > > > On 25 May 2018, at 16:06, Jacek Laskowski <ja...@japila.pl>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > > Mesos 0.27.0?! That's been a while. I'd search for the
> > > > > changes to pom.xml and see when the mesos dependency version
> > > > > changed. That'd give you the most precise answer. I think it
> > > > > could've been 1.5 or older.
> > > > >
> > > > > Pozdrawiam,
> > > > > Jacek Laskowski
> > > > > ----
> > > > > https://about.me/JacekLaskowski
> > > > > Mastering Spark SQL https://bit.ly/mastering-spark-sql
> > > > > Spark Structured Streaming https://bit.ly/spark-structured-st
> > > > > reaming
> > > > >
> > > > > Mastering Kafka Streams https://bit.ly/mastering-kafka-stream
> > > > > s
> > > > > Follow me at https://twitter.com/jaceklaskowski
> > > > >
> > > > >
> > > > > On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <zois@ics.fort
> > > > > h.gr> wrote:
> > > > > > Hello,
> > > > > >
> > > > > >
> > > > > >
> > > > > > Could you please tell me which version of Spark works with
> > > > > > Apache Mesos
> > > > > >
> > > > > > version 0.27.0? (I cannot find anything on docs at github)
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thank you very much,
> > > > > >
> > > > > > Thodoris Zois
> > > > > >
> > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------
> > > > > > ------------
> > > > > >
> > > > > > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> > > > > >
> > > > > >
> > > > > >
Re: Spark version for Mesos 0.27.0
Posted by Szuromi Tamás <tr...@gmail.com>.
I see it in the Serenity docs. Anyway, I guess you are able to use the
newest version of Spark with the Mesos 0.27 without any issues so don't
have to dispense newer Spark features and fixes.
Thodoris Zois <zo...@ics.forth.gr> ezt írta (időpont: 2018. máj. 31., Cs,
11:22):
> Hello,
>
> The reason is that I want to make some tests and use the oversubscription
> feature of Mesos along with Spark. Intel and Mesosphere have built a
> project, called Serenity that actually measures the usage slack on each
> Mesos agent and returns resources to the cluster.
>
> Unfortunately, Serenity is not compatible with newer versions of Mesos...
>
> - Thodoris
>
>
> On Thu, 2018-05-31 at 11:08 +0200, Szuromi Tamás wrote:
>
> Hey,
>
> I'm sure we used Spark 1.6 on Mesos 0.27 as well but at that time we used
> with fine-grained scheduling and not dynamic allocation. Also, newer Spark
> versions should work with older mesos versions like 0.27.
> Why do you have Mesos 0.27 btw?
>
> cheers,
> Tamas
>
> Thodoris Zois <zo...@ics.forth.gr> ezt írta (időpont: 2018. máj. 31., Cs,
> 1:56):
>
> Hello,
>
> I need Mesos 0.27 for specific purposes and unfortunately I can’t use a
> newer version. Did you find anything? Could it be Spark 1.6?
>
> Except that, from which version Spark supports dynamic allocation on
> Mesos?
>
> - Thodoris
>
> On 25 May 2018, at 16:06, Jacek Laskowski <ja...@japila.pl> wrote:
>
> Hi,
>
> Mesos 0.27.0?! That's been a while. I'd search for the changes to pom.xml
> and see when the mesos dependency version changed. That'd give you the most
> precise answer. I think it could've been 1.5 or older.
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://about.me/JacekLaskowski
> Mastering Spark SQL https://bit.ly/mastering-spark-sql
> Spark Structured Streaming https://bit.ly/spark-structured-streaming
> Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
> Follow me at https://twitter.com/jaceklaskowski
>
> On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <zo...@ics.forth.gr> wrote:
>
> Hello,
>
> Could you please tell me which version of Spark works with Apache Mesos
> version 0.27.0? (I cannot find anything on docs at github)
>
> Thank you very much,
> Thodoris Zois
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
>
Re: Spark version for Mesos 0.27.0
Posted by Thodoris Zois <zo...@ics.forth.gr>.
Hello,
The reason is that I want to make some tests and use the
oversubscription feature of Mesos along with Spark. Intel and
Mesosphere have built a project, called Serenity that actually measures
the usage slack on each Mesos agent and returns resources to the
cluster.
Unfortunately, Serenity is not compatible with newer versions of
Mesos...
- Thodoris
On Thu, 2018-05-31 at 11:08 +0200, Szuromi Tamás wrote:
> Hey,
> I'm sure we used Spark 1.6 on Mesos 0.27 as well but at that time we
> used with fine-grained scheduling and not dynamic allocation. Also,
> newer Spark versions should work with older mesos versions like 0.27.
> Why do you have Mesos 0.27 btw?
>
> cheers,
> Tamas
> Thodoris Zois <zo...@ics.forth.gr> ezt írta (időpont: 2018. máj. 31.,
> Cs, 1:56):
> > Hello,
> > I need Mesos 0.27 for specific purposes and unfortunately I can’t
> > use a newer version. Did you find anything? Could it be Spark 1.6?
> >
> > Except that, from which version Spark supports dynamic allocation
> > on Mesos?
> >
> > - Thodoris
> > On 25 May 2018, at 16:06, Jacek Laskowski <ja...@japila.pl> wrote:
> >
> > > Hi,
> > > Mesos 0.27.0?! That's been a while. I'd search for the changes to
> > > pom.xml and see when the mesos dependency version changed. That'd
> > > give you the most precise answer. I think it could've been 1.5 or
> > > older.
> > >
> > > Pozdrawiam,
> > > Jacek Laskowski
> > > ----
> > > https://about.me/JacekLaskowski
> > > Mastering Spark SQL https://bit.ly/mastering-spark-sql
> > > Spark Structured Streaming https://bit.ly/spark-structured-stream
> > > ing
> > >
> > > Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
> > > Follow me at https://twitter.com/jaceklaskowski
> > >
> > >
> > > On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <zois@ics.forth.gr
> > > > wrote:
> > > > Hello,
> > > >
> > > >
> > > >
> > > > Could you please tell me which version of Spark works with
> > > > Apache Mesos
> > > >
> > > > version 0.27.0? (I cannot find anything on docs at github)
> > > >
> > > >
> > > >
> > > > Thank you very much,
> > > >
> > > > Thodoris Zois
> > > >
> > > >
> > > >
> > > > -------------------------------------------------------------
> > > > --------
> > > >
> > > > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> > > >
> > > >
> > > >
Re: Spark version for Mesos 0.27.0
Posted by Szuromi Tamás <tr...@gmail.com>.
Hey,
I'm sure we used Spark 1.6 on Mesos 0.27 as well but at that time we used
with fine-grained scheduling and not dynamic allocation. Also, newer Spark
versions should work with older mesos versions like 0.27.
Why do you have Mesos 0.27 btw?
cheers,
Tamas
Thodoris Zois <zo...@ics.forth.gr> ezt írta (időpont: 2018. máj. 31., Cs,
1:56):
> Hello,
>
> I need Mesos 0.27 for specific purposes and unfortunately I can’t use a
> newer version. Did you find anything? Could it be Spark 1.6?
>
> Except that, from which version Spark supports dynamic allocation on
> Mesos?
>
> - Thodoris
>
> On 25 May 2018, at 16:06, Jacek Laskowski <ja...@japila.pl> wrote:
>
> Hi,
>
> Mesos 0.27.0?! That's been a while. I'd search for the changes to pom.xml
> and see when the mesos dependency version changed. That'd give you the most
> precise answer. I think it could've been 1.5 or older.
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://about.me/JacekLaskowski
> Mastering Spark SQL https://bit.ly/mastering-spark-sql
> Spark Structured Streaming https://bit.ly/spark-structured-streaming
> Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
> Follow me at https://twitter.com/jaceklaskowski
>
> On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <zo...@ics.forth.gr> wrote:
>
>> Hello,
>>
>> Could you please tell me which version of Spark works with Apache Mesos
>> version 0.27.0? (I cannot find anything on docs at github)
>>
>> Thank you very much,
>> Thodoris Zois
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
>
Re: Spark version for Mesos 0.27.0
Posted by Thodoris Zois <zo...@ics.forth.gr>.
Hello,
I need Mesos 0.27 for specific purposes and unfortunately I can’t use a newer version. Did you find anything? Could it be Spark 1.6?
Except that, from which version Spark supports dynamic allocation on Mesos?
- Thodoris
> On 25 May 2018, at 16:06, Jacek Laskowski <ja...@japila.pl> wrote:
>
> Hi,
>
> Mesos 0.27.0?! That's been a while. I'd search for the changes to pom.xml and see when the mesos dependency version changed. That'd give you the most precise answer. I think it could've been 1.5 or older.
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://about.me/JacekLaskowski
> Mastering Spark SQL https://bit.ly/mastering-spark-sql
> Spark Structured Streaming https://bit.ly/spark-structured-streaming
> Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
> Follow me at https://twitter.com/jaceklaskowski
>
>> On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <zo...@ics.forth.gr> wrote:
>> Hello,
>>
>> Could you please tell me which version of Spark works with Apache Mesos
>> version 0.27.0? (I cannot find anything on docs at github)
>>
>> Thank you very much,
>> Thodoris Zois
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>
Re: Spark version for Mesos 0.27.0
Posted by Jacek Laskowski <ja...@japila.pl>.
Hi,
Mesos 0.27.0?! That's been a while. I'd search for the changes to pom.xml
and see when the mesos dependency version changed. That'd give you the most
precise answer. I think it could've been 1.5 or older.
Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
Mastering Spark SQL https://bit.ly/mastering-spark-sql
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
Follow me at https://twitter.com/jaceklaskowski
On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <zo...@ics.forth.gr> wrote:
> Hello,
>
> Could you please tell me which version of Spark works with Apache Mesos
> version 0.27.0? (I cannot find anything on docs at github)
>
> Thank you very much,
> Thodoris Zois
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>