You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by algermissen1971 <al...@icloud.com> on 2015/06/04 22:24:19 UTC
Re: Roadmap for Spark with Kafka on Scala 2.11?
Hi Iulian,
On 26 May 2015, at 13:04, Iulian Dragoș <iu...@typesafe.com> wrote:
>
> On Tue, May 26, 2015 at 10:09 AM, algermissen1971 <al...@icloud.com> wrote:
> Hi,
>
> I am setting up a project that requires Kafka support and I wonder what the roadmap is for Scala 2.11 Support (including Kafka).
>
> Can we expect to see 2.11 support anytime soon?
>
> The upcoming 1.4 release (now at RC2) includes support for Kafka and Scala 2.11.6. It'd be great if you could give it a try. You can find the binaries (and staging repository including 2.11 artifacts) here:
>
> https://www.mail-archive.com/dev@spark.apache.org/msg09347.html
>
Feedback after a coupl eof days:
- I am using 1.4.0-rc4 now without problems
- Not used Kafka support yet
- I am using this with akka-2.3.11 and akka-http 1.0-RC3 (and sbt-assembly) and this has produced a dependency nightmare. I am even adding guava manually to the assembly because I just could not get sbt-assembly to not complain.
I am far from a good understanding of sbt / maven internals, but it seems that the ‘compile’ scope set in the spark POM for a lot of dependencies is somehow not honored and the libs end up causing conflicts in sbt-assembly.
(I am writing this to share experience, not to complain. Thanks for the great work!!)
onward...
Jan
> iulian
>
>
> Jan
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
>
>
> --
>
> --
> Iulian Dragos
>
> ------
> Reactive Apps on the JVM
> www.typesafe.com
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Roadmap for Spark with Kafka on Scala 2.11?
Posted by Tathagata Das <td...@databricks.com>.
But compile scope is supposed to be added to the assembly.
https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
On Thu, Jun 4, 2015 at 1:24 PM, algermissen1971 <al...@icloud.com>
wrote:
> Hi Iulian,
>
> On 26 May 2015, at 13:04, Iulian Dragoș <iu...@typesafe.com>
> wrote:
>
> >
> > On Tue, May 26, 2015 at 10:09 AM, algermissen1971 <
> algermissen1971@icloud.com> wrote:
> > Hi,
> >
> > I am setting up a project that requires Kafka support and I wonder what
> the roadmap is for Scala 2.11 Support (including Kafka).
> >
> > Can we expect to see 2.11 support anytime soon?
> >
> > The upcoming 1.4 release (now at RC2) includes support for Kafka and
> Scala 2.11.6. It'd be great if you could give it a try. You can find the
> binaries (and staging repository including 2.11 artifacts) here:
> >
> > https://www.mail-archive.com/dev@spark.apache.org/msg09347.html
> >
>
> Feedback after a coupl eof days:
>
> - I am using 1.4.0-rc4 now without problems
> - Not used Kafka support yet
> - I am using this with akka-2.3.11 and akka-http 1.0-RC3 (and
> sbt-assembly) and this has produced a dependency nightmare. I am even
> adding guava manually to the assembly because I just could not get
> sbt-assembly to not complain.
>
> I am far from a good understanding of sbt / maven internals, but it seems
> that the ‘compile’ scope set in the spark POM for a lot of dependencies is
> somehow not honored and the libs end up causing conflicts in sbt-assembly.
>
> (I am writing this to share experience, not to complain. Thanks for the
> great work!!)
>
> onward...
>
> Jan
>
>
>
>
>
> > iulian
> >
> >
> > Jan
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> > For additional commands, e-mail: user-help@spark.apache.org
> >
> >
> >
> >
> > --
> >
> > --
> > Iulian Dragos
> >
> > ------
> > Reactive Apps on the JVM
> > www.typesafe.com
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>