[jira] [Commented] (SPARK-33077) Spark 3 / Cats 2.2.0 classpath issue

["Sean R. Owen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-33077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17215461#comment-17215461 ] 

Sean R. Owen commented on SPARK-33077:
--------------------------------------

Adding a little flavor - the issue ends up being the dependency chain breeze 1.0 -> spire 1.17.0-M1 -> algebra 1.something -> cats 2.0.0-M4. Updating breeze to 1.1 would be cool enough but doesn't change spire's version. Manually managing it up to spire 1.17.0 does get cats to 2.1.1 which helps on this one. That last step is not hard in Spark, maybe mildly controversial just because we manually manage something that isn't used directly to resolve a downstream dependency. But not out of the question IMHO.

> Spark 3 / Cats 2.2.0 classpath issue
> ------------------------------------
>
>                 Key: SPARK-33077
>                 URL: https://issues.apache.org/jira/browse/SPARK-33077
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core, Spark Shell
>    Affects Versions: 3.0.1
>            Reporter: Sami Dalouche
>            Priority: Major
>
> A small project with minimal dependencies as well as instructions on how to reproduce the issue is available at:
> [https://github.com/samidalouche/spark3-cats220]
> Executing this code works fine with cats 2.1.1 but fails with cats 2.2.0, which is quite surprising since the spark and cats dependencies are pretty much distinct from each other.
>  
> {code:java}
> java.lang.NoSuchMethodError: 'void cats.kernel.CommutativeSemigroup.$init$(cats.kernel.CommutativeSemigroup)'
>  at cats.UnorderedFoldable$$anon$1.<init>(UnorderedFoldable.scala:78)
>  at cats.UnorderedFoldable$.<init>(UnorderedFoldable.scala:78)
>  at cats.UnorderedFoldable$.<clinit>(UnorderedFoldable.scala)
>  at cats.data.NonEmptyListInstances$$anon$2.<init>(NonEmptyList.scala:539)
>  at cats.data.NonEmptyListInstances.<init>(NonEmptyList.scala:539)
>  at cats.data.NonEmptyList$.<init>(NonEmptyList.scala:458)
>  at cats.data.NonEmptyList$.<clinit>(NonEmptyList.scala)
>  at catsspark.Boom$.assumeValid_$bang(boom.scala:19)
>  at catsspark.Boom$.boom(boom.scala:14)
>  ... 47 elided{code}
> Thanks in advance for looking into this.
> I submitted the same issue to cat's bug tracker: https://github.com/typelevel/cats/issues/3628



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org