You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sandy Ryza (JIRA)" <ji...@apache.org> on 2014/11/13 04:49:42 UTC
[jira] [Commented] (SPARK-4375) Assembly built with Maven is
missing most of repl classes
[ https://issues.apache.org/jira/browse/SPARK-4375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209217#comment-14209217 ]
Sandy Ryza commented on SPARK-4375:
-----------------------------------
The issue here is that the activeByDefault Maven option doesn't work as expected: it turns off all default profiles if any profile is given on the command line. Circumventing this by removing the scala-2.10 profile and setting properties that get overridden when the scala-2.11 profile is turned on is pretty straightforward. Except in a couple cases:
* When using Scala 2.11, we don't want to build the external/kafka module
* When using Scala 2.11, we don't want the examples module to include the kafka module as a dependency
* When using Scala 2.11, we don't want the examples module to include algebird as a dependency
As far as I can tell, there aren't Maven doesn't let us do these things in any ways that aren't deeply hacky.
For dealing with the first issue, I had talked with Patrick about adding profiles for all of the external modules (including kafka). An issue with this is that the examples depend on all these modules, so we would need to create separate modules or directories for the examples tied to each of these modules.
Another thing we could do is add a "scala-2.10-only" profile that turns on building the kafka module and turns on the examples and examples dependencies that only work with scala-2.10.
> Assembly built with Maven is missing most of repl classes
> ---------------------------------------------------------
>
> Key: SPARK-4375
> URL: https://issues.apache.org/jira/browse/SPARK-4375
> Project: Spark
> Issue Type: Bug
> Affects Versions: 1.2.0
> Reporter: Sandy Ryza
> Priority: Blocker
>
> In particular, the ones in the split scala-2.10/scala-2.11 directories aren't being added
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org