spark-core "compile"-scope transitive-dependency on scalatest

[Ryan Williams <ry...@gmail.com>]

spark-core depends on spark-tags (compile scope) which depends on scalatest
(compile scope), so spark-core leaks test-deps into downstream libraries'
"compile"-scope classpath.

The cause is that spark-core has logical "test->test" and
"compile->compile" dependencies on spark-tags, but spark-tags publishes
both its test-oriented and non-test-oriented bits in its default
("compile") artifact.

spark-tags' test-bits should be in a "-tests"-JAR that spark-core can
"test"-scope depend on (in addition to "compile"-scope depending on
spark-tags as it does today).

SPARK-17807 <https://issues.apache.org/jira/browse/SPARK-17807> was "Not a
Problem"d but I don't think that's the right outcome; spark-core should not
be leaking test-deps into downstream libraries' classpaths when depended on
in "compile" scope.

Re: spark-core "compile"-scope transitive-dependency on scalatest

[Marcelo Vanzin <va...@cloudera.com>]

I posted a PR; the solution I suggested seems to work (and is simpler
than breaking spark-tags into multiple artifacts).

On Thu, Dec 15, 2016 at 4:46 PM, Ryan Williams
<ry...@gmail.com> wrote:
> ah I see this thread now, thanks; interestingly I don't think the solution
> I've proposed here (splitting spark-tags' test-bits into a "-tests" JAR and
> having spark-core "test"-depend on that) is discussed there.
>
> thanks for re-opening the JIRA; I can't promise a PR for it atm but I will
> think about it :)
>
> On Thu, Dec 15, 2016 at 7:41 PM Marcelo Vanzin <va...@cloudera.com> wrote:
>>
>> You're right; we had a discussion here recently about this.
>>
>> I'll re-open that bug, if you want to send a PR. (I think it's just a
>> matter of making the scalatest dependency "provided" in spark-tags, if
>> I remember the discussion.)
>>
>> On Thu, Dec 15, 2016 at 4:15 PM, Ryan Williams
>> <ry...@gmail.com> wrote:
>> > spark-core depends on spark-tags (compile scope) which depends on
>> > scalatest
>> > (compile scope), so spark-core leaks test-deps into downstream
>> > libraries'
>> > "compile"-scope classpath.
>> >
>> > The cause is that spark-core has logical "test->test" and
>> > "compile->compile"
>> > dependencies on spark-tags, but spark-tags publishes both its
>> > test-oriented
>> > and non-test-oriented bits in its default ("compile") artifact.
>> >
>> > spark-tags' test-bits should be in a "-tests"-JAR that spark-core can
>> > "test"-scope depend on (in addition to "compile"-scope depending on
>> > spark-tags as it does today).
>> >
>> > SPARK-17807 was "Not a Problem"d but I don't think that's the right
>> > outcome;
>> > spark-core should not be leaking test-deps into downstream libraries'
>> > classpaths when depended on in "compile" scope.
>> >
>>
>>
>>
>> --
>> Marcelo



-- 
Marcelo

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org

Re: spark-core "compile"-scope transitive-dependency on scalatest

[Ryan Williams <ry...@gmail.com>]

ah I see this thread
<http://apache-spark-developers-list.1001551.n3.nabble.com/Spark-has-a-compile-dependency-on-scalatest-td19639.html>
now, thanks; interestingly I don't think the solution I've proposed here
(splitting spark-tags' test-bits into a "-tests" JAR and having spark-core
"test"-depend on that) is discussed there.

thanks for re-opening the JIRA; I can't promise a PR for it atm but I will
think about it :)

On Thu, Dec 15, 2016 at 7:41 PM Marcelo Vanzin <va...@cloudera.com> wrote:

> You're right; we had a discussion here recently about this.
>
> I'll re-open that bug, if you want to send a PR. (I think it's just a
> matter of making the scalatest dependency "provided" in spark-tags, if
> I remember the discussion.)
>
> On Thu, Dec 15, 2016 at 4:15 PM, Ryan Williams
> <ry...@gmail.com> wrote:
> > spark-core depends on spark-tags (compile scope) which depends on
> scalatest
> > (compile scope), so spark-core leaks test-deps into downstream libraries'
> > "compile"-scope classpath.
> >
> > The cause is that spark-core has logical "test->test" and
> "compile->compile"
> > dependencies on spark-tags, but spark-tags publishes both its
> test-oriented
> > and non-test-oriented bits in its default ("compile") artifact.
> >
> > spark-tags' test-bits should be in a "-tests"-JAR that spark-core can
> > "test"-scope depend on (in addition to "compile"-scope depending on
> > spark-tags as it does today).
> >
> > SPARK-17807 was "Not a Problem"d but I don't think that's the right
> outcome;
> > spark-core should not be leaking test-deps into downstream libraries'
> > classpaths when depended on in "compile" scope.
> >
>
>
>
> --
> Marcelo
>

Re: spark-core "compile"-scope transitive-dependency on scalatest

[Marcelo Vanzin <va...@cloudera.com>]

You're right; we had a discussion here recently about this.

I'll re-open that bug, if you want to send a PR. (I think it's just a
matter of making the scalatest dependency "provided" in spark-tags, if
I remember the discussion.)

On Thu, Dec 15, 2016 at 4:15 PM, Ryan Williams
<ry...@gmail.com> wrote:
> spark-core depends on spark-tags (compile scope) which depends on scalatest
> (compile scope), so spark-core leaks test-deps into downstream libraries'
> "compile"-scope classpath.
>
> The cause is that spark-core has logical "test->test" and "compile->compile"
> dependencies on spark-tags, but spark-tags publishes both its test-oriented
> and non-test-oriented bits in its default ("compile") artifact.
>
> spark-tags' test-bits should be in a "-tests"-JAR that spark-core can
> "test"-scope depend on (in addition to "compile"-scope depending on
> spark-tags as it does today).
>
> SPARK-17807 was "Not a Problem"d but I don't think that's the right outcome;
> spark-core should not be leaking test-deps into downstream libraries'
> classpaths when depended on in "compile" scope.
>



-- 
Marcelo

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org