[jira] [Commented] (SPARK-8725) Test modules in topological order in dev/run-tests and python/run-tests

["Josh Rosen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-8725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15114067#comment-15114067 ] 

Josh Rosen commented on SPARK-8725:
-----------------------------------

Here's an example of why this is a problem: I only changed files in {{hive}} and this was the resulting test command:

{code}
[info] Running Spark tests using SBT with these arguments:  -Pyarn -Phadoop-2.3 -Phive-thriftserver -Phive -Dtest.exclude.tags=org.apache.spark.tags.ExtendedYarnTest sql/test mllib/test hive-thriftserver/test hive/test examples/test catalyst/test
{code} 

Here, SQL should have been tested before MLlib.

> Test modules in topological order in dev/run-tests and python/run-tests
> -----------------------------------------------------------------------
>
>                 Key: SPARK-8725
>                 URL: https://issues.apache.org/jira/browse/SPARK-8725
>             Project: Spark
>          Issue Type: Improvement
>          Components: Build, Project Infra
>            Reporter: Josh Rosen
>            Priority: Minor
>
> Before testing a set of modules in dev/run-tests or python/run-tests, we should sort the modules in a topological ordering according to their dependencies. This will help to ensure that bugs in upstream projects are not misattributed to downstream projects because those projects' tests were the first ones to exhibit the failure.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org