You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by srowen <gi...@git.apache.org> on 2014/08/07 10:46:21 UTC

[GitHub] spark pull request: SPARK-2903 [BUILD] [SQL] Spark SQL tests fail ...

GitHub user srowen opened a pull request:

    https://github.com/apache/spark/pull/1833

    SPARK-2903 [BUILD] [SQL] Spark SQL tests fail to compile due to dependency structure, misplaced test class

    Since recently, I find that the SQL modules' test code fails to compile with a simple `mvn clean compile test-compile`. `sql/core` fails with a large number of errors, beginning like so:
    
    ```
    [error] /Users/srowen/Documents/spark/sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala:23: not found: type PlanTest
    [error] class QueryTest extends PlanTest {
    [error]                         ^
    [error] /Users/srowen/Documents/spark/sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala:28: package org.apache.spark.sql.test is not a value
    [error]   test("SPARK-1669: cacheTable should be idempotent") {
    ...
    ```
    
    I think a couple files and dependencies are in the wrong place, and it is resolved with the following:
    
    - Move `org.apache.spark.sql.test.TestSQLContext` from under `sql/core/src/main/scala`, to `sql/core/src/test/scala` , which seems like where it belongs as test code
    - Move `org.apache.spark.sql.parquet.ParquetTestData` likewise from main to test
    - QueryTest in sql/core depends on `PlanTest`, which appears to be catalyst-specific, instead of `FunSuite`. It shouldn't, and isn't needed.
    - Remove dependency from `sql/core` tests on `sql/catalyst` tests, which seems reversed
    - Introduce dependency from `sql/hive` tests on `sql/core` tests to restore access to the files moved above
    
    What I'm still not clear on is why it has only started failing to compile in the last week or so. But, these seem like the right changes to make. It compiles and tests pass.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/srowen/spark SPARK-2903

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/1833.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1833
    
----
commit 96a47268eddd5d03b22cb20af71c510b430854aa
Author: Sean Owen <sr...@gmail.com>
Date:   2014-08-07T08:36:10Z

    - Move org.apache.spark.sql.test.TestSQLContext from under sql/core/src/*main*/scala, to sql/core/src/*test*/scala , which seems like where it belongs as test code
    - Move org.apache.spark.sql.parquet.ParquetTestData likewise from main to test
    - QueryTest in sql/core depends on FunSuite instead of PlanTest, which appears to be catalyst-specific and not needed
    - Remove dependency from sql/core tests on sql/catalyst tests, which seems reversed
    - Introduce dependency from sql/hive tests on sql/core tests to restore access to the files moved above.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org