You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by liancheng <gi...@git.apache.org> on 2015/03/03 10:16:34 UTC

[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

GitHub user liancheng opened a pull request:

    https://github.com/apache/spark/pull/4872

    [SPARK-6136] [SQL] Removed JDBC integration tests which depends on docker-client

    Integration test suites in the JDBC data source (`MySQLIntegration` and `PostgresIntegration`) depend on docker-client 2.7.5, which transitively depends on Guava 17.0. Unfortunately, Guava 17.0 is causing test runtime binary compatibility issues. Considering `MySQLIntegration` and `PostgresIntegration` are ignored right now, I'd suggest moving them from the Spark project to the [Spark integration tests] [1] project.
    
    This PR removes both the JDBC data source integration tests and the docker-client test dependency.
    
    [1]: |https://github.com/databricks/spark-integration-tests


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/liancheng/spark remove-docker-client

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/4872.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4872
    
----
commit 159b24a1e47e4fa8118e4b81049fbc7bc3406433
Author: Cheng Lian <li...@databricks.com>
Date:   2015-03-03T09:11:11Z

    Removed JDBC integration tests which depends on docker-client

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/4872#issuecomment-76914295
  
      [Test build #28230 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28230/consoleFull) for   PR 4872 at commit [`1f4169e`](https://github.com/apache/spark/commit/1f4169ef6283834c53eb6a2b31f816f7afc565eb).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/4872#issuecomment-76924369
  
      [Test build #28230 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28230/consoleFull) for   PR 4872 at commit [`1f4169e`](https://github.com/apache/spark/commit/1f4169ef6283834c53eb6a2b31f816f7afc565eb).
     * This patch **passes all tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:

    https://github.com/apache/spark/pull/4872#issuecomment-76916058
  
    Isn't it a little extreme to remove the tests? what about just excluding the Guava dep so that 14.0 is used? it may just work


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/4872#issuecomment-76911443
  
      [Test build #28229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28229/consoleFull) for   PR 4872 at commit [`159b24a`](https://github.com/apache/spark/commit/159b24a1e47e4fa8118e4b81049fbc7bc3406433).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on the pull request:

    https://github.com/apache/spark/pull/4872#issuecomment-76950011
  
    @srowen These two test suites do a bunch of time consuming jobs like setting up docker contains, pulling in MySQL/PostgreSQL, etc., and are thus ignored from the very beginning. (You may see the `@Ignore` annotations.) I've put moving them to the [Spark integration tests project] [1] on my todo list. I believe that's a better place for us to run these complex integration tests, especially because that project has already been well dockerized.
    
    [1]: https://github.com/databricks/spark-integration-tests


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

Posted by marmbrus <gi...@git.apache.org>.
Github user marmbrus commented on the pull request:

    https://github.com/apache/spark/pull/4872#issuecomment-77045250
  
    +1 to moving these to integration tests, especially if they are causing build problems.  Can you make sure there is a JIRA somewhere (where?) to make sure we don't forget though.  These tests actually did find all kinds of quirks with the various drivers.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on the pull request:

    https://github.com/apache/spark/pull/4872#issuecomment-76950210
  
    /cc @marmbrus Just to double check, is it OK to remove them?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/4872#issuecomment-76912787
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28229/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/4872#issuecomment-76912780
  
      [Test build #28229 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28229/consoleFull) for   PR 4872 at commit [`159b24a`](https://github.com/apache/spark/commit/159b24a1e47e4fa8118e4b81049fbc7bc3406433).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/4872#issuecomment-76924377
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28230/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:

    https://github.com/apache/spark/pull/4872#issuecomment-76950359
  
    @liancheng Ah I looked right past the `@Ignore`. Well, I tend to agree with your logic. You guys know better than I would.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-6136] [SQL] Removed JDBC integration te...

Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on the pull request:

    https://github.com/apache/spark/pull/4872#issuecomment-77073614
  
    @marmbrus @srowen Filed [SPARK-6147] [1] to track this. Thanks for the comments!
    
    [1]: https://issues.apache.org/jira/browse/SPARK-6147


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org