You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/10/02 05:46:37 UTC
[GitHub] [spark] maropu opened a new pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
maropu opened a new pull request #29932:
URL: https://github.com/apache/spark/pull/29932
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
This PR intends to update database versions below for integration tests;
- ibmcom/db2:11.5.0.0a => ibmcom/db2:11.5.4.0 in `DB2[Krb]IntegrationSuite`
- mysql:5.7.28 => mysql:5.7.31 in `MySQLIntegrationSuite`
- postgres:12.0 => postgres:13.0 in `Postgres[Krb]IntegrationSuite`
Also, this added environmental variables so that we can test with any database version.
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
To improve tests.
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
If no, write 'No'.
-->
No.
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
Manually checked.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498668876
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ override val imageName = sys.env.getOrElse("MARIADB_DOCKER_IMAGE_NAME", "mariadb:10.4")
Review comment:
Started to have a look...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498648698
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ override val imageName = sys.env.getOrElse("MARIADB_DOCKER_IMAGE_NAME", "mariadb:10.4")
Review comment:
This test with the latest one, 10.5, failed consistently, but 10.4 looks fine on my local env.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498647708
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ override val imageName = sys.env.getOrElse("MARIADB_DOCKER_IMAGE_NAME", "mariadb:10.4")
Review comment:
This is not changed, but I failed. So, cc @gaborgsomogyi
```
MariaDBKrbIntegrationSuite:
org.apache.spark.sql.jdbc.MariaDBKrbIntegrationSuite *** ABORTED ***
Exception encountered when invoking run on a nested suite - The code passed to eventually never returned normally. Attempted 26 times over 2.269450662233333 minutes. Last failure message: Could not connect to address=(host=192.168.1.114)(port=63328)(type=master) : Socket fail to connect to host:192.168.1.114, port:63328. connect timed out. (DockerJDBCIntegrationSuite.scala:159)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702591876
**[Test build #129343 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129343/testReport)** for PR 29932 at commit [`877bc2a`](https://github.com/apache/spark/commit/877bc2a1bb9bc62a5c80949251bc1bf3bbf3ef6a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498691206
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ override val imageName = sys.env.getOrElse("MARIADB_DOCKER_IMAGE_NAME", "mariadb:10.4")
Review comment:
Ah, ok. Thanks for the check. I will re-run it on the latest one later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498778268
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,9 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ // If you change `imageName`, you need to update the version of `mariadb-plugin-gssapi-server`
+ // in `resources/mariadb_docker_entrypoint.sh` accordingly.
Review comment:
Yea, it looks nice if we can automate it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498649198
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ override val imageName = sys.env.getOrElse("MARIADB_DOCKER_IMAGE_NAME", "mariadb:10.4")
Review comment:
Could you check this, @gaborgsomogyi ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702603724
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498693900
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ override val imageName = sys.env.getOrElse("MARIADB_DOCKER_IMAGE_NAME", "mariadb:10.4")
Review comment:
I've had a look at why `10.5` is failing. Please see that the gssapi plugin version is pinned to a specific version: https://github.com/apache/spark/blob/9b88aca2954cd931c94a7cc788c3c3f7a33e99b7/external/docker-integration-tests/src/test/resources/mariadb_docker_entrypoint.sh#L21
When the version changed to `1:10.5.5+maria~focal` then it works like charm :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498724821
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ override val imageName = sys.env.getOrElse("MARIADB_DOCKER_IMAGE_NAME", "mariadb:10.4")
Review comment:
Oh, I see! Thanks a lot! I'll check and update it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498646859
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MySQLIntegrationSuite.scala
##########
@@ -27,7 +27,7 @@ import org.apache.spark.tags.DockerTest
@DockerTest
class MySQLIntegrationSuite extends DockerJDBCIntegrationSuite {
override val db = new DatabaseOnDocker {
- override val imageName = "mysql:5.7.28"
+ override val imageName = sys.env.getOrElse("MYSQL_DOCKER_IMAGE_NAME", "mysql:5.7.31")
Review comment:
This is verified.
```
MySQLIntegrationSuite:
- Basic test
- Numeric types
- Date types
- String types
- Basic write test
- query JDBC option
```
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MsSqlServerIntegrationSuite.scala
##########
@@ -27,7 +27,8 @@ import org.apache.spark.tags.DockerTest
@DockerTest
class MsSqlServerIntegrationSuite extends DockerJDBCIntegrationSuite {
override val db = new DatabaseOnDocker {
- override val imageName = "mcr.microsoft.com/mssql/server:2019-GA-ubuntu-16.04"
+ override val imageName = sys.env.getOrElse("MSSQLSERVER_DOCKER_IMAGE_NAME",
+ "mcr.microsoft.com/mssql/server:2019-GA-ubuntu-16.04")
Review comment:
This is not changed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498647349
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DB2KrbIntegrationSuite.scala
##########
@@ -35,7 +35,7 @@ class DB2KrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "db2.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "ibmcom/db2:11.5.0.0a"
+ override val imageName = sys.env.getOrElse("DB2_DOCKER_IMAGE_NAME", "ibmcom/db2:11.5.4.0")
Review comment:
This is verified.
```
DB2KrbIntegrationSuite:
Java config name: /Users/dongjoon/PRS/SPARK-REVIEW/external/docker-integration-tests/target/tmp/spark-37736442-df33-4ba9-a4e5-edb2df23aee9/1601622016820/krb5.conf
Loaded from Java config
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702684069
okay, I've checked the latest mariadb in `MariaDBKrbIntegrationSuite` worked well on my local env. Anyone can check this again? Thanks! @HyukjinKwon @dongjoon-hyun @gaborgsomogyi
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498691206
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ override val imageName = sys.env.getOrElse("MARIADB_DOCKER_IMAGE_NAME", "mariadb:10.4")
Review comment:
Ah, ok. Thanks for the check. I will re-run it on the latest one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702603724
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-703161497
Thank you, @maropu and all!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702565939
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu closed pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu closed pull request #29932:
URL: https://github.com/apache/spark/pull/29932
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702713032
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702621605
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498689492
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ override val imageName = sys.env.getOrElse("MARIADB_DOCKER_IMAGE_NAME", "mariadb:10.4")
Review comment:
I've executed the mentioned test with this PR (`mariadb:10.4`) but it works fine on my machine.
<img width="397" alt="Screenshot 2020-10-02 at 10 23 09" src="https://user-images.githubusercontent.com/18561820/94904123-1d1fd700-049b-11eb-8275-74a0f2cc3fd1.png">
What I can imagine is that the docker version is different. When I've started the development in this area I remember there was quite a stuggle to install a working version. I'm still not on the latest.
<img width="373" alt="Screenshot 2020-10-02 at 10 17 49" src="https://user-images.githubusercontent.com/18561820/94904337-7425ac00-049b-11eb-8e3e-f1f8b0b37062.png">
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702713015
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33955/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498646386
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/PostgresIntegrationSuite.scala
##########
@@ -29,7 +29,7 @@ import org.apache.spark.tags.DockerTest
@DockerTest
class PostgresIntegrationSuite extends DockerJDBCIntegrationSuite {
override val db = new DatabaseOnDocker {
- override val imageName = "postgres:12.0-alpine"
+ override val imageName = sys.env.getOrElse("POSTGRES_DOCKER_IMAGE_NAME", "postgres:13.0-alpine")
Review comment:
I verified this.
```
PostgresIntegrationSuite:
- Type mapping for various types
- Basic write test
- Creating a table with shorts and floats
- SPARK-20557: column type TIMESTAMP with TIME ZONE and TIME with TIME ZONE should be recognized
- SPARK-22291: Conversion error when transforming array types of uuid, inet and cidr to StingType in PostgreSQL
- query JDBC option
- write byte as smallint
- character type tests
- SPARK-32576: character array type tests
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702685043
**[Test build #129343 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129343/testReport)** for PR 29932 at commit [`877bc2a`](https://github.com/apache/spark/commit/877bc2a1bb9bc62a5c80949251bc1bf3bbf3ef6a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702713032
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702557659
**[Test build #129338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129338/testReport)** for PR 29932 at commit [`961f36c`](https://github.com/apache/spark/commit/961f36c8e2655559a197a59b3a3eaf2bbbea8497).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702565950
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129338/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702538831
cc: @HyukjinKwon @dongjoon-hyun
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498646479
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/PostgresKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class PostgresKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "postgres.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "postgres:12.0"
+ override val imageName = sys.env.getOrElse("POSTGRES_DOCKER_IMAGE_NAME", "postgres:13.0")
Review comment:
I verified this, too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-779505113
Oh, I forgot to do so. Thanks for letting me know, @dongjoon-hyun
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702607092
Double checked and LGTM, except I suggest to bump the MariaDB version to 10.5 w/ the suggested change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702565013
**[Test build #129338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129338/testReport)** for PR 29932 at commit [`961f36c`](https://github.com/apache/spark/commit/961f36c8e2655559a197a59b3a3eaf2bbbea8497).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702565939
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498689492
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ override val imageName = sys.env.getOrElse("MARIADB_DOCKER_IMAGE_NAME", "mariadb:10.4")
Review comment:
I've executed the mentioned test with this PR but it works fine on my machine.
<img width="397" alt="Screenshot 2020-10-02 at 10 23 09" src="https://user-images.githubusercontent.com/18561820/94904123-1d1fd700-049b-11eb-8275-74a0f2cc3fd1.png">
What I can imagine is that the docker version is different. When I've started the development in this area I remember there was quite a stuggle to install a working version. I'm still not on the latest.
<img width="373" alt="Screenshot 2020-10-02 at 10 17 49" src="https://user-images.githubusercontent.com/18561820/94904337-7425ac00-049b-11eb-8e3e-f1f8b0b37062.png">
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702690043
**[Test build #129344 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129344/testReport)** for PR 29932 at commit [`0ef813b`](https://github.com/apache/spark/commit/0ef813b4171b9942e341ec788ddb41151bdb3110).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498630046
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ override val imageName = sys.env.getOrElse("MARIADB_DOCKER_IMAGE_NAME", "mariadb:10.4")
Review comment:
Is this the latest one?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498648698
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,7 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ override val imageName = sys.env.getOrElse("MARIADB_DOCKER_IMAGE_NAME", "mariadb:10.4")
Review comment:
This test with the latest one, 10.5, failed consistently, but 10.4 worked on my local env.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702621605
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702559920
cc @gaborgsomogyi for Kerberos integration testing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702595070
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33951/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702567033
Thanks for your manual checks, @dongjoon-hyun !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702791888
**[Test build #129344 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129344/testReport)** for PR 29932 at commit [`0ef813b`](https://github.com/apache/spark/commit/0ef813b4171b9942e341ec788ddb41151bdb3110).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498771254
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MariaDBKrbIntegrationSuite.scala
##########
@@ -30,7 +30,9 @@ class MariaDBKrbIntegrationSuite extends DockerKrbJDBCIntegrationSuite {
override protected val keytabFileName = "mariadb.keytab"
override val db = new DatabaseOnDocker {
- override val imageName = "mariadb:10.4"
+ // If you change `imageName`, you need to update the version of `mariadb-plugin-gssapi-server`
+ // in `resources/mariadb_docker_entrypoint.sh` accordingly.
Review comment:
I think it's doable to automate it with an additional env var but it's a nice to have in my view.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702621198
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33954/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702686391
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702566368
Thank you for this updates, @maropu . I verified mostly. For the rest, I'll leave it to you and @gaborgsomogyi .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702705283
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33955/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702793503
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702613176
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33954/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29932:
URL: https://github.com/apache/spark/pull/29932#discussion_r498646002
##########
File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DB2IntegrationSuite.scala
##########
@@ -28,7 +28,7 @@ import org.apache.spark.tags.DockerTest
@DockerTest
class DB2IntegrationSuite extends DockerJDBCIntegrationSuite {
override val db = new DatabaseOnDocker {
- override val imageName = "ibmcom/db2:11.5.0.0a"
+ override val imageName = sys.env.getOrElse("DB2_DOCKER_IMAGE_NAME", "ibmcom/db2:11.5.4.0")
Review comment:
I verified this manually.
```
DB2IntegrationSuite:
- Basic test
- Numeric types
- Date types
- String types
- Basic write test
- query JDBC option
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702557659
**[Test build #129338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129338/testReport)** for PR 29932 at commit [`961f36c`](https://github.com/apache/spark/commit/961f36c8e2655559a197a59b3a3eaf2bbbea8497).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-779430215
Hi, @maropu .
Could you resolve the JIRA issue, SPARK-33052?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702603387
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33951/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702541181
okay, thanks for the check, @HyukjinKwon
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-703113669
Thanks, @srowen ! Merged to master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi edited a comment on pull request #29932: [SPARK-33052][SQL][TEST] Make database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi edited a comment on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702607092
Double checked and LGTM, except I suggest to bump the MariaDB version to 10.5 w/ the suggested change described [here](https://github.com/apache/spark/pull/29932#discussion_r498693900).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702793503
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702686391
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702591876
**[Test build #129343 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129343/testReport)** for PR 29932 at commit [`877bc2a`](https://github.com/apache/spark/commit/877bc2a1bb9bc62a5c80949251bc1bf3bbf3ef6a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29932: [SPARK-33052][SQL][TEST] Make all the database versions up-to-date for integration tests
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29932:
URL: https://github.com/apache/spark/pull/29932#issuecomment-702690043
**[Test build #129344 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129344/testReport)** for PR 29932 at commit [`0ef813b`](https://github.com/apache/spark/commit/0ef813b4171b9942e341ec788ddb41151bdb3110).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org