You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "LucaCanali (via GitHub)" <gi...@apache.org> on 2023/09/26 09:21:19 UTC
[GitHub] [spark] LucaCanali opened a new pull request, #43123: [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing
LucaCanali opened a new pull request, #43123:
URL: https://github.com/apache/spark/pull/43123
### What changes were proposed in this pull request?
This proposes to update the Docker image used for integration tests and builds to Oracle Database 23c Free.
### Why are the changes needed?
The Docker image used for integration tests and builds currently uses Oracle XE version 21.3.0. Oracle 21 support ends in April 2024. The latest Oracle release is 23c, it is a long-term release supported till 2032. With Oracle 23c, Oracle has changed the name of the free version of its database, from Oracle XE (Express Edition) to Oracle Database Free.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing test infrastructure.
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #43123:
URL: https://github.com/apache/spark/pull/43123#issuecomment-1752118967
After waiting until the current test finishes, let me re-trigger the master branch CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #43123: [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing
URL: https://github.com/apache/spark/pull/43123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #43123:
URL: https://github.com/apache/spark/pull/43123#discussion_r1368968449
##########
.github/workflows/build_and_test.yml:
##########
@@ -938,7 +938,7 @@ jobs:
HIVE_PROFILE: hive2.3
GITHUB_PREV_SHA: ${{ github.event.before }}
SPARK_LOCAL_IP: localhost
- ORACLE_DOCKER_IMAGE_NAME: gvenzl/oracle-xe:21.3.0
+ ORACLE_DOCKER_IMAGE_NAME: gvenzl/oracle-free:23.3
Review Comment:
Should we backport this patch to branch-3.x? Otherwise, their daily tests will use `gvenzl/oracle-free:23.3`, but the test code in branch-3.x still uses `gvenzl/oracle-xe:21.3.0`.
It seems that the `OracleIntegrationSuite` in the `Docker integration tests` task of the daily tests for branch-3.x has failed recently.
- branch-3.3: https://github.com/apache/spark/actions/runs/6609791712/job/17950549755
- branch-3.4: https://github.com/apache/spark/actions/runs/6611049884/job/17954225189
- branch-3.5: https://github.com/apache/spark/actions/runs/6612344747/job/17958021656
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #43123:
URL: https://github.com/apache/spark/pull/43123#discussion_r1369486101
##########
.github/workflows/build_and_test.yml:
##########
@@ -938,7 +938,7 @@ jobs:
HIVE_PROFILE: hive2.3
GITHUB_PREV_SHA: ${{ github.event.before }}
SPARK_LOCAL_IP: localhost
- ORACLE_DOCKER_IMAGE_NAME: gvenzl/oracle-xe:21.3.0
+ ORACLE_DOCKER_IMAGE_NAME: gvenzl/oracle-free:23.3
Review Comment:
> Or can we use a solution similar to #43306?
>
> For example, for branch-3.x, explicitly add the environment variable `ORACLE_DOCKER_IMAGE_NAME` like
>
> ```
> envs: >-
> {
> "SCALA_PROFILE": "scala2.13",
> "PYTHON_TO_TEST": "",
> "ORACLE_DOCKER_IMAGE_NAME": "gvenzl/oracle-xe:21.3.0"
> }
> ```
Hmm... Are we sure that this method can't override this environment variable? If possible, this solution would be relatively simple ... @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #43123:
URL: https://github.com/apache/spark/pull/43123#discussion_r1369486101
##########
.github/workflows/build_and_test.yml:
##########
@@ -938,7 +938,7 @@ jobs:
HIVE_PROFILE: hive2.3
GITHUB_PREV_SHA: ${{ github.event.before }}
SPARK_LOCAL_IP: localhost
- ORACLE_DOCKER_IMAGE_NAME: gvenzl/oracle-xe:21.3.0
+ ORACLE_DOCKER_IMAGE_NAME: gvenzl/oracle-free:23.3
Review Comment:
> Or can we use a solution similar to #43306?
>
> For example, for branch-3.x, explicitly add the environment variable `ORACLE_DOCKER_IMAGE_NAME` like
>
> ```
> envs: >-
> {
> "SCALA_PROFILE": "scala2.13",
> "PYTHON_TO_TEST": "",
> "ORACLE_DOCKER_IMAGE_NAME": "gvenzl/oracle-xe:21.3.0"
> }
> ```
Hmm... Are we sure that this method can't override this environment variable? If possible, this solution would be relatively simple ... Can you help to confirm this? @HyukjinKwon Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing [spark]
Posted by "LucaCanali (via GitHub)" <gi...@apache.org>.
LucaCanali commented on PR #43123:
URL: https://github.com/apache/spark/pull/43123#issuecomment-1752547787
Thank you @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #43123:
URL: https://github.com/apache/spark/pull/43123#issuecomment-1752162564
It passed at the second try. So, we are good~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on code in PR #43123:
URL: https://github.com/apache/spark/pull/43123#discussion_r1368994891
##########
.github/workflows/build_and_test.yml:
##########
@@ -938,7 +938,7 @@ jobs:
HIVE_PROFILE: hive2.3
GITHUB_PREV_SHA: ${{ github.event.before }}
SPARK_LOCAL_IP: localhost
- ORACLE_DOCKER_IMAGE_NAME: gvenzl/oracle-xe:21.3.0
+ ORACLE_DOCKER_IMAGE_NAME: gvenzl/oracle-free:23.3
Review Comment:
Backporting would be better for all downstream, @LuciferYang . Feel free to backport.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #43123:
URL: https://github.com/apache/spark/pull/43123#discussion_r1368993170
##########
.github/workflows/build_and_test.yml:
##########
@@ -938,7 +938,7 @@ jobs:
HIVE_PROFILE: hive2.3
GITHUB_PREV_SHA: ${{ github.event.before }}
SPARK_LOCAL_IP: localhost
- ORACLE_DOCKER_IMAGE_NAME: gvenzl/oracle-xe:21.3.0
+ ORACLE_DOCKER_IMAGE_NAME: gvenzl/oracle-free:23.3
Review Comment:
Or can we use a solution similar to https://github.com/apache/spark/pull/43306?
For example, for branch-3.x, explicitly add the environment variable `ORACLE_DOCKER_IMAGE_NAME` like
```
envs: >-
{
"SCALA_PROFILE": "scala2.13",
"PYTHON_TO_TEST": "",
"ORACLE_DOCKER_IMAGE_NAME": "gvenzl/oracle-xe:21.3.0"
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #43123:
URL: https://github.com/apache/spark/pull/43123#discussion_r1369494922
##########
.github/workflows/build_and_test.yml:
##########
@@ -938,7 +938,7 @@ jobs:
HIVE_PROFILE: hive2.3
GITHUB_PREV_SHA: ${{ github.event.before }}
SPARK_LOCAL_IP: localhost
- ORACLE_DOCKER_IMAGE_NAME: gvenzl/oracle-xe:21.3.0
+ ORACLE_DOCKER_IMAGE_NAME: gvenzl/oracle-free:23.3
Review Comment:
> Or can we use a solution similar to #43306?
>
> For example, for branch-3.x, explicitly add the environment variable `ORACLE_DOCKER_IMAGE_NAME` like
>
> ```
> envs: >-
> {
> "SCALA_PROFILE": "scala2.13",
> "PYTHON_TO_TEST": "",
> "ORACLE_DOCKER_IMAGE_NAME": "gvenzl/oracle-xe:21.3.0"
> }
> ```
Let me see if I can test this in my local repository.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45336][TEST] Use Oracle Database 23 Free for integration and testing [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #43123:
URL: https://github.com/apache/spark/pull/43123#issuecomment-1752118564
Although this pass the PR builder, this seems to fail in `master` CI environment.
```
[info] OracleIntegrationSuite:
[info] org.apache.spark.sql.jdbc.v2.OracleIntegrationSuite *** ABORTED *** (7 minutes, 55 seconds)
[info] The code passed to eventually never returned normally. Attempted 410 times over 7.016186794716666 minutes. Last failure message: ORA-12541: Cannot connect. No listener at host 10.1.0.75 port 35471. (CONNECTION_ID=flPajzZKRBul4914CIPekw==). (DockerJDBCIntegrationSuite.scala:166)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org