You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "efaracci018 (via GitHub)" <gi...@apache.org> on 2025/05/22 19:59:29 UTC
[PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
efaracci018 opened a new pull request, #50989:
URL: https://github.com/apache/spark/pull/50989
### What changes were proposed in this pull request?
Fix the version parsing logic in `HiveExternalCatalogVersionsSuite` to properly handle new artifact paths in https://dist.apache.org/repos/dist/release/spark/ so that "backward compatibility" test can be run.
This change creates a constant `val SparkVersionPattern = """<a href="spark-(\d.\d.\d)/">""".r` for more precise version matching, and removes redundant `.filterNot(_.contains("preview"))` which is no longer needed.
### Why are the changes needed?
The suite is failing to execute the "backward compatibility" test due to parsing errors with testing versions. The current implementation fails to parse versions when encountering new paths like `spark-connect-swift-0.1.0/` and `spark-kubernetes-operator-0.1.0/ in https://dist.apache.org/repos/dist/release/spark/. This leads to `PROCESS_TABLES.testingVersions` being empty, and in turn a logError: "Exception encountered when invoking run on a nested suite - Fail to get the latest Spark versions to test". As a result, the condition is not met to run the test.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Executed local build and test for `HiveExternalCatalogVersionsSuite`:
`build/mvn -pl sql/hive-Dtest=none -DwildcardSuites=org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite test-compile scalatest:test`
Verified that the reported error no longer appears, "backward compatibility" test runs successfully, and `PROCESS_TABLES.testingVersions` now correctly contains "3.5.5" when printed out, which was previously empty.
### Was this patch authored or co-authored using generative AI tooling?
No.
@cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "efaracci018 (via GitHub)" <gi...@apache.org>.
efaracci018 commented on code in PR #50989:
URL: https://github.com/apache/spark/pull/50989#discussion_r2127336045
##########
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala:
##########
@@ -96,7 +96,10 @@ class HiveExternalCatalogVersionsSuite extends SparkSubmitTestUtils {
mirrors.distinct :+ "https://archive.apache.org/dist" :+ PROCESS_TABLES.releaseMirror
logInfo(s"Trying to download Spark $version from $sites")
for (site <- sites) {
- val filename = s"spark-$version-bin-hadoop3-scala2.13.tgz"
+ val filename = version match {
Review Comment:
Thank you, I've put up a revision with this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #50989:
URL: https://github.com/apache/spark/pull/50989#issuecomment-2945435201
Once the tests pass, I will merge this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #50989:
URL: https://github.com/apache/spark/pull/50989#discussion_r2127844186
##########
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala:
##########
@@ -96,7 +96,12 @@ class HiveExternalCatalogVersionsSuite extends SparkSubmitTestUtils {
mirrors.distinct :+ "https://archive.apache.org/dist" :+ PROCESS_TABLES.releaseMirror
logInfo(s"Trying to download Spark $version from $sites")
for (site <- sites) {
- val filename = s"spark-$version-bin-hadoop3-scala2.13.tgz"
+ val scala_version = version match {
Review Comment:
```suggestion
val scalaVersion = version match {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #50989:
URL: https://github.com/apache/spark/pull/50989#issuecomment-2903266768
It seems that new issues may still arise after the 4.0 release with this pr:
```
build/sbt "hive/testOnly org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite" -Phive
```
```
[info] HiveExternalCatalogVersionsSuite:
12:55:31.252 WARN org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite: Failed to download Spark 4.0.0 from https://dlcdn.apache.org//spark/spark-4.0.0/spark-4.0.0-bin-hadoop3-scala2.13.tgz: https://dlcdn.apache.org//spark/spark-4.0.0/spark-4.0.0-bin-hadoop3-scala2.13.tgz
12:55:32.485 WARN org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite: Failed to download Spark 4.0.0 from https://archive.apache.org/dist/spark/spark-4.0.0/spark-4.0.0-bin-hadoop3-scala2.13.tgz: https://archive.apache.org/dist/spark/spark-4.0.0/spark-4.0.0-bin-hadoop3-scala2.13.tgz
12:55:33.414 WARN org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite: Failed to download Spark 4.0.0 from https://dist.apache.org/repos/dist/release/spark/spark-4.0.0/spark-4.0.0-bin-hadoop3-scala2.13.tgz: https://dist.apache.org/repos/dist/release/spark/spark-4.0.0/spark-4.0.0-bin-hadoop3-scala2.13.tgz
12:55:33.597 WARN org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite:
===== POSSIBLE THREAD LEAK IN SUITE o.a.s.sql.hive.HiveExternalCatalogVersionsSuite, threads: Keep-Alive-Timer (daemon=true) =====
[info] org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite *** ABORTED *** (3 minutes, 41 seconds)
[info] Unable to download Spark 4.0.0 (HiveExternalCatalogVersionsSuite.scala:125)
[info] org.scalatest.exceptions.TestFailedException:
[info] at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
[info] at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
[info] at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
[info] at org.scalatest.Assertions.fail(Assertions.scala:933)
[info] at org.scalatest.Assertions.fail$(Assertions.scala:929)
[info] at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
[info] at org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite.tryDownloadSpark(HiveExternalCatalogVersionsSuite.scala:125)
[info] at org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite.$anonfun$beforeAll$4(HiveExternalCatalogVersionsSuite.scala:212)
[info] at org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite.$anonfun$beforeAll$4$adapted(HiveExternalCatalogVersionsSuite.scala:209)
[info] at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
[info] at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
[info] at scala.collection.AbstractIterable.foreach(Iterable.scala:935)
[info] at org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite.beforeAll(HiveExternalCatalogVersionsSuite.scala:209)
[info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
[info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
[info] at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
[info] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[info] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[info] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[info] at java.base/java.lang.Thread.run(Thread.java:840)
[info] Run completed in 3 minutes, 42 seconds.
[info] Total number of tests run: 0
[info] Suites: completed 0, aborted 1
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] *** 1 SUITE ABORTED ***
[error] Error during tests:
[error] org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite
[error] (hive / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "vrozov (via GitHub)" <gi...@apache.org>.
vrozov commented on code in PR #50989:
URL: https://github.com/apache/spark/pull/50989#discussion_r2127229573
##########
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala:
##########
@@ -96,7 +96,10 @@ class HiveExternalCatalogVersionsSuite extends SparkSubmitTestUtils {
mirrors.distinct :+ "https://archive.apache.org/dist" :+ PROCESS_TABLES.releaseMirror
logInfo(s"Trying to download Spark $version from $sites")
for (site <- sites) {
- val filename = s"spark-$version-bin-hadoop3-scala2.13.tgz"
+ val filename = version match {
+ case v if v.startsWith("3") => s"spark-$version-bin-hadoop3-scala2.13.tgz"
+ case _ => s"spark-$version-bin-hadoop3.tgz"
Review Comment:
consider using "4." for the version match and fail the test for unknown/not expected version.
##########
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala:
##########
@@ -96,7 +96,10 @@ class HiveExternalCatalogVersionsSuite extends SparkSubmitTestUtils {
mirrors.distinct :+ "https://archive.apache.org/dist" :+ PROCESS_TABLES.releaseMirror
logInfo(s"Trying to download Spark $version from $sites")
for (site <- sites) {
- val filename = s"spark-$version-bin-hadoop3-scala2.13.tgz"
+ val filename = version match {
+ case v if v.startsWith("3") => s"spark-$version-bin-hadoop3-scala2.13.tgz"
Review Comment:
nit: use "3." for the version match
##########
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala:
##########
@@ -96,7 +96,10 @@ class HiveExternalCatalogVersionsSuite extends SparkSubmitTestUtils {
mirrors.distinct :+ "https://archive.apache.org/dist" :+ PROCESS_TABLES.releaseMirror
logInfo(s"Trying to download Spark $version from $sites")
for (site <- sites) {
- val filename = s"spark-$version-bin-hadoop3-scala2.13.tgz"
+ val filename = version match {
Review Comment:
consider
```
val filename = s"spark-$version-bin-hadoop3$scala_version.tgz"
```
and define
```
val scala_version = ...
```
using version match logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "vrozov (via GitHub)" <gi...@apache.org>.
vrozov commented on PR #50989:
URL: https://github.com/apache/spark/pull/50989#issuecomment-2941496359
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "efaracci018 (via GitHub)" <gi...@apache.org>.
efaracci018 commented on PR #50989:
URL: https://github.com/apache/spark/pull/50989#issuecomment-2949806985
Thanks @LuciferYang, my Jira account is efaracci018.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "efaracci018 (via GitHub)" <gi...@apache.org>.
efaracci018 commented on PR #50989:
URL: https://github.com/apache/spark/pull/50989#issuecomment-2914443877
Thank you @LuciferYang for pointing out this recent issue, I am interesting in taking a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #50989:
URL: https://github.com/apache/spark/pull/50989#issuecomment-2949832024
Merged into master and branch-4.0. Thanks @efaracci018 and welcome to the Apache Spark community.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "efaracci018 (via GitHub)" <gi...@apache.org>.
efaracci018 commented on code in PR #50989:
URL: https://github.com/apache/spark/pull/50989#discussion_r2129526168
##########
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala:
##########
@@ -96,7 +96,12 @@ class HiveExternalCatalogVersionsSuite extends SparkSubmitTestUtils {
mirrors.distinct :+ "https://archive.apache.org/dist" :+ PROCESS_TABLES.releaseMirror
logInfo(s"Trying to download Spark $version from $sites")
for (site <- sites) {
- val filename = s"spark-$version-bin-hadoop3-scala2.13.tgz"
+ val scala_version = version match {
Review Comment:
Sounds good, I've updated `scalaVersion`, and `sparkVersionPattern` as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #50989:
URL: https://github.com/apache/spark/pull/50989#discussion_r2127844911
##########
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala:
##########
@@ -96,7 +96,12 @@ class HiveExternalCatalogVersionsSuite extends SparkSubmitTestUtils {
mirrors.distinct :+ "https://archive.apache.org/dist" :+ PROCESS_TABLES.releaseMirror
logInfo(s"Trying to download Spark $version from $sites")
for (site <- sites) {
- val filename = s"spark-$version-bin-hadoop3-scala2.13.tgz"
+ val scala_version = version match {
Review Comment:
Variables should be named using camelCase in Spark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #50989:
URL: https://github.com/apache/spark/pull/50989#issuecomment-2947951904
@efaracci018 Could you please provide your Jira account? I need to add your Jira account to the Spark Contributor group.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "efaracci018 (via GitHub)" <gi...@apache.org>.
efaracci018 commented on PR #50989:
URL: https://github.com/apache/spark/pull/50989#issuecomment-2902742472
@cloud-fan please review, thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang closed pull request #50989: [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite
URL: https://github.com/apache/spark/pull/50989
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-52265][SQL][TEST] Fix regex leading to empty PROCESS_TABLES.testingVersions in HiveExternalCatalogVersionsSuite [spark]
Posted by "efaracci018 (via GitHub)" <gi...@apache.org>.
efaracci018 commented on PR #50989:
URL: https://github.com/apache/spark/pull/50989#issuecomment-2940879859
Requiring identifier 'scala-2.13' was introduced by [this following PR](https://github.com/apache/spark/pull/45124), with a comment that noted
> We can change it back to *-hadoop3.tgz at Apache Spark 4.1 after the official Apache Spark 4.0 release.
Given official 4.0 release is out, we can drop the 'scala-2.13' identifier for 4.0, but need to maintain it for any 3.x version.
This fix allows the test to run, and I verified that scala 2.13 is used:
```
~/w/O/spark ❯❯❯ ls -al /tmp/test-spark/spark-3.5.6/jars/scala*
-rw-r--r--@ 1 efaracci wheel 5596 May 22 23:20 /tmp/test-spark/spark-3.5.6/jars/scala-collection-compat_2.13-2.7.0.jar
-rw-r--r--@ 1 efaracci wheel 12097183 May 22 23:20 /tmp/test-spark/spark-3.5.6/jars/scala-compiler-2.13.8.jar
-rw-r--r--@ 1 efaracci wheel 6003601 May 22 23:20 /tmp/test-spark/spark-3.5.6/jars/scala-library-2.13.8.jar
-rw-r--r--@ 1 efaracci wheel 1127123 May 22 23:20 /tmp/test-spark/spark-3.5.6/jars/scala-parallel-collections_2.13-1.0.4.jar
-rw-r--r--@ 1 efaracci wheel 189332 May 22 23:20 /tmp/test-spark/spark-3.5.6/jars/scala-parser-combinators_2.13-2.3.0.jar
-rw-r--r--@ 1 efaracci wheel 3772083 May 22 23:20 /tmp/test-spark/spark-3.5.6/jars/scala-reflect-2.13.8.jar
-rw-r--r--@ 1 efaracci wheel 483090 May 22 23:20 /tmp/test-spark/spark-3.5.6/jars/scala-xml_2.13-2.1.0.jar
~/w/O/spark ❯❯❯ ls -al /tmp/test-spark/spark-4.0.0/jars/scala*
-rw-r--r--@ 1 efaracci wheel 5596 May 19 01:26 /tmp/test-spark/spark-4.0.0/jars/scala-collection-compat_2.13-2.7.0.jar
-rw-r--r--@ 1 efaracci wheel 12308669 May 19 01:26 /tmp/test-spark/spark-4.0.0/jars/scala-compiler-2.13.16.jar
-rw-r--r--@ 1 efaracci wheel 5926340 May 19 01:26 /tmp/test-spark/spark-4.0.0/jars/scala-library-2.13.16.jar
-rw-r--r--@ 1 efaracci wheel 1120803 May 19 01:26 /tmp/test-spark/spark-4.0.0/jars/scala-parallel-collections_2.13-1.2.0.jar
-rw-r--r--@ 1 efaracci wheel 189196 May 19 01:26 /tmp/test-spark/spark-4.0.0/jars/scala-parser-combinators_2.13-2.4.0.jar
-rw-r--r--@ 1 efaracci wheel 3801119 May 19 01:26 /tmp/test-spark/spark-4.0.0/jars/scala-reflect-2.13.16.jar
-rw-r--r--@ 1 efaracci wheel 546013 May 19 01:26 /tmp/test-spark/spark-4.0.0/jars/scala-xml_2.13-2.3.0.jar
```
Please note the following constraints:
1. We can remove match expression once 3.x is no longer being tested
2. This will likely have to be changed again when Spark moves to scala 3.x
@LuciferYang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org