You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sedona.apache.org by GitBox <gi...@apache.org> on 2021/11/11 14:06:18 UTC
[GitHub] [incubator-sedona] Kimahriman opened a new pull request #562: Add support for Scala 2.13 build
Kimahriman opened a new pull request #562:
URL: https://github.com/apache/incubator-sedona/pull/562
## Is this PR related to a proposed Issue?
https://issues.apache.org/jira/browse/SEDONA-73
## What changes were proposed in this PR?
Adds a Scala 2.13 Maven profile and updates the code base to support 2.13 in addition to existing versions. A new dependency was added that provides backward compatibility for various Java-Scala conversion utilities. I enabled the Java tests to also run with 2.13.
It will be a little trickier to figure out how to run the Python or R tests with Scala 2.13, because I think both just download official releases to do it. It looks like there are official releases for Scala 2.13, but I am not trying to tackle that in this PR. The Python tests would likely have to be updated to download the official release versus installing from Pypi, because that only comes with the default Scala version. This is mostly for experimental purposes, and for making any new additions be Scala 2.13 compatible to make the release in the future easier.
I also did a little refactoring on the raster functions that I missed last time. They were double eval'ing everything to check the type. I updated to just use the common parent type instead to convert to a double array.
## How was this patch tested?
Existing tests, added 2.13 build to java GitHub workflow tests.
## Did this PR include necessary documentation updates?
Internal update right now, can update documentation when creating official 2.13 release support.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] Kimahriman commented on pull request #562: [SEDONA-73] Add support for Scala 2.13 build
Posted by GitBox <gi...@apache.org>.
Kimahriman commented on pull request #562:
URL: https://github.com/apache/incubator-sedona/pull/562#issuecomment-1030490328
Good catch, made #577
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] Kimahriman commented on a change in pull request #562: [SEDONA-73] Add support for Scala 2.13 build
Posted by GitBox <gi...@apache.org>.
Kimahriman commented on a change in pull request #562:
URL: https://github.com/apache/incubator-sedona/pull/562#discussion_r748345778
##########
File path: .github/workflows/java.yml
##########
@@ -23,6 +23,9 @@ jobs:
scala: 2.11.8
- spark: 3.0.3
scala: 2.11.8
+ include:
Review comment:
Doesn't
```
run: if [ ${SPARK_VERSION:0:1} == "3" ]; then mvn -q clean install -Dscala=${SCALA_VERSION:0:4} -Dspark=3.0 ; else mvn -q clean install -Dscala=${SCALA_VERSION:0:4} -Dspark=2.4 ; fi
```
already handle that? It looks like that test did run with scala 2.13 in the actions (I see deprecation warnings). Also, since the patch version is ignored in the tests should the scala version just be set to the minor version in the matrix?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu merged pull request #562: [SEDONA-73] Add support for Scala 2.13 build
Posted by GitBox <gi...@apache.org>.
jiayuasu merged pull request #562:
URL: https://github.com/apache/incubator-sedona/pull/562
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu commented on pull request #562: [SEDONA-73] Add support for Scala 2.13 build
Posted by GitBox <gi...@apache.org>.
jiayuasu commented on pull request #562:
URL: https://github.com/apache/incubator-sedona/pull/562#issuecomment-1030229216
@Kimahriman Hi Adam, I received reports from several users.
This PR introduces a dependency [Scala Collection Compat](https://mvnrepository.com/artifact/org.scala-lang.modules/scala-collection-compat) in the main POM.xml.
The entire [scala-library](https://mvnrepository.com/artifact/org.scala-lang/scala-library) is the compile dependency of Scala Collection Compat. As a result, Scala library is packaged in all Sedona releases.
This leads to (1) big Sedona jars (2) potential version conflict. Can we exclude Scala library from Scala Collection Compat?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu commented on a change in pull request #562: [SEDONA-73] Add support for Scala 2.13 build
Posted by GitBox <gi...@apache.org>.
jiayuasu commented on a change in pull request #562:
URL: https://github.com/apache/incubator-sedona/pull/562#discussion_r748631542
##########
File path: .github/workflows/java.yml
##########
@@ -23,6 +23,9 @@ jobs:
scala: 2.11.8
- spark: 3.0.3
scala: 2.11.8
+ include:
Review comment:
Sorry, my bad. Forgot this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu commented on a change in pull request #562: Add support for Scala 2.13 build
Posted by GitBox <gi...@apache.org>.
jiayuasu commented on a change in pull request #562:
URL: https://github.com/apache/incubator-sedona/pull/562#discussion_r747998469
##########
File path: .github/workflows/java.yml
##########
@@ -23,6 +23,9 @@ jobs:
scala: 2.11.8
- spark: 3.0.3
scala: 2.11.8
+ include:
Review comment:
You need to enable Scala 2.13 in this yml when compile Sedona so the Scala/Java test can actually run. There are some examples in the yml file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org