You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sedona.apache.org by GitBox <gi...@apache.org> on 2021/01/18 01:24:21 UTC

[GitHub] [incubator-sedona] jiayuasu opened a new pull request #505: Change the project structure according to the voting result

jiayuasu opened a new pull request #505:
URL: https://github.com/apache/incubator-sedona/pull/505


   ## Is this PR related to a proposed Issue?
   
   1. Change the artifact name to 1.0.0-incubating
   2. Generate sha512 checksum
   3. Publish Parent pom for Spark 3.0+Scala 2.12, not publish Parent pom for other combinations
   4. Remove Geotools binary from Python Adapter binary (TO-DO)
   
   ## What changes were proposed in this PR?
   
   ## How was this patch tested?
   
   ## Did this PR include necessary documentation updates?
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-sedona] jiayuasu merged pull request #505: Change the project structure according to the voting result

Posted by GitBox <gi...@apache.org>.
jiayuasu merged pull request #505:
URL: https://github.com/apache/incubator-sedona/pull/505


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-sedona] jiayuasu commented on pull request #505: Change the project structure according to the voting result

Posted by GitBox <gi...@apache.org>.
jiayuasu commented on pull request #505:
URL: https://github.com/apache/incubator-sedona/pull/505#issuecomment-761983945


   @Imbruced Hi Pawel, I am changing the project again according to ASF requirement and the comments from Von Gosling and Felix (in our dev list).
   
   One of the changes I want to make is that completely remove GeoTools binaries from Sedona Python-adapter binary because they are under LGPL, an ASF Category X license. Otherwise, I am afraid this will be another blocker for the release and will cause future API change to the end user.
   
   However, after I added Geotools jars to Python Testbase.py and even download GeoTools jars to SPARK_HOME/jars, python tests still show ClassNotFound exception. Could you please help me with that?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-sedona] jiayuasu commented on pull request #505: Change the project structure according to the voting result

Posted by GitBox <gi...@apache.org>.
jiayuasu commented on pull request #505:
URL: https://github.com/apache/incubator-sedona/pull/505#issuecomment-762568062


   @Imbruced I also tried spark.jars.excludes but it didn't work. My plan is (see my latest commit)
   
   1. Do not package any GeoTools jars by default. The published Python-adapter jar (on Maven, GitHub and ASF Dist) will use "provided" scope for geotools.
   2. If the end users wants to use CRS transformation and ShapefileReader, they have to compile the source code by themselves as follows and copy the jar to SPARK_HOME/jars/:
   ```
   mvn clean install -DskipTests -Dgeotools
   ```
   
   I think this is the only option we have right now. What do you think?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-sedona] Imbruced commented on pull request #505: Change the project structure according to the voting result

Posted by GitBox <gi...@apache.org>.
Imbruced commented on pull request #505:
URL: https://github.com/apache/incubator-sedona/pull/505#issuecomment-762455631


   ```xml
   <exclusion>
       <groupId>org.locationtech.jts</groupId>
       <artifactId>jts-core</artifactId>
   </exclusion>
   <exclusion>
       <groupId>com.fasterxml.jackson.core</groupId>
       <artifactId>*</artifactId>
   </exclusion>
   ```
   
   Exclusions cause the issue, I am not sure if that way is manageable from Python side. Tried to use exclusions **spark.jars.excludes** without success. Maybe we should keep this module as separate ? Or make all external dependencies as provided and share the code with all dependencies packed ? WDYT ? 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org