You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sedona.apache.org by GitBox <gi...@apache.org> on 2020/11/03 14:17:30 UTC
[GitHub] [incubator-sedona] Imbruced opened a new pull request #487: [Sedona-3] Sedona release 3.0 python adapter [WIP]
Imbruced opened a new pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487
## Is this PR related to a proposed Issue?
It is related to existing PR [Sedona-3]
## What changes were proposed in this PR?
Package name change
## How was this patch tested?
No additional tests were required
## Did this PR include necessary documentation updates?
yes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] Imbruced commented on pull request #487: [Sedona-3] Sedona release 3.0 python adapter [WIP]
Posted by GitBox <gi...@apache.org>.
Imbruced commented on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-721870578
@jiayuasu It's ready to review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] Imbruced edited a comment on pull request #487: [Sedona-3] Sedona release 3.0 python adapter [WIP]
Posted by GitBox <gi...@apache.org>.
Imbruced edited a comment on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-720739657
@jiayuasu Only changes in documentation left to finish. Please look, if I missed something. What actually has been done:
1 Python package name change geospark -> sedona
2 All names within code which has geospark in name were changed to sedona (even variable names)
3 Example notebooks were changed accordingly
4 Adapter code between python and scala was moved to separate module, where additional packages was also included, which allowed to decrease number of lines within travis build file. That also will help Python users, one jar file is required to use Sedona core and SedonaSQL (sedona-python-adapter.jar).
5 I had to modify travis.yml file. I do not why travis changes virtual machines over time but to make it work I had to change xenial release to bionic
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] Imbruced commented on pull request #487: [Sedona-3] Sedona release 3.0 python adapter [WIP]
Posted by GitBox <gi...@apache.org>.
Imbruced commented on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-721373900
@jiayuasu Can you restart my build ? I dont have such privileges, and I think Travis build get stuck.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu merged pull request #487: [SEDONA-3] Sedona release 3.0 python adapter [WIP]
Posted by GitBox <gi...@apache.org>.
jiayuasu merged pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] Imbruced commented on pull request #487: [Sedona-3] Sedona release 3.0 python adapter [WIP]
Posted by GitBox <gi...@apache.org>.
Imbruced commented on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-720739657
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] Imbruced commented on pull request #487: [SEDONA-3] Sedona release 3.0 python adapter [WIP]
Posted by GitBox <gi...@apache.org>.
Imbruced commented on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-723544195
@jiayuasu I dicided to add python-adapter for simplicity reasons, only sedona-python-adapter is required to get all functionality. As you can see within travis script.
```yml
- find python-adapter/target/ -iregex "python-adapter\/target\/sedona-python-adapter-[0-9]\.[0-9]\.[0-9]-incubator\(-SNAPSHOT\)?\.jar" -exec cp {} $SPARK_HOME/jars \;
- (cd python;pipenv run pytest tests)
```
Assuming that user has Spark installed and properly set up Spark environment, he needs to add only one jar file. He can do that using SparkSession config argument or by putting the jar file within $SPARK_HOME/jars location.
I removed confusing upload_jars function (and also dependency, **find_spark** which can cause issues with other libraries versions). sedona-python-adapter.jar file will be included as additional files on PyPI and I think it should not be uploaded on maven.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu closed pull request #487: [Sedona-3] Sedona release 3.0 python adapter [WIP]
Posted by GitBox <gi...@apache.org>.
jiayuasu closed pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu commented on pull request #487: [SEDONA-3] Sedona release 3.0 python adapter [WIP]
Posted by GitBox <gi...@apache.org>.
jiayuasu commented on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-723653296
@Imbruced I think the reason why travis-ci took a lot of time on "job received" is that we are now under Apache organization on Travis-ci. This organization has hundreds of builds every moment so the long waiting time is expected. You can take a look at here ("average queue time"): https://travis-ci.org/github/apache?tab=insights
Let me ask Felix whether we can improve the integration speed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sedona] jiayuasu commented on pull request #487: [Sedona-3] Sedona release 3.0 python adapter [WIP]
Posted by GitBox <gi...@apache.org>.
jiayuasu commented on pull request #487:
URL: https://github.com/apache/incubator-sedona/pull/487#issuecomment-721378091
@Imbruced Unfortunately, I don't have the privilege as well. You can still push new content to your branch.
If you are done and have nothing to commit any more, I will review the PR later tonight and merge it to the 3.0 release branch to trigger the build. Then I will merge it back to the master branch only when it passes all test cases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org