You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/05/15 22:04:18 UTC
[GitHub] [incubator-hudi] vingov opened a new pull request #1632: Added python3 to the spark_base docker image to support pyspark
vingov opened a new pull request #1632:
URL: https://github.com/apache/incubator-hudi/pull/1632
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a pull request.*
## What is the purpose of the pull request
This pull-request adds python3 to the apachehudi/sparkbase docker image, which is required to run pyspark shell command. This is part of the effort to add official python-support to hudi package.
We can run the following pyspark command on sparkmaster, sparkworker, sparkadhoc docker images after merging this pull request.
## Brief change log
- *Added python-support to the hudi demo docker images to enable pyspark shell*
## Verify this pull request
This change added tests and can be verified as follows:
- *Manually verified the change by building the docker image locally, registering with the local Docker registry, using the local docker image in the docker-compose file, started all the docker images and tested the change by invoking pyspark from the adhoc-1 machine via command-line.*
The details of the testing can be found in this gist: https://gist.github.com/vingov/347548f2df9af11892c8331123b42967
## Committer checklist
- [x] Has a corresponding JIRA in PR title & commit
- [x] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] vingov commented on pull request #1632: [HUDI-783] Added python3 to the spark_base docker image to support pyspark
Posted by GitBox <gi...@apache.org>.
vingov commented on pull request #1632:
URL: https://github.com/apache/incubator-hudi/pull/1632#issuecomment-629521740
@bhasudha - As we discussed, I've followed the steps to test the docker images using local registry, check out the detailed testing report [here](https://gist.github.com/vingov/347548f2df9af11892c8331123b42967)
Can you please review and merge this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org