You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sedona.apache.org by "sshiv012 (via GitHub)" <gi...@apache.org> on 2025/02/24 08:48:56 UTC
[PR] [SEDONA-715] Add apache zeppelin notebook support for the docker image [sedona]
sshiv012 opened a new pull request, #1826:
URL: https://github.com/apache/sedona/pull/1826
Did you read the Contributor Guide?
Yes, I have read Contributor Rules and Contributor Development Guide
Is this PR related to a JIRA ticket?
Yes, the URL of the associated JIRA ticket is https://issues.apache.org/jira/browse/SEDONA-715. The PR name follows the format [SEDONA-XXX] my subject.
What changes were proposed in this PR?
Added Zeppelin notebook support for the sedona docker image along with visualization helium plugin
How was this patch tested?
Tested locally by building the image and running it. Visualized with the sample notebook that has been bundled with the commit.
Did this PR include necessary documentation updates? Yes, added additional information that may help the users.
## Did you read the Contributor Guide?
- Yes, I have read the [Contributor Rules](https://sedona.apache.org/latest-snapshot/community/rule/) and [Contributor Development Guide](https://sedona.apache.org/latest-snapshot/community/develop/)
- No, I haven't read it.
## Is this PR related to a JIRA ticket?
- Yes, the URL of the associated JIRA ticket is https://issues.apache.org/jira/browse/SEDONA-XXX. The PR name follows the format `[SEDONA-XXX] my subject`.
- No:
- this is a documentation update. The PR name follows the format `[DOCS] my subject`
- this is a CI update. The PR name follows the format `[CI] my subject`
## What changes were proposed in this PR?
## How was this patch tested?
## Did this PR include necessary documentation updates?
- Yes, I am adding a new API. I am using the [current SNAPSHOT version number](https://github.com/apache/sedona/blob/99239524f17389fc4ae9548ea88756f8ea538bb9/pom.xml#L29) in `vX.Y.Z` format.
- Yes, I have updated the documentation.
- No, this PR does not affect any public API so no need to change the documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SEDONA-715] Add apache zeppelin notebook support for the docker image [sedona]
Posted by "sshiv012 (via GitHub)" <gi...@apache.org>.
sshiv012 commented on PR #1826:
URL: https://github.com/apache/sedona/pull/1826#issuecomment-2705722652
Hello @jiayuasu, ahh there is a version mismatch in the zeppelin configuration for spark's sedona dependency. I have added the below command (finds the jar file name from the directory and replaces it in the config json file) to ensure this config is over written during build.
# Extract version from the actual JAR file and update interpreter.json
RUN jar_name=$(find "${SPARK_HOME}/jars/" -name "sedona-spark-shaded-*.jar" | head -n 1) && \
sed -i "s|\(\"groupArtifactVersion\": \).*|\1\"${jar_name}\",|" "${ZEPPELIN_HOME}/conf/interpreter.json"
Can I raise a separate pull request? Also should I push it to docker hub?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SEDONA-715] Add apache zeppelin notebook support for the docker image [sedona]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu merged PR #1826:
URL: https://github.com/apache/sedona/pull/1826
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SEDONA-715] Add apache zeppelin notebook support for the docker image [sedona]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu commented on PR #1826:
URL: https://github.com/apache/sedona/pull/1826#issuecomment-2691220246
@sshiv012 Your last commit seems to break the CI. Please fix it. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SEDONA-715] Add apache zeppelin notebook support for the docker image [sedona]
Posted by "sshiv012 (via GitHub)" <gi...@apache.org>.
sshiv012 commented on PR #1826:
URL: https://github.com/apache/sedona/pull/1826#issuecomment-2691367373
If the files already have the +x flag set, git update-index --chmod=+x does nothing and git thinks there's nothing to commit, even though the flag isn't being saved into the repo. Had to remove the flag, run the git command, then put the flag back. Also set git config core.filemode to false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SEDONA-715] Add apache zeppelin notebook support for the docker image [sedona]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu commented on PR #1826:
URL: https://github.com/apache/sedona/pull/1826#issuecomment-2686452135
@sshiv012 please run `pre-commit run --all-files` locally to fix lint issues
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SEDONA-715] Add apache zeppelin notebook support for the docker image [sedona]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu commented on code in PR #1826:
URL: https://github.com/apache/sedona/pull/1826#discussion_r1968058005
##########
docs/setup/docker.md:
##########
@@ -144,6 +144,23 @@ This docker image can only be built against Sedona 1.7.0+ and Spark 3.3+
* Spark job UI: http://localhost:4040
* Spark master web UI: http://localhost:8080/
* Spark work web UI: http://localhost:8081/
+* Apache Zeppelin: http://localhost:8085/
+
+### Zeppelin Configuration
+
+1. Access Zeppelin at [http://localhost:8085/classic/#/helium](http://localhost:8085/classic/#/helium).
+2. Enable the `sedona-zeppelin` helium plugin.
+3. Navigate to [http://localhost:8085/classic/#/interpreter](http://localhost:8085/classic/#/interpreter), search for `spark`, and click `edit`.
Review Comment:
@sshiv012 This is awesome! Is there a way to enable all these by default so the user does not need to config himself?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SEDONA-715] Add apache zeppelin notebook support for the docker image [sedona]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu commented on PR #1826:
URL: https://github.com/apache/sedona/pull/1826#issuecomment-2705097047
@sshiv012 I've tried the Zeppelin notebook. It seems to have some issues and cannot run.
The docker images have been pushed to Sedona docker hub as `latest` tag https://hub.docker.com/r/apache/sedona/tags. Can you take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SEDONA-715] Add apache zeppelin notebook support for the docker image [sedona]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu commented on PR #1826:
URL: https://github.com/apache/sedona/pull/1826#issuecomment-2709367510
@sshiv012 please raise a separate PR to fix this. We will auto push it to DockerHub once the PR is merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SEDONA-715] Add apache zeppelin notebook support for the docker image [sedona]
Posted by "sshiv012 (via GitHub)" <gi...@apache.org>.
sshiv012 commented on code in PR #1826:
URL: https://github.com/apache/sedona/pull/1826#discussion_r1972413234
##########
docs/setup/docker.md:
##########
@@ -144,6 +144,23 @@ This docker image can only be built against Sedona 1.7.0+ and Spark 3.3+
* Spark job UI: http://localhost:4040
* Spark master web UI: http://localhost:8080/
* Spark work web UI: http://localhost:8081/
+* Apache Zeppelin: http://localhost:8085/
+
+### Zeppelin Configuration
+
+1. Access Zeppelin at [http://localhost:8085/classic/#/helium](http://localhost:8085/classic/#/helium).
+2. Enable the `sedona-zeppelin` helium plugin.
+3. Navigate to [http://localhost:8085/classic/#/interpreter](http://localhost:8085/classic/#/interpreter), search for `spark`, and click `edit`.
Review Comment:
Yes, added the config files in the second commit. The documentation has been revised as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org