You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/10/17 02:08:02 UTC
[GitHub] [dolphinscheduler] jieguangzhou opened a new issue, #12394: [Improvement][Jupyter Task Plugin] Execute Jupyter notebook on Jupyter Lab with Idap authentication
jieguangzhou opened a new issue, #12394:
URL: https://github.com/apache/dolphinscheduler/issues/12394
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
Use papermill to execute notebook remotely on jupyter Lab， such as jupyer lab launched on kubeflow, and use ldap authentication
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] EricGao888 commented on issue #12394: [Improvement][Jupyter Task Plugin] Execute Jupyter notebook on Jupyter Lab with Idap authentication
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on issue #12394:
URL: https://github.com/apache/dolphinscheduler/issues/12394#issuecomment-1280248346
May I ask whether you want to implement this feature by yourself or have other volunteers do this? If later, I will add a `help wanted` label to this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] jieguangzhou commented on issue #12394: [Improvement][Jupyter Task Plugin] Execute Jupyter notebook on Jupyter Lab with Idap authentication
Posted by GitBox <gi...@apache.org>.
jieguangzhou commented on issue #12394:
URL: https://github.com/apache/dolphinscheduler/issues/12394#issuecomment-1281742508
This issue said if we want to run a .ipynb script via papermill on a remote server, we have to implement CustomEngine.
https://github.com/nteract/papermill/issues/361
In DolphinScheduler, can we exec in pod directly?
Such as we start a JupyterLab with custom image in kubelow which have dolphinscheduler worker? Or other connect method?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] EricGao888 commented on issue #12394: [Improvement][Jupyter Task Plugin] Execute Jupyter notebook on Jupyter Lab with Idap authentication
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on issue #12394:
URL: https://github.com/apache/dolphinscheduler/issues/12394#issuecomment-1281843801
> This issue said if we want to run a .ipynb script via papermill on a remote server, we have to implement CustomEngine. [nteract/papermill#361](https://github.com/nteract/papermill/issues/361)
>
> In DolphinScheduler, can we exec in pod directly? Such as we start a JupyterLab with custom image in kubelow which have dolphinscheduler worker? Or other connect method?
@jieguangzhou Two ways to do so:
1. If there is `Yarn` in remote cluster, you could make papermill run in yarn containers.
2. If the remote cluster is a K8S cluster, you could build papermill, jupyter and related libs into an image and run it in K8S pod.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12394: [Improvement][Jupyter Task Plugin] Execute Jupyter notebook on Jupyter Lab with Idap authentication
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #12394:
URL: https://github.com/apache/dolphinscheduler/issues/12394#issuecomment-1280173682
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] jieguangzhou commented on issue #12394: [Improvement][Jupyter Task Plugin] Execute Jupyter notebook on Jupyter Lab with Idap authentication
Posted by GitBox <gi...@apache.org>.
jieguangzhou commented on issue #12394:
URL: https://github.com/apache/dolphinscheduler/issues/12394#issuecomment-1280895315
> May I ask whether you want to implement this feature by yourself or have other volunteers do this? If latter, I will add a `help wanted` label to this issue.
I'll add the tag first, and if no one takes it, I'll implement it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org