You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "jieguangzhou (via GitHub)" <gi...@apache.org> on 2023/03/27 06:23:05 UTC

[GitHub] [dolphinscheduler] jieguangzhou opened a new issue, #13794: [Feature][Task Plugin] RemoteShell Task Plugin

jieguangzhou opened a new issue, #13794:
URL: https://github.com/apache/dolphinscheduler/issues/13794

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Description
   
   A new task plugin, this allows us to run tasks on machines that do not have worker installed
   
   
   
   ### Use case
   
   ![](https://user-images.githubusercontent.com/31528124/226569941-c9a36718-ef5c-4175-a6d0-5c0dad3ab647.png)
   
   
   ![](https://user-images.githubusercontent.com/31528124/226570059-de326368-267c-4368-840b-2089d405e185.png)
   
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [dolphinscheduler] EricGao888 commented on issue #13794: [Feature][Task Plugin] RemoteShell Task Plugin

Posted by "EricGao888 (via GitHub)" <gi...@apache.org>.
EricGao888 commented on issue #13794:
URL: https://github.com/apache/dolphinscheduler/issues/13794#issuecomment-1484654482

   @jieguangzhou Great! Looking forward to this feature. 
   
   Also I have a couple of questions:
   1. How to monitor the status of tasks after DS submits them to remote machine?
   2. How to stop / kill / restart ssh tasks?
   3. How to manage resource files of ssh tasks?
   
   BTW, I see you label this feature with 4.0.0-alpha, why not 3.2.0?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [dolphinscheduler] zhongjiajie closed issue #13794: [Feature][Task Plugin] RemoteShell Task Plugin

Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie closed issue #13794: [Feature][Task Plugin] RemoteShell Task Plugin
URL: https://github.com/apache/dolphinscheduler/issues/13794


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [dolphinscheduler] jieguangzhou commented on issue #13794: [Feature][Task Plugin] RemoteShell Task Plugin

Posted by "jieguangzhou (via GitHub)" <gi...@apache.org>.
jieguangzhou commented on issue #13794:
URL: https://github.com/apache/dolphinscheduler/issues/13794#issuecomment-1484725768

   > @jieguangzhou Great! Looking forward to this feature.
   > 
   > Also I have a couple of questions:
   > 
   > 1. How to monitor the status of tasks after DS submits them to remote machine?
   > 2. How to stop / kill / restart ssh tasks?
   > 3. How to manage resource files of ssh tasks?
   > 
   > BTW, I see you label this feature with 4.0.0-alpha, why not 3.2.0?
   
    I  have already updated the label to 3.2.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #13794: [Feature][Task Plugin] RemoteShell Task Plugin

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #13794:
URL: https://github.com/apache/dolphinscheduler/issues/13794#issuecomment-1484564441

   Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
   * In order for us to understand your request as soon as possible, please provide detailed information, version or pictures.
   * If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [dolphinscheduler] jieguangzhou commented on issue #13794: [Feature][Task Plugin] RemoteShell Task Plugin

Posted by "jieguangzhou (via GitHub)" <gi...@apache.org>.
jieguangzhou commented on issue #13794:
URL: https://github.com/apache/dolphinscheduler/issues/13794#issuecomment-1484724083

   > @jieguangzhou Great! Looking forward to this feature.
   > 
   > Also I have a couple of questions:
   > 
   > 1. How to monitor the status of tasks after DS submits them to remote machine?
   > 2. How to stop / kill / restart ssh tasks?
   > 3. How to manage resource files of ssh tasks?
   > 
   > BTW, I see you label this feature with 4.0.0-alpha, why not 3.2.0?
   
   I want to use [Apache MINA SSHD](https://github.com/apache/mina-sshd) to run commands on the remote machine.
   1.  I plan to give a special identifier to the program when it runs, such as "dolphinscheduler-remoteshell". Later, to monitor the program, we just need to check if there are processes with this identifier (for example, using the command "ps -ef | grep dolphinscheduler-remoteshell-taskid"). Additionally, the program's logs will be stored in a specific path on the remote machine, and the task status will be monitored by combining the process status with the logs.
   2. For process management, we can use Mina-sshd to send the corresponding commands to the server, such as "kill -9".
   3. I haven't dealt with resource files yet because I assumed from the beginning that the program would be running on a remote server that already has all the required resources, such as environment, data, and resource files.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [dolphinscheduler] Radeity commented on issue #13794: [Feature][Task Plugin] RemoteShell Task Plugin

Posted by "Radeity (via GitHub)" <gi...@apache.org>.
Radeity commented on issue #13794:
URL: https://github.com/apache/dolphinscheduler/issues/13794#issuecomment-1486332629

   > 3. I haven't dealt with resource files yet because I assumed from the beginning that the program would be running on a remote server that already has all the required resources, such as environment, data, and resource files.
   
   It seems like we can also use [sshd-scp](https://github.com/apache/mina-sshd/blob/master/docs/scp.md) provided by [Apache MINA SSHD](https://github.com/apache/mina-sshd) to send resources files managed by DS to remote machine.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [dolphinscheduler] jieguangzhou commented on issue #13794: [Feature][Task Plugin] RemoteShell Task Plugin

Posted by "jieguangzhou (via GitHub)" <gi...@apache.org>.
jieguangzhou commented on issue #13794:
URL: https://github.com/apache/dolphinscheduler/issues/13794#issuecomment-1484573126

   It's similar to this, but here it runs as a component https://github.com/apache/dolphinscheduler/pull/12736


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org