You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/08/23 10:40:59 UTC
[GitHub] [dolphinscheduler] MonsterChenzhuo opened a new issue, #11615: [Feature][Module Name] Pedigree analysis support
MonsterChenzhuo opened a new issue, #11615:
URL: https://github.com/apache/dolphinscheduler/issues/11615
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
The current dolphinscheduler is still too complicated to configure, and often requires pointing and clicking, which is a pain in the ass.
I was wondering if we could just parse a sql into a DAG for scheduling. This becomes a task similar to flinksql, source->map->sink.
Then source corresponds to: probe dependent partition [select from table partition] whether to generate +select operation, sink corresponds to insert operation.
<img width="1135" alt="图片" src="https://user-images.githubusercontent.com/60029759/186134686-a769465e-8f71-4112-990a-d665f749adc6.png">
Production practice articles：
https://dolphinscheduler.apache.org/zh-cn/blog/How_Does_Ziru_Build_A_Job_Scheduling_System_Popular_Among_Data_Analysts.html
### Use case
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] MonsterChenzhuo commented on issue #11615: [Feature][New Module] dql: generate DAG scheduling on dolphinscheduler through a piece of sql
Posted by GitBox <gi...@apache.org>.
MonsterChenzhuo commented on issue #11615:
URL: https://github.com/apache/dolphinscheduler/issues/11615#issuecomment-1226860149
@zhongjiajie Can you open a new project for me under dolphinscheduler and give me permission to write or I can contribute the project directly to dolphinscheduler after writing it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #11615: [Feature][Module Name] Pedigree analysis support
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #11615:
URL: https://github.com/apache/dolphinscheduler/issues/11615#issuecomment-1223908889
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #11615: [Feature][New Module] dql: generate DAG scheduling on dolphinscheduler through a piece of sql
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #11615:
URL: https://github.com/apache/dolphinscheduler/issues/11615#issuecomment-1225678079
I think we prefer add this feature to our ecosystem(although is not exists currently)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #11615: [Feature][Module Name] Pedigree analysis support
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #11615:
URL: https://github.com/apache/dolphinscheduler/issues/11615#issuecomment-1223908621
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
The current dolphinscheduler is still too complicated to configure, and often requires pointing and clicking, which is a pain in the ass.
I was wondering if we could just parse a sql into a DAG for scheduling. This becomes a task similar to flinksql, source->map->sink.
Then source corresponds to: probe dependent partition [select from table partition] whether to generate +select operation, sink corresponds to insert operation.
<img width="1135" alt="图片" src="https://user-images.githubusercontent.com/60029759/186134686-a769465e-8f71-4112-990a-d665f749adc6.png">
Production practice articles：
https://dolphinscheduler.apache.org/zh-cn/blog/How_Does_Ziru_Build_A_Job_Scheduling_System_Popular_Among_Data_Analysts.html
### Use case
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] ruanwenjun commented on issue #11615: [Feature][New Module] dql: generate DAG scheduling on dolphinscheduler through a piece of sql
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #11615:
URL: https://github.com/apache/dolphinscheduler/issues/11615#issuecomment-1225604885
So this issue is about using DQL to generate a DAG containing some SQL tasks? Please change the issue title.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] MonsterChenzhuo commented on issue #11615: [Feature][New Module] using DQL to generate a DAG containing some SQL tasks
Posted by GitBox <gi...@apache.org>.
MonsterChenzhuo commented on issue #11615:
URL: https://github.com/apache/dolphinscheduler/issues/11615#issuecomment-1226861246
@ruanwenjun Yes, use the sql statement to generate the DAG scheduling task for dolphinscheduler
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org