You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/08/01 03:05:04 UTC
[GitHub] [dolphinscheduler] caishunfeng opened a new issue, #11223: [Feature][Stream] Support stream task
caishunfeng opened a new issue, #11223:
URL: https://github.com/apache/dolphinscheduler/issues/11223
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
This feature issue is about supporting stream task:
- stream task definition

- stream task instance

- stream task in DAG

Create real-time tasks (such as: flink-stream) through workflows. When real-time tasks appear in DAG, the connections before and after the tasks are displayed as dotted lines (this node will not actually run in the workflow, and the real-time tasks will run alone)
### Use case
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] EricGao888 commented on issue #11223: [Feature][Stream] Support stream task
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on issue #11223:
URL: https://github.com/apache/dolphinscheduler/issues/11223#issuecomment-1200647807
> @caishunfeng, IMO, the streaming task support needs to introduce event based scheduling mechanism in scheduler. Current scheduling mechanism depends on the status of the task, but streaming task is long running. cc @davidzollo ,@EricGao888
@SteNicholas Sounds pretty cool to me. Since this is a big feature, would you like to bring it up in the bi-weekly community conference for discussion and send a proposal to the community dev mail list? Thx~ cc @davidzollo @caishunfeng
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng closed issue #11223: [Feature][Stream] Support stream task
Posted by GitBox <gi...@apache.org>.
caishunfeng closed issue #11223: [Feature][Stream] Support stream task
URL: https://github.com/apache/dolphinscheduler/issues/11223
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SteNicholas commented on issue #11223: [Feature][Stream] Support stream task
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on issue #11223:
URL: https://github.com/apache/dolphinscheduler/issues/11223#issuecomment-1200844340
@EricGao888 @caishunfeng, I would like to make a proposal to the community dev mail list. Go ahead to discuss this scheduling feature on the weekly meeting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #11223: [Feature][Stream] Support stream task
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #11223:
URL: https://github.com/apache/dolphinscheduler/issues/11223#issuecomment-1200635883
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SteNicholas commented on issue #11223: [Feature][Stream] Support stream task
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on issue #11223:
URL: https://github.com/apache/dolphinscheduler/issues/11223#issuecomment-1200641102
@caishunfeng, IMO, the streaming task support needs to introduce event based scheduling mechanism in scheduler. Current scheduling mechanism depends on the status of the task, but streaming task is long running.
cc @EricGao888
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on issue #11223: [Feature][Stream] Support stream task
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on issue #11223:
URL: https://github.com/apache/dolphinscheduler/issues/11223#issuecomment-1200741809
> @caishunfeng, IMO, the streaming task support needs to introduce event based scheduling mechanism in scheduler. Current scheduling mechanism depends on the status of the task, but streaming task is long running. cc @davidzollo ,@EricGao888
@SteNicholas Thanks your reply, I'm agree with you, and I think it's a big feature too. In this feature, stream task will do nothing when workflow running, so maybe they don't conflict, we can follow up to introduce event based scheduling, WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org