You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/11/01 08:58:44 UTC
[GitHub] [dolphinscheduler] zhongjiajie opened a new issue #6658: [Bug] [python] Get same task code in bulk definition
zhongjiajie opened a new issue #6658:
URL: https://github.com/apache/dolphinscheduler/issues/6658
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
When I define workflow by code using python, it failed with not exists task. My code definitions as below:
```py
from pydolphinscheduler.core.process_definition import ProcessDefinition
from pydolphinscheduler.tasks.shell import Shell
# make sure your tenant exists
tenant = "zhongjiajie"
num_workflows = 100
num_tasks = 10
# Whether task should dependent or not, if True would create dependent like `pre-task -> task -> next-task`
is_chain = True
for wf in range(0, num_workflows):
with ProcessDefinition(name=f"wf:{wf}", tenant=tenant) as pd:
for t in range(0, num_tasks):
task = Shell(name=f"task:{t}-wf:{wf}", command=f"echo task {t}")
if is_chain and t > 0:
pre_task_name = f"task:{t-1}-wf:{wf}"
pd.get_one_task_by_name(pre_task_name) >> task
pd.submit()
```
It would sometime failed with error
```log
ValueError: Can not find task with name task:1-wf:0
```
or something like that. I debug and find it due to
https://github.com/apache/dolphinscheduler/blob/dev/dolphinscheduler-python/src/main/java/org/apache/dolphinscheduler/server/PythonGatewayServer.java#L132-L135. Python code run so fast, then we get same `code` and in
https://github.com/apache/dolphinscheduler/blob/daca3baf6662b9bcf3de856722ece79aad753773/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/core/task.py?_pjax=%23js-repo-pjax-container#L149-L150
task do not add to workflow
### What you expected to happen
Should not get error
### How to reproduce
run what I paste above
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6658: [Bug] [python] Get same task code in bulk definition
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6658:
URL: https://github.com/apache/dolphinscheduler/issues/6658#issuecomment-956050851
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie closed issue #6658: [Bug] [python] Get same task code in bulk definition
Posted by GitBox <gi...@apache.org>.
zhongjiajie closed issue #6658:
URL: https://github.com/apache/dolphinscheduler/issues/6658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #6658: [Bug] [python] Get same task code in bulk definition
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #6658:
URL: https://github.com/apache/dolphinscheduler/issues/6658#issuecomment-963852899
This issue is closed by #6744
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org