You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/10/27 09:23:01 UTC
[GitHub] [dolphinscheduler] Dev-Mao opened a new issue #6615: [Feature][dolphinscheduler-api] Problems encountered by the second open
Dev-Mao opened a new issue #6615:
URL: https://github.com/apache/dolphinscheduler/issues/6615
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
1. User switchover failed to load environment variables
For example,
a）：Dolphin will run python tasks sudo -u root python xxx.py. Dolphin will not load Python if I configure Python environment variables on Linux and not dolphin_env.
b）：The dolphin loading Python environment simply loads Python by default, but if python3 or something else, there will be problems and you need to copy and rename it to Python
2.Encapsulation of global exception modules
There are currently issues with exception encapsulation in the API. Some primary key collisions will be played directly in the console. Bad for troubleshooting
3.API issues
For example,
a）：For some automated tasks. The current API returns null data and cannot concatenate each phase of the process.
b）：For update operations, if you need to return a long processDefinitionjson(the string of processDefinitionjson that was saved), it is recommended to split the processDefinition apart and update the values as the user changes them.
4.System application problems
a）：When the system is running without tasks, the memory footprint is significant.
b）：Master is under too much pressure and has too many tasks to do. Can we split the tasks
### Use case
We are currently considering moving Airflow to DolphinScheduler where we receive hundreds of millions of dispatchers per day. I am currently working on dolphin.
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dev-Mao commented on issue #6615: [Feature][dolphinscheduler-api] Problems encountered by the second open
Posted by GitBox <gi...@apache.org>.
Dev-Mao commented on issue #6615:
URL: https://github.com/apache/dolphinscheduler/issues/6615#issuecomment-952915071
ok。thx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] removed a comment on issue #6615: [Feature][dolphinscheduler-api] Problems encountered by the second open
Posted by GitBox <gi...@apache.org>.
github-actions[bot] removed a comment on issue #6615:
URL: https://github.com/apache/dolphinscheduler/issues/6615#issuecomment-952718629
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] CalvinKirs commented on issue #6615: [Feature][dolphinscheduler-api] Problems encountered by the second open
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on issue #6615:
URL: https://github.com/apache/dolphinscheduler/issues/6615#issuecomment-952773267
@lenboo hi, Can you help look at this problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on issue #6615: [Feature][dolphinscheduler-api] Problems encountered by the second open
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on issue #6615:
URL: https://github.com/apache/dolphinscheduler/issues/6615#issuecomment-952777209
Wow, it's a big issue with many different point. Should it split?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6615: [Feature][dolphinscheduler-api] Problems encountered by the second open
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6615:
URL: https://github.com/apache/dolphinscheduler/issues/6615#issuecomment-952718629
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] lenboo commented on issue #6615: [Feature][dolphinscheduler-api] Problems encountered by the second open
Posted by GitBox <gi...@apache.org>.
lenboo commented on issue #6615:
URL: https://github.com/apache/dolphinscheduler/issues/6615#issuecomment-952806079
There seems to be a lot to do. We'll consider it.
- about the python path in environment variables, we can set the default python path to '/usr/bin/python', and you can use the function of environment configuration in version 2.0 to set environment config for different workers.
- 'There are currently issues with exception encapsulation in the API. Some primary key collisions will be played directly in the console. Bad for troubleshooting'
We have optimized the API in version 2.0. Our purpose is to make it easier for users to use API calls. If you find some problems, submit issues and let's fix them together
- b）：For update operations, if you need to return a long processDefinitionjson(the string of processDefinitionjson that was saved), it is recommended to split the processDefinition apart and update the values as the user changes them.
we have split the json in 2.0.0-alpha
- System application problems
a）：When the system is running without tasks, the memory footprint is significant.
b）：Master is under too much pressure and has too many tasks to do. Can we split the tasks
do you use the 1.3.x ？ you can try the 2.0.0-alpha, we make some improvements in the 2.0.
very appreciate you can submit these issues, it's useful for dolphinscheduler ability improvement.
Your follow-up feedback is welcome.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org