You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/03/15 11:05:36 UTC
[GitHub] [dolphinscheduler] caishunfeng opened a new issue #8915: [Improvement][Worker] Get yarn application id by application tag
caishunfeng opened a new issue #8915:
URL: https://github.com/apache/dolphinscheduler/issues/8915
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
Now DS get the yarn application id from parsing task log, which is not graceful.
The idea to query application id by the giving unique tag, such as 'tag-uuid', the tag can set by user from UI, and then we can use the yarn cli to query application id directly by `./bin/yarn application --list | grep tag-uuid`.
### Use case
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #8915: [Discussion][Worker] Get yarn application id by application tag
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #8915:
URL: https://github.com/apache/dolphinscheduler/issues/8915#issuecomment-1069825937
> > That's a good idea. But I think `yarn application -list` is not as good as parsing task log.
> > In yarn application, only application name can be customized by users. However, some components do not support setting application name in yarn. Such as `hive on tez`. Which is highly recommanded by AWS.
>
> @SbloodyS thanks for your feedback, so now the way to parse task log can also support `hive on tez`? If you have other good idea, please let me know, thanks~
Yes. All hive programs will output the ```yarn application id``` to ```stdout```.
I haven't thought of a better way at present.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #8915: [Improvement][Worker] Get yarn application id by application tag
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #8915:
URL: https://github.com/apache/dolphinscheduler/issues/8915#issuecomment-1069002732
That's a good idea. But I think ```yarn application -list``` is not as good as parsing task log. In yarn application, only application name can be customized by users. However, some components do not support setting application name in yarn. Such as ```hive on tez```. Which is highly recommanded by AWS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS edited a comment on issue #8915: [Improvement][Worker] Get yarn application id by application tag
Posted by GitBox <gi...@apache.org>.
SbloodyS edited a comment on issue #8915:
URL: https://github.com/apache/dolphinscheduler/issues/8915#issuecomment-1069002732
That's a good idea. But I think ```yarn application -list``` is not as good as parsing task log.
In yarn application, only application name can be customized by users. However, some components do not support setting application name in yarn. Such as ```hive on tez```. Which is highly recommanded by AWS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #8915: [Improvement][Worker] Get yarn application id by application tag
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #8915:
URL: https://github.com/apache/dolphinscheduler/issues/8915#issuecomment-1067854097
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on issue #8915: [Improvement][Worker] Get yarn application id by application tag
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on issue #8915:
URL: https://github.com/apache/dolphinscheduler/issues/8915#issuecomment-1069824174
> That's a good idea. But I think `yarn application -list` is not as good as parsing task log.
>
> In yarn application, only application name can be customized by users. However, some components do not support setting application name in yarn. Such as `hive on tez`. Which is highly recommanded by AWS.
@SbloodyS thanks for your feedback, so now the way to parse task log can also support `hive on tez`? If you have other good idea, please let me know, thanks~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org