You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Rajesh Balamohan (Jira)" <ji...@apache.org> on 2021/01/28 05:20:00 UTC
[jira] [Updated] (TEZ-4273) Clear off staging files when
TezYarnClient is unable to submit applications
[ https://issues.apache.org/jira/browse/TEZ-4273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rajesh Balamohan updated TEZ-4273:
----------------------------------
Attachment: TEZ-4273.1.patch
> Clear off staging files when TezYarnClient is unable to submit applications
> ---------------------------------------------------------------------------
>
> Key: TEZ-4273
> URL: https://issues.apache.org/jira/browse/TEZ-4273
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Priority: Major
> Attachments: TEZ-4273.1.patch
>
>
> Currently it leaves behind few resources like "tez-conf.pb" etc when exception is encountered during app submission. This causes issues in cluster, when apps continue to submit queries continuously.
> {noformat}
> drwx------ - hive supergroup 0 2021-01-28 01:58 /tmp/hive/hive/_tez_session_dir/806cd302-abd0-4694-be4a-b2b2473e75a8/.tez/application_1611791897439_0042
> -rw-r--r-- 3 hive supergroup 135519 2021-01-28 01:58 /tmp/hive/hive/_tez_session_dir/806cd302-abd0-4694-be4a-b2b2473e75a8/.tez/application_1611791897439_0042/tez-conf.pb
> -rw-r--r-- 3 hive supergroup 1056 2021-01-28 01:58 /tmp/hive/hive/_tez_session_dir/806cd302-abd0-4694-be4a-b2b2473e75a8/.tez/application_1611791897439_0042/tez.session.local-resources.pb
> Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1611791897439_0042 to YARN : org.apache.hadoop.security.AccessControlException: Queue root.default already has X applications from user hive cannot accept submission of application: application_1611791897439_0042
> at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:322) ~[hadoop-yarn-client-3.x:?]
> at org.apache.tez.client.TezYarnClient.submitApplication(TezYarnClient.java:77) ~[tez-api-0.9.x.jar:0.9.x]
> at org.apache.tez.client.TezClient.start(TezClient.java:405) ~[tez-api-0.9.x.jar:0.9.x]
> at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:535) ~[hive-exec-3.x]{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)