You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "László Bodor (Jira)" <ji...@apache.org> on 2024/03/09 16:14:00 UTC
[jira] [Assigned] (HIVE-28112) Clear dagId from MDC/NDC when re-executing the query with new dagId
[ https://issues.apache.org/jira/browse/HIVE-28112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
László Bodor reassigned HIVE-28112:
-----------------------------------
Assignee: László Bodor
> Clear dagId from MDC/NDC when re-executing the query with new dagId
> -------------------------------------------------------------------
>
> Key: HIVE-28112
> URL: https://issues.apache.org/jira/browse/HIVE-28112
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
>
> 1. dag fails (dag_1709708735265_0007_94, hive_20240307162529_5ae040ec-7d46-4d79-9730-f4cd3f184ced)
> {code}
> <14>1 2024-03-07T16:29:54.292Z hiveserver2-0 hiveserver2 1 87c9f023-150f-4923-b044-3b4207506951 [mdc@18060 class="client.DAGClientImpl" dagId="dag_1709708735265_0007_94" level="INFO" operationLogLevel="EXECUTION" queryId="hive_20240307162529_5ae040ec-7d46-4d79-9730-f4cd3f184ced" sessionId="0ac1f6bf-98ea-4442-bb38-f5da5a36aeab" thread="HiveServer2-Background-Pool: Thread-1432633"] DAG completed. FinalState=FAILED
> {code}
> 2. AM lost plugin decides to re-execute:
> {code}
> <14>1 2024-03-07T16:29:54.301Z hiveserver2-0 hiveserver2 1 87c9f023-150f-4923-b044-3b4207506951 [mdc@18060 class="reexec.ReExecuteLostAMQueryPlugin" dagId="dag_1709708735265_0007_94" level="INFO" operationLogLevel="EXECUTION" queryId="hive_20240307162529_5ae040ec-7d46-4d79-9730-f4cd3f184ced" sessionId="0ac1f6bf-98ea-4442-bb38-f5da5a36aeab" thread="HiveServer2-Background-Pool: Thread-1432633"] Got exception message: AM record not found (likely died) in zookeeper for application id: application_1709708735265_0007 retryPossible: true
> {code}
> 3. there are messages, that belong to a new execution (when there is no DAG at all), still showing the last dagId, which is confusing, e.g:
> {code}
> <14>1 2024-03-07T16:29:54.348Z hiveserver2-0 hiveserver2 1 87c9f023-150f-4923-b044-3b4207506951 [mdc@18060 class="ql.Driver" dagId="dag_1709708735265_0007_94" level="INFO" operationLogLevel="EXECUTION" queryId="hive_20240307162529_5ae040ec-7d46-4d79-9730-f4cd3f184ced" sessionId="0ac1f6bf-98ea-4442-bb38-f5da5a36aeab" thread="HiveServer2-Background-Pool: Thread-1432633"] Compiling command(...
> {code}
> while compiling a query, a dag id is not even supposed to be present, dag id is got upon dag submission from the am
> the new dag id will correspond to the same hive query id, so hive query id can be used to keep the connection between query attempts
> the last message, that makes sense for the failed/last dag id is:
> {code}
> <14>1 2024-03-07T16:29:54.309Z hiveserver2-0 hiveserver2 1 87c9f023-150f-4923-b044-3b4207506951 [mdc@18060 class="reexec.ReExecDriver" dagId="dag_1709708735265_0007_94" level="INFO" operationLogLevel="EXECUTION" queryId="hive_20240307162529_5ae040ec-7d46-4d79-9730-f4cd3f184ced" sessionId="0ac1f6bf-98ea-4442-bb38-f5da5a36aeab" thread="HiveServer2-Background-Pool: Thread-1432633"] Preparing to re-execute query
> {code}
> so we might want to delete the dagId from MDC/NDC around this point: "Preparing to re-execute query"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)