[jira] [Comment Edited] (HIVE-22173) HiveServer2: Query with multiple lateral view hung forever during compile stage

["Stamatis Zampetakis (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-22173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642144#comment-17642144 ] 

Stamatis Zampetakis edited comment on HIVE-22173 at 12/1/22 7:55 PM:
---------------------------------------------------------------------

I will simplify a bit the query to demonstrate the problem. Consider the following setting:
{code:sql}
CREATE TABLE customer(orders array<string>);

EXPLAIN SELECT *
FROM customer
lateral view explode(orders) v as c1
lateral view explode(orders) v as c2
lateral view explode(orders) v as c3
lateral view explode(orders) v as c4
; 
{code}

The structure of the operator plan [before logical optimizations|https://github.com/apache/hive/blob/b053c61e6d9ffe36f3197a3efe88732bef726ecf/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L12904] is shown in  [^op_plan_4_lateral_views.pdf] .

The plan is comprised from 4 sub-graphs (one for each lateral view appearing in the query) connected together in a linear fashion. We can observe that from each LVF node we have exactly two paths that merge onto an LVJ node. 

Recursive algorithms follow a DFS traversal pattern some allowing a node to be visited again (e.g., when trying to enumerate all possible paths in a DAG) and some not. Such kind of algorithms appear in various places in Hive and two that cause problems in this case are shown below:
* PreOrderOnceWalker uses DFS and allows a node to be visited again
* The JSON plan serialization (ExplainTask#getJSONPlan) uses DFS and allows a node to be visited again

By allowing a node to be visited again we are more or less enumerating all paths from the root to every other node in the DAG. Observe that for this particular plan with the lateral views we have 2^N paths from the TS (root) to the FS (leaf) where N is the number of lateral views in the query. When the N is small (e.g., in this case 4) the query finishes normally but as soon as it grows a bit (15 to 20) the query will get stuck (or die with OOM) in every place where there is recursion (DFS traversal) with repetition.

Next steps/potential solutions:
* Check if it is possible to change the shape of the operator tree;
* Check if the DFS algorithms that allow nodes to be visited again can be changed/fixed;
* Detect potentially expensive traversals and selectively disable optimizations.


was (Author: zabetak):
I will simplify a bit the query to demonstrate the problem. Consider the following setting:
{code:sql}
CREATE TABLE customer(orders array<string>);

EXPLAIN SELECT *
FROM customer
lateral view explode(orders) v as c1
lateral view explode(orders) v as c2
lateral view explode(orders) v as c3
lateral view explode(orders) v as c4
; [^op_plan_4_lateral_views.pdf] 
{code}

The structure of the operator plan [before logical optimizations|https://github.com/apache/hive/blob/b053c61e6d9ffe36f3197a3efe88732bef726ecf/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L12904] is shown in  [^op_plan_4_lateral_views.pdf] .

The plan is comprised from 4 sub-graphs (one for each lateral view appearing in the query) connected together in a linear fashion. We can observe that from each LVF node we have exactly two paths that merge onto an LVJ node. 

Recursive algorithms follow a DFS traversal pattern some allowing a node to be visited again (e.g., when trying to enumerate all possible paths in a DAG) and some not. Such kind of algorithms appear in various places in Hive and two that cause problems in this case are shown below:
* PreOrderOnceWalker uses DFS and allows a node to be visited again
* The JSON plan serialization (ExplainTask#getJSONPlan) uses DFS and allows a node to be visited again

By allowing a node to be visited again we are more or less enumerating all paths from the root to every other node in the DAG. Observe that for this particular plan with the lateral views we have 2^N paths from the TS (root) to the FS (leaf) where N is the number of lateral views in the query. When the N is small (e.g., in this case 4) the query finishes normally but as soon as it grows a bit (15 to 20) the query will get stuck (or die with OOM) in every place where there is recursion (DFS traversal) with repetition.

Next steps/potential solutions:
* Check if it is possible to change the shape of the operator tree;
* Check if the DFS algorithms that allow nodes to be visited again can be changed/fixed;
* Detect potentially expensive traversals and selectively disable optimizations.

> HiveServer2: Query with multiple lateral view hung forever during compile stage
> -------------------------------------------------------------------------------
>
>                 Key: HIVE-22173
>                 URL: https://issues.apache.org/jira/browse/HIVE-22173
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 3.1.1, 4.0.0-alpha-1
>         Environment: Hive-3.1.1, Java-8
>            Reporter: Rajkumar Singh
>            Priority: Critical
>         Attachments: op_plan_4_lateral_views.pdf, thread-progress.log
>
>
> Steps To Repro:
> {code:java}
> -- create table 
> CREATE EXTERNAL TABLE `jsontable`( 
> `json_string` string) 
> ROW FORMAT SERDE 
> 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
> STORED AS INPUTFORMAT 
> 'org.apache.hadoop.mapred.TextInputFormat' 
> OUTPUTFORMAT 
> 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' ;
> -- Run explain of the query
> explain SELECT
> *
> FROM jsontable
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.addr.city'), "\\[|\\]|\"", ""),',')) t1 as c1
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.addr.country'), "\\[|\\]|\"", ""),',')) t2 as c2
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.addr'), "\\[|\\]|\"", ""),',')) t3 as c3
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.addr.postalCode'), "\\[|\\]|\"", ""),',')) t4 as c4
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.addr.state'), "\\[|\\]|\"", ""),',')) t5 as c5
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.addr.streetAddressLine'), "\\[|\\]|\"", ""),',')) t6 as c6
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.dummyfield'), "\\[|\\]|\"", ""),',')) t7 as c7
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.dummyfield'), "\\[|\\]|\"", ""),',')) t8 as c8
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.dummyfield.name.suffix'), "\\[|\\]|\"", ""),',')) t9 as c9
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.id.extension'), "\\[|\\]|\"", ""),',')) t10 as c10
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.id'), "\\[|\\]|\"", ""),',')) t11 as c11
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.id.root'), "\\[|\\]|\"", ""),',')) t12 as c12
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.telecom.'), "\\[|\\]|\"", ""),',')) t13 as c13
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.dummyfield1.use'), "\\[|\\]|\"", ""),',')) t14 as c14
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield.dummyfield1.value'), "\\[|\\]|\"", ""),',')) t15 as c15
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield1.dummyfield1.code'), "\\[|\\]|\"", ""),',')) t16 as c16
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield1.dummyfield1.value'), "\\[|\\]|\"", ""),',')) t17 as c17
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield2.city'), "\\[|\\]|\"", ""),',')) t18 as c18
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield2.city'), "\\[|\\]|\"", ""),',')) t19 as c19
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield2.country'), "\\[|\\]|\"", ""),',')) t20 as c20
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield2.country'), "\\[|\\]|\"", ""),',')) t21 as c21
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield'), "\\[|\\]|\"", ""),',')) t22 as c22
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield2.postalCode'), "\\[|\\]|\"", ""),',')) t23 as c23
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield2.postalCode'), "\\[|\\]|\"", ""),',')) t24 as c24
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield2.state'), "\\[|\\]|\"", ""),',')) t25 as c25
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield2.state'), "\\[|\\]|\"", ""),',')) t26 as c26
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield2'), "\\[|\\]|\"", ""),',')) t27 as c27
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield2.streetAddressLine'), "\\[|\\]|\"", ""),',')) t28 as c28
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield2.use'), "\\[|\\]|\"", ""),',')) t29 as c29
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield3'), "\\[|\\]|\"", ""),',')) t30 as c30
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield3'), "\\[|\\]|\"", ""),',')) t31 as c31
> lateral view explode(split(regexp_replace(get_json_object(jsontable.json_string, '$.jsonfield4'), "\\[|\\]|\"", ""),',')) t32 as c32
> ;
> -- it will hung forever
> {code}
> – HS2 jstacks
> {code:java}
> // 
> "8ed37c3a-be03-4f74-9afd-419d05609b9c HiveServer2-Handler-Pool: Thread-85" #85 prio=5 os_prio=0 tid=0x00007f3bd873f800 nid=0x90b94 runnable [0x00007f3baa6e2000]"8ed37c3a-be03-4f74-9afd-419d05609b9c HiveServer2-Handler-Pool: Thread-85" #85 prio=5 os_prio=0 tid=0x00007f3bd873f800 nid=0x90b94 runnable [0x00007f3baa6e2000]   java.lang.Thread.State: RUNNABLE at java.util.regex.Pattern$Curly.match0(Pattern.java:4272) at java.util.regex.Pattern$Curly.match(Pattern.java:4234) at java.util.regex.Pattern$Slice.match(Pattern.java:3972) at java.util.regex.Pattern$GroupHead.match(Pattern.java:4658) at java.util.regex.Matcher.match(Matcher.java:1270) at java.util.regex.Matcher.matches(Matcher.java:604) at org.apache.hadoop.hive.ql.lib.RuleRegExp.costPatternWithWildCardChar(RuleRegExp.java:236) at org.apache.hadoop.hive.ql.lib.RuleRegExp.cost(RuleRegExp.java:279) at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:72) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:43) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.PreOrderOnceWalker.walk(PreOrderOnceWalker.java:54) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:120) at org.apache.hadoop.hive.ql.ppd.SyntheticJoinPredicate.transform(SyntheticJoinPredicate.java:106) at org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:250) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12423) at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1816) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1811) at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197) at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:262) at org.apache.hive.service.cli.operation.Operation.run(Operation.java:247) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:575) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:561) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) at com.sun.proxy.$Proxy45.executeStatementAsync(Unknown Source) at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:315) at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:566) at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557) at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
> {code}
>  
> attached 10 jstacks taken at the interval of 30 seconds.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)