You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by dan young <da...@gmail.com> on 2017/08/14 17:19:36 UTC
Long time compiling query/explain.....
Running Hive 2.2 w/ LLAP enabled (tried the same thing in Hive 2.3 w/
LLAP), queries working but when we submit queries like the following (via
our automated test framework), they just seem to hang with Parsing
Command....Other queries seem to work fine.... Any idea on what's going on
or how to debug? I have debug log set and all we're seeing in the
HiveServer2 logs are:
2017-08-14T17:02:09,919 DEBUG [33a7d77a-688a-4766-b897-dbffef8ba6d3
HiveServer2-Handler-Pool: Thread-148] log.PerfLogger: <PERFLOG method=parse
from=org.apache.hadoop.hive.ql.Driver>
2017-08-14T17:02:09,933 DEBUG [33a7d77a-688a-4766-b897-dbffef8ba6d3
HiveServer2-Handler-Pool: Thread-148] parse.ParseDriver: Parsing command:
SELECT CASE WHEN (((TIMESTAMP('2015-11-30 23:30:00.000000000')) >=
((TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP(CONCAT(ADD_MONTHS(CAST(CONCAT(CAST(YEAR(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01 00:00:00.000000000')
, 'MM'))),11)))) AS STRING), '-',
LPAD(CAST(((CAST(CEIL(MONTH(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) / 3) AS INT) - 1) * 3)
+ 1 AS STRING), 2, '0'), '-01 00:00:00') AS TIMESTAMP),
-1),SUBSTRING(CAST(CONCAT(CAST(YEAR(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))
),11)))) AS STRING), '-',
LPAD(CAST(((CAST(CEIL(MONTH(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) / 3) AS INT) - 1) * 3) + 1 AS
STRING), 2, '0'), '-01 00:00:00') AS TIMESTAMP),11))), 'MM'))),
-3),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP(CONCAT(ADD_MONTHS(CAST(CONCAT(CAST(YEAR(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))), 1),SUBSTRING(TIMESTAMP(DATE(
TRUNC(TIMESTAMP('2015-12-01 00:00:00.000000000'), 'MM'))),11)))) AS
STRING), '-',
LPAD(CAST(((CAST(CEIL(MONTH(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01 00:00:00.00
0000000'), 'MM'))),11)))) / 3) AS INT) - 1) * 3) + 1 AS STRING), 2, '0'),
'-01 00:00:00') AS TIMESTAMP),
-1),SUBSTRING(CAST(CONCAT(CAST(YEAR(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))), 1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TI
MESTAMP('2015-12-01 00:00:00.000000000'), 'MM'))),11)))) AS STRING), '-',
LPAD(CAST(((CAST(CEIL(MONTH(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01 00:00:00.000000000'
), 'MM'))),11)))) / 3) AS INT) - 1) * 3) + 1 AS STRING), 2, '0'), '-01
00:00:00') AS TIMESTAMP),11))), 'MM'))),11))))) AND (TIMESTAMP('2015-11-30
23:30:00.000000000')) <
((TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP
(CONCAT(ADD_MONTHS(CAST(CONCAT(CAST(YEAR(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) AS STRING), '-',
LPAD(CAST(((CAST(CEIL(MONT
H(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) / 3) AS INT) - 1) * 3) + 1 AS STRING),
2, '0'), '-01 00:00:00') AS TIMESTAMP), -1)
,SUBSTRING(CAST(CONCAT(CAST(YEAR(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) AS STRING), '-',
LPAD(CAST(((CAST(CEIL(MONTH(TIMEST
AMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) / 3) AS INT) - 1) * 3) + 1 AS STRING),
2, '0'), '-01 00:00:00') AS TIMESTAMP),11))), 'MM')
)),
-3),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP(CONCAT(ADD_MONTHS(CAST(CONCAT(CAST(YEAR(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11
)))) AS STRING), '-',
LPAD(CAST(((CAST(CEIL(MONTH(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) / 3) AS INT) - 1) * 3) + 1 AS STRI
NG), 2, '0'), '-01 00:00:00') AS TIMESTAMP),
-1),SUBSTRING(CAST(CONCAT(CAST(YEAR(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) AS
STRING), '-',
LPAD(CAST(((CAST(CEIL(MONTH(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) / 3) AS INT) - 1) * 3) + 1 AS STRING),
2,
'0'), '-01 00:00:00') AS TIMESTAMP),11))), 'MM'))),11))), 'MM'))),
3),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP(CONCAT(ADD_MONTHS(CAST(CONCAT(CAST(YEAR(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00
.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) AS STRING), '-',
LPAD(CAST(((CAST(CEIL(MONTH(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) / 3) AS INT) - 1) * 3) + 1 AS STRING),
2, '0'), '-01 00:00:00') AS TIMESTAMP),
-1),SUBSTRING(CAST(CONCAT(CAST(YEAR(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) AS STRING), '-',
LPAD(CAST(((CAST(CEIL(MONTH(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP($ATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) / 3) AS INT) - 1) * 3) + 1 AS STRING),
2, '0'), '-01 00:00:00') AS TIMESTAMP),11))), 'MM'))),
-3),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP(CONCAT(ADD_MONTHS(CAST(CONCAT(CAST(YEAR(TIMESTAMP(CONCAT(ADD_MONTHS(TI$ESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) AS STRING), '-',
LPAD(CAST(((CAST(CEIL(MONTH(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-0$
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) / 3) AS INT) - 1) * 3) + 1 AS STRING),
2, '0'), '-01 00:00:00') AS TIMESTAMP),
-1),SUBSTRING(CAST(CONCAT(CAST(YEAR(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP($ATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) AS STRING), '-',
LPAD(CAST(((CAST(CEIL(MONTH(TIMESTAMP(CONCAT(ADD_MONTHS(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:$0.000000000'), 'MM'))),
1),SUBSTRING(TIMESTAMP(DATE(TRUNC(TIMESTAMP('2015-12-01
00:00:00.000000000'), 'MM'))),11)))) / 3) AS INT) - 1) * 3) + 1 AS STRING),
2, '0'), '-01 00:00:00') AS TIMESTAMP),11))), 'MM'))),11))),
'MM'))),11))))))) THEN 1 ELSE 0 END
AS x_results
Re: Long time compiling query/explain.....
Posted by dan young <da...@gmail.com>.
Thank you Gopal, I'll try to upgrade to 2.3 and see how that goes.
On Mon, Aug 14, 2017 at 4:08 PM Gopal Vijayaraghavan <go...@apache.org>
wrote:
>
>
> > Running Hive 2.2 w/ LLAP enabled (tried the same thing in Hive 2.3 w/
> LLAP), queries working but when we submit queries like the following (via
> our automated test framework), they just seem to hang with Parsing
> Command....Other queries seem to work fine.... Any idea on what's going on
> or how to debug? I have debug log set and all we're seeing in the
> HiveServer2 logs are:
>
> Get a jstack - like 20 jstacks 1s apart.
>
> I think this be similar to the parenthesis parsing bug fixed in 2.3.0.
>
> https://issues.apache.org/jira/browse/HIVE-16074
>
> Also getting a dummy table schema would help me run explain on my own
> builds.
>
> Cheers,
> Gopal
>
>
>
>
Re: Long time compiling query/explain.....
Posted by Gopal Vijayaraghavan <go...@apache.org>.
> Running Hive 2.2 w/ LLAP enabled (tried the same thing in Hive 2.3 w/ LLAP), queries working but when we submit queries like the following (via our automated test framework), they just seem to hang with Parsing Command....Other queries seem to work fine.... Any idea on what's going on or how to debug? I have debug log set and all we're seeing in the HiveServer2 logs are:
Get a jstack - like 20 jstacks 1s apart.
I think this be similar to the parenthesis parsing bug fixed in 2.3.0.
https://issues.apache.org/jira/browse/HIVE-16074
Also getting a dummy table schema would help me run explain on my own builds.
Cheers,
Gopal