You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Hive QA (JIRA)" <ji...@apache.org> on 2017/02/02 08:26:51 UTC
[jira] [Commented] (HIVE-15388) HiveParser spends lots of time in
parsing queries with lots of "("
[ https://issues.apache.org/jira/browse/HIVE-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849646#comment-15849646 ]
Hive QA commented on HIVE-15388:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12850587/HIVE-15388.04.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3321/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3321/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3321/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-02-02 08:26:44.137
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-3321/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-02-02 08:26:44.139
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 7cca097 HIVE-14420: Fix orc_llap_counters.q test failure in master (Prasanth Jayachandran reviewed by Siddharth Seth)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 7cca097 HIVE-14420: Fix orc_llap_counters.q test failure in master (Prasanth Jayachandran reviewed by Siddharth Seth)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-02-02 08:26:45.130
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java: No such file or directory
error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g: No such file or directory
error: a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFInternalInterval.java: No such file or directory
error: a/ql/src/test/org/apache/hadoop/hive/ql/parse/TestParseDriverIntervals.java: No such file or directory
error: a/ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFInternalInterval.java: No such file or directory
error: a/ql/src/test/queries/clientpositive/interval_alt.q: No such file or directory
error: a/ql/src/test/queries/clientpositive/udf_in.q: No such file or directory
error: a/ql/src/test/queries/clientpositive/udf_notop.q: No such file or directory
error: a/ql/src/test/results/clientnegative/char_pad_convert_fail2.q.out: No such file or directory
error: a/ql/src/test/results/clientnegative/invalid_select_expression.q.out: No such file or directory
error: a/ql/src/test/results/clientnegative/ptf_negative_DistributeByOrderBy.q.out: No such file or directory
error: a/ql/src/test/results/clientnegative/ptf_negative_PartitionBySortBy.q.out: No such file or directory
error: a/ql/src/test/results/clientnegative/ptf_window_boundaries.q.out: No such file or directory
error: a/ql/src/test/results/clientnegative/ptf_window_boundaries2.q.out: No such file or directory
error: a/ql/src/test/results/clientnegative/subquery_missing_from.q.out: No such file or directory
error: a/ql/src/test/results/clientnegative/subquery_subquery_chain.q.out: No such file or directory
error: a/ql/src/test/results/clientnegative/subquery_with_or_cond.q.out: No such file or directory
error: a/ql/src/test/results/clientnegative/udf_in.q.out: No such file or directory
error: a/ql/src/test/results/clientpositive/llap/vector_between_in.q.out: No such file or directory
error: a/ql/src/test/results/clientpositive/llap/vector_interval_arithmetic.q.out: No such file or directory
error: a/ql/src/test/results/clientpositive/perf/query14.q.out: No such file or directory
error: a/ql/src/test/results/clientpositive/show_functions.q.out: No such file or directory
error: a/ql/src/test/results/clientpositive/spark/vector_between_in.q.out: No such file or directory
error: a/ql/src/test/results/clientpositive/udf_between.q.out: No such file or directory
error: a/ql/src/test/results/clientpositive/udf_in.q.out: No such file or directory
error: a/ql/src/test/results/clientpositive/udf_notop.q.out: No such file or directory
error: a/ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out: No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12850587 - PreCommit-HIVE-Build
> HiveParser spends lots of time in parsing queries with lots of "("
> ------------------------------------------------------------------
>
> Key: HIVE-15388
> URL: https://issues.apache.org/jira/browse/HIVE-15388
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 2.2.0
> Reporter: Rajesh Balamohan
> Assignee: Pengcheng Xiong
> Attachments: HIVE-15388.01.patch, HIVE-15388.02.patch, HIVE-15388.03.patch, HIVE-15388.04.patch, hive-15388.stacktrace.txt
>
>
> Branch: apache-master (applicable with previous releases as well)
> Queries generated via tools can have lots of "(" for "AND/OR" conditions. This causes huge delays in parsing phase when the number of expressions are high.
> e.g
> {noformat}
> SELECT `iata`,
> `airport`,
> `city`,
> `state`,
> `country`,
> `lat`,
> `lon`
> FROM airports
> WHERE ((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((`airports`.`airport` = "Thigpen"
> OR `airports`.`airport` = "Astoria Regional")
> OR `airports`.`airport` = "Warsaw Municipal")
> OR `airports`.`airport` = "John F Kennedy Memorial")
> OR `airports`.`airport` = "Hall-Miller Municipal")
> OR `airports`.`airport` = "Atqasuk")
> OR `airports`.`airport` = "William B Hartsfield-Atlanta Intl")
> OR `airports`.`airport` = "Artesia Municipal")
> OR `airports`.`airport` = "Outagamie County Regional")
> OR `airports`.`airport` = "Watertown Municipal")
> OR `airports`.`airport` = "Augusta State")
> OR `airports`.`airport` = "Aurora Municipal")
> OR `airports`.`airport` = "Alakanuk")
> OR `airports`.`airport` = "Austin Municipal")
> OR `airports`.`airport` = "Auburn Municipal")
> OR `airports`.`airport` = "Auburn-Opelik")
> OR `airports`.`airport` = "Austin-Bergstrom International")
> OR `airports`.`airport` = "Wausau Municipal")
> OR `airports`.`airport` = "Mecklenburg-Brunswick Regional")
> OR `airports`.`airport` = "Alva Regional")
> OR `airports`.`airport` = "Asheville Regional")
> OR `airports`.`airport` = "Avon Park Municipal")
> OR `airports`.`airport` = "Wilkes-Barre/Scranton Intl")
> OR `airports`.`airport` = "Marana Northwest Regional")
> OR `airports`.`airport` = "Catalina")
> OR `airports`.`airport` = "Washington Municipal")
> OR `airports`.`airport` = "Wainwright")
> OR `airports`.`airport` = "West Memphis Municipal")
> OR `airports`.`airport` = "Arlington Municipal")
> OR `airports`.`airport` = "Algona Municipal")
> OR `airports`.`airport` = "Chandler")
> OR `airports`.`airport` = "Altus Municipal")
> OR `airports`.`airport` = "Neil Armstrong")
> OR `airports`.`airport` = "Angel Fire")
> OR `airports`.`airport` = "Waycross-Ware County")
> OR `airports`.`airport` = "Colorado City Municipal")
> OR `airports`.`airport` = "Hazelhurst")
> OR `airports`.`airport` = "Kalamazoo County")
> OR `airports`.`airport` = "Granville")
> OR `airports`.`airport` = "Silver Springs")
> OR `airports`.`airport` = "Whitford")
> OR `airports`.`airport` = "Biddeford Municipal")
> OR `airports`.`airport` = "Sugarloaf Regional")
> OR `airports`.`airport` = "Barnes Municipal")
> OR `airports`.`airport` = "Columbus Municipal")
> OR `airports`.`airport` = "Battle Mountain")
> OR `airports`.`airport` = "Huron County Memorial")
> OR `airports`.`airport` = "New Braunfels Municipal")
> OR `airports`.`airport` = "Benson Municipal")
> OR `airports`.`airport` = "Curtis")
> OR `airports`.`airport` = "Marlboro County")
> OR `airports`.`airport` = "Broken Bow Municipal")
> OR `airports`.`airport` = "Virginia Tech")
> OR `airports`.`airport` = "Bryce Canyon")
> OR `airports`.`airport` = "Black River Falls Area")
> OR `airports`.`airport` = "Boca Raton")
> OR `airports`.`airport` = "Birchwood")
> OR `airports`.`airport` = "Baudette International")
> OR `airports`.`airport` = "Blanding Muni")
> OR `airports`.`airport` = "Bradley International")
> OR `airports`.`airport` = "Morrilton Municipal")
> OR `airports`.`airport` = "Igor I Sikorsky Memorial")
> OR `airports`.`airport` = "Broadus")
> OR `airports`.`airport` = "Laurence G Hanscom")
> OR `airports`.`airport` = "Southwest Michigan Regional")
> OR `airports`.`airport` = "Bethel")
> OR `airports`.`airport` = "Bradford Reg")
> OR `airports`.`airport` = "Scotts Bluff County")
> OR `airports`.`airport` = "Boeing Field/King County Intl")
> OR `airports`.`airport` = "Buffalo Municipal")
> OR `airports`.`airport` = "Meadows")
> OR `airports`.`airport` = "Mobile Downtown")
> OR `airports`.`airport` = "Virgil I Grissom Municipal")
> OR `airports`.`airport` = "Silver Bay Municipal");
> {noformat}
> This query would take more than minutes to parse and compile.
> Temp Workaround: If all "("s in "where" condition are removed manually, this query would complete in 2-3 seconds.
> Problem exists in older versions as well (but in older versions this can make HS2 unresponsive for large amount of time, as Driver ends up taking global compiler lock in this phase).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)