You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Zheng Shao (JIRA)" <ji...@apache.org> on 2009/04/15 09:08:14 UTC
[jira] Created: (HIVE-416) Get rid of backtrack in Hive.g
Get rid of backtrack in Hive.g
------------------------------
Key: HIVE-416
URL: https://issues.apache.org/jira/browse/HIVE-416
Project: Hadoop Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.4.0
Reporter: Zheng Shao
Assignee: Zheng Shao
Fix For: 0.4.0
Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699898#action_12699898 ]
Zheng Shao commented on HIVE-416:
---------------------------------
> About the comment on optional brackets - clearly these are optional in expressions. So how do we support those expression e.g (a+b) and a+b are both valid sql expressoins if we cannot support this without backtracking...
We do support "(a+b)" and "a+b".
The problem is that there is no easy way of supporting both "((((a))), b)" and "((((a)))), b". No matter what is k, it's not possible to determine whether the first "(" is the optional bracket for the expression list, or just part of the first expression.
I will need to go over the antlr book to know more about Semantic/Syntactic predicate to know whether that is possible.
> Identifier DOT Identifier.
Treating it as a lexical rule won't allow both T.a.b and a.b. I am making a first Identifier a TOK_TABLE_OR_COL. I will let SemanticAnalyzer to decide whether it is a table name or column name. Not sure that should go into the same transaction or not since it's a much bigger change.
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.1.patch, HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-416:
----------------------------
Status: Patch Available (was: Open)
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HIVE-416) Get rid of backtrack in
Hive.g
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700230#action_12700230 ]
Namit Jain edited comment on HIVE-416 at 4/17/09 9:11 AM:
----------------------------------------------------------
The changes look good to me.
+1
I can test and commit them, but will wait for a couple of hours if anyone has any issues, since many people have been reviewing this code.
I will wait till Friday evening (say around 5pm), if no-one has any objections, I will commit this (if the tests pass)
was (Author: namit):
The changes look good to me.
+1
I can test and commit them, but will wait for a couple of hours if anyone has any issues, since many people have been reviewing this code.
I will wait till Firday evening (say around 5pm), if no-one has any objections, I will commit this (if the tests pass)
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.1.patch, HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699957#action_12699957 ]
Raghotham Murthy commented on HIVE-416:
---------------------------------------
How about the following:
Add the production:
{{{
expr -> expr ',' expr
}}}
With this production and expr -> '(' expr ')' we can support arbitrarily nested parentheses. The issue is that the new production will create a left-deep tree of comma-expressions. We could implement a method which takes such a tree and flatten out comma expressions into expression lists.
Also, I remember Venky asking for arbitrarily nested parentheses around queries for his query authoring tool. We could do something similar and create comma-query-expressions.
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.1.patch, HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700013#action_12700013 ]
Zheng Shao commented on HIVE-416:
---------------------------------
@Raghu: I just had a second thought on that approach. The new production you add is left-recursive and it's not permitted in LL(k), but it's possible to use precedence rules to fix that.
However given all the change including the flattening, it seems to me that's too much work with very little benefit - who care about the optional brackets, for usage it's exactly the same.
For Venky's case, it's a separate problem. Venky's case is more like supporting "a" and "((((a))))". We should be able to support it easily once we allow omitting the sub query alias.
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.1.patch, HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-416:
----------------------------
Attachment: HIVE-416.1.2.branch-0.3.patch
This patch is for branch-0.3. It seems without this, the TestNegativeCliDriver sometimes produce wrong error messages (undeterministically). That is probably caused by the backtracking which is fixed with this patch.
Please commit this one to branch-0.3.
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.1.patch, HIVE-416.1.2.branch-0.3.patch, HIVE-416.1.2.patch, HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699874#action_12699874 ]
Ashish Thusoo commented on HIVE-416:
------------------------------------
Can we use semantic/syntactic predicates to support the optional brackets?
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-416:
----------------------------
Attachment: HIVE-416.1.1.patch
Extracted common prefix for ALTER TABLE, and Removed all "{k=5}".
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.1.patch, HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699840#action_12699840 ]
Namit Jain commented on HIVE-416:
---------------------------------
I had some questions.
1. Wont it be better to factor out the common part in a seperate rule instead of providing the look-ahead.
For eg:
instead of:
alterStatement
options {k=5;}
@init { msgs.push("alter statement"); }
@after { msgs.pop(); }
: alterStatementRename
| alterStatementAddCol
| alterStatementDropPartitions
| alterStatementAddPartitions
| alterStatementProperties
| alterStatementSerdeProperties
;
wont it be better to factor out < ALTER TABLE identifier> in a common and then have the remaining rules ?
2. On the same lines, I could not understand the reason for brackets around SELECT TRANSFORM and
no brackets around MAP/REDUCE.
Instead of this:
selectClause
@init { msgs.push("select clause"); }
@after { msgs.pop(); }
:
KW_SELECT (KW_ALL | dist=KW_DISTINCT)?
selectList -> {$dist == null}? ^(TOK_SELECT selectList)
-> ^(TOK_SELECTDI selectList)
|
trfmClause ->^(TOK_SELECT ^(TOK_SELEXPR trfmClause) )
;
if we factor out:
KW_SELECT for the first part and the transform clause, brackets should become optional.
Am I missing something here ?
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699853#action_12699853 ]
Zheng Shao commented on HIVE-416:
---------------------------------
1. I checked the generated code for {k=5;}, it's a nested if so there is no performance penalty. But I agree most grammars have k up to 3, and it should be easy to extract the common prefix, so I will do it.
2. optional brackets won't be possible with a LL(k) parser with any k (without backtrack), because I can construct an arbitarily long string like "(((((((a+b..." and it's not possible to know whether the first "(" is the optional bracket or not.
Most people who has been using "SELECT TRANSFORM" are adding the brackets, while those using "MAP/REDUCE" are probably not (think "MAP" / "REDUCE" similar to "SELECT"), that's why I made the choice like that. We can discuss more on this if needed.
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700230#action_12700230 ]
Namit Jain commented on HIVE-416:
---------------------------------
The changes look good to me.
+1
I can test and commit them, but will wait for a couple of hours if anyone has any issues, since many people have been reviewing this code.
I will wait till Firday evening (say around 5pm), if no-one has any objections, I will commit this (if the tests pass)
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.1.patch, HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-416:
----------------------------
Attachment: HIVE-416.1.2.patch
@Ashish:
This patch removes the unnecessary "{k=5}".
We cannot extract the common "SELECT" out because we allow "MAP", "REDUCE", as well as "SELECT TRANSFORM" in the trfmClause.
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.1.patch, HIVE-416.1.2.patch, HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-416:
----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed. Thanks Zheng
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.1.patch, HIVE-416.1.2.patch, HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-416:
----------------------------
Attachment: HIVE-416.1.patch
This patch changed Hive.g to disable backtrack.
All ambiguity are resolved except an unresolvable one: "Identifier DOT Identifier". This one will require some work in the SemanticAnalyzer as well.
We had to add a little restriction to the grammar:
1. "SELECT TRANSFORM" will have to have brackets around all expression List after it. Originally the brackets are optional;
2. "MAP" and "REDUCE" cannot have brackets around all expression list after it (just like SELECT). Originally we allow optional brackets.
Both changes are required if we want to disable backtrack.
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700261#action_12700261 ]
Ashish Thusoo commented on HIVE-416:
------------------------------------
A couple of minor comments:
1. Do we still need k=5 in the alterStatement rule considering that the ties can be broken by k=3 lookahead in alterStatementSuffix.
2. In the select list and select clause can factor out KW_SELECT as that is the common part in the two options in the selectClause rule.
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.1.patch, HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-416) Get rid of backtrack in Hive.g
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699873#action_12699873 ]
Ashish Thusoo commented on HIVE-416:
------------------------------------
I think for Identifier DOT Identifier we should probably treat it as a lexical rule rather than a grammar rule. It will also make it much simpler to support optional aliasing with complex types.
Right now
select T.a.b FROM T
and
select a.b FROM T
is very hard to handle in the SemanticAnalyzer as the grammar treats a as a table alias instead of a complex column name.
About the comment on optional brackets - clearly these are optional in expressions. So how do we support those expression e.g (a+b) and a+b are both valid sql expressoins if we cannot support this without backtracking...
> Get rid of backtrack in Hive.g
> ------------------------------
>
> Key: HIVE-416
> URL: https://issues.apache.org/jira/browse/HIVE-416
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.4.0
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: HIVE-416.1.patch
>
>
> Hive.g now still uses "backtrack=true". "backtrack" not only slows down the parsing in case of error, it can also produce wrong syntax error messages (usually based on the last try of the backtracking).
> We should follow http://www.antlr.org/wiki/display/ANTLR3/How+to+remove+global+backtracking+from+your+grammar to remove the need of doing backtrack.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.