Experimental removal syntax support for "select extract(ident from expr)"

["Wang, Youwei A" <yo...@intel.com>]

Greetings everyone.

The reason I want to do the removal mentioned in the caption is I have tried to add syntax support for the following function for Impala-889 like:
Select btrim(heading/tailing ExprY KW_FROM ExprZ);
By adding following code in the sql-parser.cup file:
  | function_name:fn_name LPAREN IDENT:x expr:y KW_FROM expr:z RPAREN
  {:  RESULT = new TrimExpr(fn_name, x, y, z); :}
  ;

I have implemented corresponding logic in the new class TrimExpr.java in this directory:
/root/Impala/fe/src/main/java/com/cloudera/impala/analysis

After that, I re-built all components: front-end/back-end/common.
However, the front-end parser prompts some error like:

Query: select btrim(heading "a%" from "abc%%defg%%%%%")
Query submitted at: 2016-09-19 12:13:53 (Coordinator: http://debian:25000)
ERROR: AnalysisException: Syntax error in line 1:
select btrim(heading "a%" from "abc%%defg%%%%%")
                                 ^
Encountered: STRING LITERAL
Expected: AND, BETWEEN, DIV, FROM, IGNORE, ILIKE, IN, IREGEXP, IS, LIKE, NOT, OR, REGEXP, RLIKE, COMMA
CAUSED BY: Exception: Syntax error

So this attempt is a failure. I am not sure which part is wrong.
I conducted a simple experiment to verify whether I have missed something by removing the syntax support for this expression:
Select extract(year from now());

First I commented out this syntax declaration in the sql-parser.cup file:
//{:  RESULT = new ExtractFromExpr(fn_name, u, t); :}
Then I removed the directory: /root/Impala/fe/generated-sources
And I run the following command to rebuild the front-end:
mvn clean; mvn package  -Dmaven.test.skip=true

After that, I started the impala front-end and run:
[debian:21000] > select extract(year from now());
Query: select extract(year from now())
2016
Fetched 1 row(s) in 0.00s

So my idea is: there must be something I missed to remove/clean so even the parser code for this "extract" is removed, the above query still works. I have also checked these two files:
./target/generated-sources/cup/com/cloudera/impala/analysis/SqlParser.java
./target/generated-sources/cup/com/cloudera/impala/analysis/SqlParserSymbols.java
Nothing related to "extract" is found in them after re-build the front-end.

And if I can trace out the reason, perhaps I can solve my initial issue of unrecognized syntax of "select btrim(heading "a%" from "abc%%defg%%%%%")".

Thank you everyone for taking time reading my mail.
Any suggestion/help/hint is highly appreciated. :)

Re: Experimental removal syntax support for "select extract(ident from expr)"

[Alex Behm <al...@cloudera.com>]

On Mon, Sep 19, 2016 at 8:03 AM, Wang, Youwei A <yo...@intel.com>
wrote:

> Greetings everyone.
>
> The reason I want to do the removal mentioned in the caption is I have
> tried to add syntax support for the following function for Impala-889 like:
> Select btrim(heading/tailing ExprY KW_FROM ExprZ);
> By adding following code in the sql-parser.cup file:
>   | function_name:fn_name LPAREN IDENT:x expr:y KW_FROM expr:z RPAREN
>   {:  RESULT = new TrimExpr(fn_name, x, y, z); :}
>   ;
>
> I have implemented corresponding logic in the new class TrimExpr.java in
> this directory:
> /root/Impala/fe/src/main/java/com/cloudera/impala/analysis
>
> After that, I re-built all components: front-end/back-end/common.
> However, the front-end parser prompts some error like:
>
> Query: select btrim(heading "a%" from "abc%%defg%%%%%")
> Query submitted at: 2016-09-19 12:13:53 (Coordinator: http://debian:25000)
> ERROR: AnalysisException: Syntax error in line 1:
> select btrim(heading "a%" from "abc%%defg%%%%%")
>                                  ^
> Encountered: STRING LITERAL
> Expected: AND, BETWEEN, DIV, FROM, IGNORE, ILIKE, IN, IREGEXP, IS, LIKE,
> NOT, OR, REGEXP, RLIKE, COMMA
> CAUSED BY: Exception: Syntax error
>
>
For expressions with special syntax that does not fit into the
generic function_call_expr production, it is recommended to create a
separate production, something like:

btrim_expr ::=
  KW_BTRIM LPAREN IDENT:x expr:y KW_FROM expr:z RPAREN
  {: RESULT ... :}

Yes, it means we need to create a new keyword, but otherwise it will be
difficult to resolve conflicts in our grammar with Extract.



> So this attempt is a failure. I am not sure which part is wrong.
> I conducted a simple experiment to verify whether I have missed something
> by removing the syntax support for this expression:
> Select extract(year from now());
>
> First I commented out this syntax declaration in the sql-parser.cup file:
> //{:  RESULT = new ExtractFromExpr(fn_name, u, t); :}
> Then I removed the directory: /root/Impala/fe/generated-sources
> And I run the following command to rebuild the front-end:
> mvn clean; mvn package  -Dmaven.test.skip=true
>
>
You can have this faster by doing "mvn compile".
Then you need to restart your Impala mini cluster with
bin/start-impala-cluster.py
If that does not pick up your changes, you can try resourcing
bin/impala-config.sh and bin/set-classpath.sh, then restart the mini
cluster and then try again.


> After that, I started the impala front-end and run:
> [debian:21000] > select extract(year from now());
> Query: select extract(year from now())
> 2016
> Fetched 1 row(s) in 0.00s
>
> So my idea is: there must be something I missed to remove/clean so even
> the parser code for this "extract" is removed, the above query still works.
> I have also checked these two files:
> ./target/generated-sources/cup/com/cloudera/impala/analysis/SqlParser.java
> ./target/generated-sources/cup/com/cloudera/impala/
> analysis/SqlParserSymbols.java
> Nothing related to "extract" is found in them after re-build the front-end.
>
> And if I can trace out the reason, perhaps I can solve my initial issue of
> unrecognized syntax of "select btrim(heading "a%" from "abc%%defg%%%%%")".
>
> Thank you everyone for taking time reading my mail.
> Any suggestion/help/hint is highly appreciated. :)
>
>