[jira] [Comment Edited] (PHOENIX-2199) Support DDL in Phoenix-Calcite integration

["Kaide Mu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15179134#comment-15179134 ] 

Kaide Mu edited comment on PHOENIX-2199 at 3/4/16 1:46 AM:
-----------------------------------------------------------

Hi, dear James.

Thanks for the replay, I had been looking for the commit codes and test codes, but I'm kind of confused, I don't know how it actually works, Would you mind explain to me a bit about how is the workflow and detail of the template? 
As far as I know, now Phoenix works in the following way, taking as example CREATE SEQUNCES like:

{code:sql}
CREATE SEQUENCE tmp
  MINVALUE 1
  MAXVALUE 100
  START WITH 1
  INCREMENT BY 1;
{code}

This statement is parsed (sqlparser) to create a CreateSequenceStatement, once we have it, we pass it to CreateSequenceCompiler and then this calls to createSequence of MetaDataClient.

In case of Phoenix-Calcite integration, I guess the workflow it's the following one:
Parse statement to create a SqlCreateSequence, this is pass to PhoenixPrepareImpl to execute the proper DDL statement.

Regarding to implementation detail of the template
{code:java}
public class SqlCreateView extends SqlCall {
    public static final SqlOperator OPERATOR = new SqlDdlOperator("CREATE VIEW", SqlKind.CREATE_VIEW);

    public final SqlIdentifier name;
    public final SqlNode query;
    public final String queryString;

    /** Creates a CREATE VIEW. */
    public SqlCreateView(SqlParserPos pos, SqlIdentifier name, SqlNode query, String queryString) {
        super(pos);
        this.name = name;
        this.query = query; 
        this.queryString = queryString;
    }

    public SqlOperator getOperator() {
        return OPERATOR;
    }

    public List<SqlNode> getOperandList() {
        return ImmutableList.of(name, query);
    }

    @Override
    public void unparse(SqlWriter writer, int leftPrec, int rightPrec) {
        writer.keyword("CREATE VIEW");
        name.unparse(writer, 0, 0);
        writer.keyword(" ");
        query.unparse(writer, 0, 0);
    }
}
{code}
I have the following doubts:
- Does name refer to table name, sequence name?
- What's the difference between query and queryString, with query are you referring all the sql statement? Taking as example the initial create sequence statement, query is "MINVALUE, MAXVALUE, START WITH, and INCREMENT BY" or instead it has several sqlnode?
- unparse is used to retrieve the initial unparsed statement?

Also, would you mind give me an example or some references? I'd really appreciate it and I'm sorry for this long comment but I need more detail about this issue. Thank you.


was (Author: kdmu):
Hi, dear James.

Thanks for the replay, I had been looking for the commit codes and test codes, but I'm kind of confused, I don't know how it actually works, Would you mind explain to me a bit about how is the workflow and detail of the template? 
As far as I know, now Phoenix works in the following way, taking as example CREATE SEQUNCES like:

{code:sql}
CREATE SEQUENCE tmp
  MINVALUE 1
  MAXVALUE 100
  START WITH 1
  INCREMENT BY 1;
{code}

This statement is parsed (sqlparser) to create a CreateSequenceStatement, once we have it, we pass it to CreateSequenceCompiler and then this calls to createSequence of MetaDataClient.

In case of Phoenix-Calcite integration, I guess the workflow it's the following one:
Parse statement to create a SqlCreateSequence, this is pass to PhoenixPrepareImpl to execute the proper DDL statement.

Regarding to implementation detail of the template
{code:java}
public class SqlCreateView extends SqlCall {
    public static final SqlOperator OPERATOR = new SqlDdlOperator("CREATE VIEW", SqlKind.CREATE_VIEW);

    public final SqlIdentifier name;
    public final SqlNode query;
    public final String queryString;

    /** Creates a CREATE VIEW. */
    public SqlCreateView(SqlParserPos pos, SqlIdentifier name, SqlNode query, String queryString) {
        super(pos);
        this.name = name;
        this.query = query; 
        this.queryString = queryString;
    }

    public SqlOperator getOperator() {
        return OPERATOR;
    }

    public List<SqlNode> getOperandList() {
        return ImmutableList.of(name, query);
    }

    @Override
    public void unparse(SqlWriter writer, int leftPrec, int rightPrec) {
        writer.keyword("CREATE VIEW");
        name.unparse(writer, 0, 0);
        writer.keyword(" ");
        query.unparse(writer, 0, 0);
    }
}
{code}
I have the following doubts:
- Does name refer to table name, sequence name?
- What's the difference between query and queryString, with query are you referring all the sql statement? Taking as example the initial create sequence statement, query is "MINVALUE, MAXVALUE, START WITH, and INCREMENT BY" or instead it has several sqlnode?
- unparse is used to retrieve the initial unparsed statement?

Also, would you mind give me an example or some references? I'd really appreciate it and I'm sorry for this long comment but I need more detail about this issue.

> Support DDL in Phoenix-Calcite integration
> ------------------------------------------
>
>                 Key: PHOENIX-2199
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-2199
>             Project: Phoenix
>          Issue Type: Task
>            Reporter: James Taylor
>              Labels: calcite, gsoc2016
>
> The existing Phoenix compiler classes are of the form Create<SQL type>Compiler. For example CreateTableCompiler handles both VIEW and TABLE compilation, CreateSequenceCompiler is for CREATE SEQEUNCE, and CreateIndexCompiler is for CREATE INDEX. The compiler class does most of the validation and then calls a MetaDataClient method to update the meta data repository.
> As a general implementation strategy, we'd transfer over the validation code from the compiler methods to new classes plugged into the JavaCC compilation of Calcite and then have these new classes call the same, somewhat modified MetaDataClient APIs. A slightly more detailed breakdown of the work includes:
> - Creating new JavaCC rules using the same templating technique as we did for CREATE VIEW (see https://github.com/apache/phoenix/pull/113/commits), marking the parse node produced as DDL.
> - Producing the information required while the Calcite parser is running to enable the call to the appropriate MetaDataClient API to update the metadata repository.
> - Tweaking the MetaDataClient APIs to not depend on the objects created at parse time, but instead pass through the constituent parts. For example, instead of passing through a CreateTableStatement object in the MetaDataClient.createTable() call, only the necessary member variables would be passed through. The idea would be to break the dependency on parse-time object after compilation is complete.
> - Changing ParseNode references used in MetaDataClient, as we don't want to continue maintaining these. By ParseNode, I mean the parts in the grammar that match an expression and produce a ParseNode instance. For DDL, this is the CreateIndexStatement object, which stores a list of ParseNodes as the indexed expressions. These are currently processed in MetaDataClient, so these could be changed to either Expressions or RexNodes.
> - Collecting at parse time the list of UDFs that need to be resolved. I see this in CreateIndexStatement, but not CreateTableStatement (which I believe is an oversight - probably wouldn't work to reference a UDF in the WHERE clause of a view).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)