[jira] [Commented] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan

["Julian Hyde (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16899435#comment-16899435 ] 

Julian Hyde commented on CALCITE-3122:
--------------------------------------

I reviewed again. Thanks for all your changes. A much smaller list now:
* Syntax for links in javadoc is \{\@link String\} not \@\{\@link String\}.
* Now CALCITE-3195 is fixed you can remove the workaround in PigUDFWrapper
* Remove the CALCITE-3193 comments from the code. The JIRA case is sufficient.
* Can you expand/clarify the javadoc of VirtualTable? I don't really understand what it is or why you need it. All tables in Calcite are somewhat virtual. Your table is probably as real as the rest of them. So maybe it should be called "PigTable". Also remember there is already an interface Schema; I think your use of the term is different.
* Why did you comment out {{validateGroupList}}?
* {{sqlStatments}} should be {{sqlStatements}}

I still need to investigate {{RelOptCluster.planner}}. It really needs to stay {{final}}.

> Convert Pig Latin scripts into Calcite logical plan 
> ----------------------------------------------------
>
>                 Key: CALCITE-3122
>                 URL: https://issues.apache.org/jira/browse/CALCITE-3122
>             Project: Calcite
>          Issue Type: New Feature
>          Components: core, piglet
>            Reporter: Khai Tran
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.21.0
>
>          Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> We create an internal Calcite repo at LinkedIn and develop APIs to parse any Pig Latin scripts into Calcite logical plan. The code was tested in nearly ~1000 Pig scripts written at LinkedIn.
> Changes:
> 1. piglet: main conversion code live there, include:
>  * APIs to convert any Pig scripts into RelNode plans or SQL statements
>  * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan (DAGs)
>  * Convert Pig schemas into RelDatatype
>  * Traverse through Pig expression plan and convert Pig expressions into RexNodes
>  * Map some basic Pig UDFs to Calcite SQL operators
>  * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both Java and Python
>  * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes to RelNodes
>  * Have an optimizer rule to optimize Pig group/cogroup into Aggregate operators
> 2. core:
>  * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL
>  * Other minor changes in a few other classes to make Pig to Calcite works



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)