How to approach translating a GraphQL query AST to Relational Expr? (IE: "gql2rel", like "sql2rel")

[Gavin Ray <ra...@gmail.com>]

Hello all, happy holiday season =)
I have been reading through the tutorial materials on the Calcite site +
repo, and have watched the BOSS 2021 YouTube recording covering the Lucene
implementation (excellent workshop, by the way -- thank you!)

I am interested in working on a side project for fun & learning, but I'm
not familiar with this area, and also don't have a formal education.

My idea is to try to implement an auto-generated GraphQL API on top of
Calcite data sources.
Roughly the thought is something like:

1) Calcite adapter provides schema information/metadata
2) Some class/function consumes the schema, and generates the corresponding
GraphQL type and CRUD resolvers for it:
   - IE, for some table *"user"*, you might have *"find_user()"*,
*"insert_user()"*, *"update_user()"*, and *"delete_user()"*, which take
SQL-like filter & update expressions
3) The resolver functions translate the GraphQL query AST -> Calcite
query/expression AST, which is passed on to the adapters to execute

I am sure it is not so easy, but hopefully this gets the gist across:
==================================================
*[Server]: *makeGQLSchemaFromCalciteDataSources() --->
               GraphQL schema ---> *[Client receive]*

*[Client]:*  graphqlQuery("...")  --->
*               [Server receive]:* convertGQLQueryAstToRelExpr(query) --->
                                            Execute expression against
adapters --->
                                            Result ---> *[Client]*
==================================================

As a silly example (or if you've never heard of GraphQL), taking a possible
schema for a "collaborative" Todo app:
*user <-> user_todos <-> todo*

Here is what a GQL query might look like, and an example of the generated
Postgres SQL query + EXPLAIN ANALYZE plan for it (from a tool that does
this type of thing already):
- https://gist.github.com/GavinRay97/c667e71b595037388e72924a78708d86

I find this a really interesting problem, because of Calcite's ability to
dispatch a single query to multiple data sources.
This fits nicely with GraphQL's query model, because you can write things
like:

query {
>   user_from_db1(where: { name: { eq: "Person" } }) {
>      ...<fields>
>      todos_from_csv_file {
>           ...<fields>
>     }
> }


And then it is the server's job to fetch the right data from the right
places, efficiently.

I apologize if any of this (or the formatting) breaks mailing-list
etiquette, I've not used one before.

Best wishes,
Gavin Ray

Re: How to approach translating a GraphQL query AST to Relational Expr? (IE: "gql2rel", like "sql2rel")

[Stamatis Zampetakis <za...@gmail.com>]

Hi Gavin,

It sounds like a very interesting project.

I think it makes more sense to go from GraphQL AST to Calcite relational
algebra (RelNode).
Going forward you can draw some inspiration from CALCITE-785 [1],
CALCITE-3122 [2] doing similar work for the Pig.

Best,
Stamatis

[1] https://issues.apache.org/jira/browse/CALCITE-785
[2] https://issues.apache.org/jira/browse/CALCITE-3122

On Sun, Nov 28, 2021 at 7:40 AM Gavin Ray <ra...@gmail.com> wrote:

> Hello all, happy holiday season =)
> I have been reading through the tutorial materials on the Calcite site +
> repo, and have watched the BOSS 2021 YouTube recording covering the Lucene
> implementation (excellent workshop, by the way -- thank you!)
>
> I am interested in working on a side project for fun & learning, but I'm
> not familiar with this area, and also don't have a formal education.
>
> My idea is to try to implement an auto-generated GraphQL API on top of
> Calcite data sources.
> Roughly the thought is something like:
>
> 1) Calcite adapter provides schema information/metadata
> 2) Some class/function consumes the schema, and generates the corresponding
> GraphQL type and CRUD resolvers for it:
>    - IE, for some table *"user"*, you might have *"find_user()"*,
> *"insert_user()"*, *"update_user()"*, and *"delete_user()"*, which take
> SQL-like filter & update expressions
> 3) The resolver functions translate the GraphQL query AST -> Calcite
> query/expression AST, which is passed on to the adapters to execute
>
> I am sure it is not so easy, but hopefully this gets the gist across:
> ==================================================
> *[Server]: *makeGQLSchemaFromCalciteDataSources() --->
>                GraphQL schema ---> *[Client receive]*
>
> *[Client]:*  graphqlQuery("...")  --->
> *               [Server receive]:* convertGQLQueryAstToRelExpr(query) --->
>                                             Execute expression against
> adapters --->
>                                             Result ---> *[Client]*
> ==================================================
>
> As a silly example (or if you've never heard of GraphQL), taking a possible
> schema for a "collaborative" Todo app:
> *user <-> user_todos <-> todo*
>
> Here is what a GQL query might look like, and an example of the generated
> Postgres SQL query + EXPLAIN ANALYZE plan for it (from a tool that does
> this type of thing already):
> - https://gist.github.com/GavinRay97/c667e71b595037388e72924a78708d86
>
> I find this a really interesting problem, because of Calcite's ability to
> dispatch a single query to multiple data sources.
> This fits nicely with GraphQL's query model, because you can write things
> like:
>
> query {
> >   user_from_db1(where: { name: { eq: "Person" } }) {
> >      ...<fields>
> >      todos_from_csv_file {
> >           ...<fields>
> >     }
> > }
>
>
> And then it is the server's job to fetch the right data from the right
> places, efficiently.
>
> I apologize if any of this (or the formatting) breaks mailing-list
> etiquette, I've not used one before.
>
> Best wishes,
> Gavin Ray
>