[DataFusion] Cypher Query Language

[Anthony GRIFFON <an...@griffon.one>]

Hi, I wanted to discuss about the possible support of Cypher query planning for the Datafusion project, I would like to know if you would be open to have a work in that direction for the project.

Thank you all

Anthony Griffon

Re: [DataFusion] Cypher Query Language

[Andrew Lamb <al...@influxdata.com>]

Hi Chak-Pong,

Thank you for the interesting email. I am not familiar with the Cipher
language, but as long as it can be mapped into relational operators (e.g.
that pass record batches with the same schema) it would probably work well.

DataFusion is designed to be used as an embeddable engine -- while it
provides its own SQL interface, several projects implement alternate query
languages by creating  LogicalPlans (what comes out of the SQL planner). We
do this in IOx to implement a low level query interface with the Flux
Language[2] and are in the process of implementing the InfluxQL language in
the same way. VegaFusion[3] uses DataFusion to execute the vega
visualization language in a similar way. There are also various extension
points that can be used to add custom functions and operators, which we
use.

One way to implement a Cipher would be to parse a Cipher query and then
create a LogicalPlan from it, perhaps using LogicalPlanBuilder [4] . They
key question challenge is likely figuring out the mapping of query to plan.

You can see more about the design here [1] pg 19.

Hope that helps,
Andrew


[1]
https://www.slideshare.net/influxdata/influxdb-iox-tech-talks-query-engine-design-and-the-rustbased-datafusion-in-apache-arrow-244161934

[2]
https://github.com/influxdata/influxdb_iox/blob/main/iox_query/src/frontend/influxrpc.rs
[3] https://github.com/vegafusion/vegafusion
[4]
https://docs.rs/datafusion-expr/14.0.0/datafusion_expr/logical_plan/builder/struct.LogicalPlanBuilder.html


On Wed, Nov 16, 2022 at 11:05 PM Chak-Pong Chung <ch...@gmail.com>
wrote:

> Do you have some large cypher queries as test cases? Those from LDBC are
> relatively small.
>
> On Tue, Nov 15, 2022 at 12:40 PM Anthony GRIFFON <an...@griffon.one>
> wrote:
>
> > Hi, I wanted to discuss about the possible support of Cypher query
> > planning for the Datafusion project, I would like to know if you would be
> > open to have a work in that direction for the project.
> >
> > Thank you all
> >
> > Anthony Griffon
>
>
>
> --
> Regards,
> Chak-Pong
>

Re: [DataFusion] Cypher Query Language

[Chak-Pong Chung <ch...@gmail.com>]

Do you have some large cypher queries as test cases? Those from LDBC are
relatively small.

On Tue, Nov 15, 2022 at 12:40 PM Anthony GRIFFON <an...@griffon.one>
wrote:

> Hi, I wanted to discuss about the possible support of Cypher query
> planning for the Datafusion project, I would like to know if you would be
> open to have a work in that direction for the project.
>
> Thank you all
>
> Anthony Griffon



-- 
Regards,
Chak-Pong

Re: [DataFusion] Cypher Query Language

[vin jake <ja...@gmail.com>]

I developed graph database past.
I think it's a hard thing.
we need implement new planner, new optimizer, new executor... in fact, it's
a new system😂.

Anthony GRIFFON <an...@griffon.one> 于 2022年11月16日周三 01:40写道：

> Hi, I wanted to discuss about the possible support of Cypher query
> planning for the Datafusion project, I would like to know if you would be
> open to have a work in that direction for the project.
>
> Thank you all
>
> Anthony Griffon