Calcite adapter for SPARQL endpoints

[Nicola Vitucci <ni...@gmail.com>]

Hi all,

I have been dabbling with Calcite for a while out of interest and curiosity
and, in order to become more familiar with the code and the overall
architecture, I've tried implementing an adapter [1] to run SQL queries
over SPARQL endpoints. As Gavin has done, I have also planned to share some
notes on the process of creating an adapter from scratch, since there are
many moving parts (and I may have missed or misunderstood several).

The project is at an early stage and can be improved in many ways, but, as
I am still learning about Calcite itself, I thought it would make sense to
share the work in progress with the community and ask for some feedback
(plus, it's easier to ask for help when there is something concrete
already). The rules are still relatively basic (e.g. neither joins nor
aggregations are pushed down to SPARQL as of now); in the meantime, I am
trying to come up with as many tests as I can to make sure that everything
works (and keeps working) as expected.

On the "usefulness" side, it is true that trying to force a table view on
graph data is always going to be a challenge, and there are many approaches
to it - but I think this could be a good starting point for use cases such
as data exploration, analytics, etc.

Thank you all for the great work on Calcite, and please do share your
thoughts and comments!

Nicola

[1] https://github.com/nvitucci/calcite-sparql

Re: Calcite adapter for SPARQL endpoints

[Nicola Vitucci <ni...@gmail.com>]

Hey Gavin,

I had never heard of SPARQL, it appears to be an RDF/triplestore language.
> This is really great -- it means that you can now execute graph-based
> queries against Calcite's adapters!
>

You're right, my bad for not having introduced SPARQL. It is indeed a query
language for triplestores/quadstores, so I think what I've done with the
adapter is kind of the contrary of what you mentioned here: it can be used
to run SQL queries on triplestores that only support SPARQL for querying,
actually exposing a sort of table view of the underlying graph.


> From brief googling, it seems there are implementations of SPARQL that run
> on top of major DB's like Postgres/MySQL.
> So this could be used with the JDBC adapter then I assume?
>

If you mean to use SPARQL a bit like you've done with GraphQL (in the
"other direction", i.e. in place of SQL to query non-graph DBs) then I
think yes.


> > "I have also planned to share some notes on the process of creating an
> adapter from scratch, since there are
> many moving parts (and I may have missed or misunderstood several)."
>
> This would be great, you appear to have a better understanding of the
> pieces of Calcite and have implemented a lot more custom behavior.
> Would love to read a walkthrough/explainer or guide and it would likely be
> invaluable to others.
>

Absolutely. It took some effort to put all the pieces in place, and I'd be
glad to share my experience with that.

Nicola

Re: Calcite adapter for SPARQL endpoints

[Gavin Ray <ra...@gmail.com>]

I had never heard of SPARQL, it appears to be an RDF/triplestore language.
This is really great -- it means that you can now execute graph-based
queries against Calcite's adapters!

From brief googling, it seems there are implementations of SPARQL that run
on top of major DB's like Postgres/MySQL.
So this could be used with the JDBC adapter then I assume?

> "I have also planned to share some notes on the process of creating an
adapter from scratch, since there are
many moving parts (and I may have missed or misunderstood several)."

This would be great, you appear to have a better understanding of the
pieces of Calcite and have implemented a lot more custom behavior.
Would love to read a walkthrough/explainer or guide and it would likely be
invaluable to others.

On Sun, Jan 23, 2022 at 5:45 PM Nicola Vitucci <ni...@gmail.com>
wrote:

> Hi all,
>
> I have been dabbling with Calcite for a while out of interest and curiosity
> and, in order to become more familiar with the code and the overall
> architecture, I've tried implementing an adapter [1] to run SQL queries
> over SPARQL endpoints. As Gavin has done, I have also planned to share some
> notes on the process of creating an adapter from scratch, since there are
> many moving parts (and I may have missed or misunderstood several).
>
> The project is at an early stage and can be improved in many ways, but, as
> I am still learning about Calcite itself, I thought it would make sense to
> share the work in progress with the community and ask for some feedback
> (plus, it's easier to ask for help when there is something concrete
> already). The rules are still relatively basic (e.g. neither joins nor
> aggregations are pushed down to SPARQL as of now); in the meantime, I am
> trying to come up with as many tests as I can to make sure that everything
> works (and keeps working) as expected.
>
> On the "usefulness" side, it is true that trying to force a table view on
> graph data is always going to be a challenge, and there are many approaches
> to it - but I think this could be a good starting point for use cases such
> as data exploration, analytics, etc.
>
> Thank you all for the great work on Calcite, and please do share your
> thoughts and comments!
>
> Nicola
>
> [1] https://github.com/nvitucci/calcite-sparql
>