SHACL & TDB

[Adrian Gschwend <ml...@netlabs.org>]

Hi group,

I recently started playing with SHACL validation in Jena, worked well
with the cli for the few things I tested, thanks for the work!

What I don't really get is where SHACL support in Jena is heading. From
what I understand in the documentation this is currently only usable in
in-memory scenarios. According to the doc the same is true for using
SHACL in Fuseki.

I guess that this will work up to a certain size of the graph. Is there
a plan/scenario where I could do the same on TDB indexes/backend?

The use-case I'm thinking about would be validating observations in RDF
data cubes with SHACL. This could potentially be millions of
observations so I guess there is a certain limit of what I could
validate in the in-memory setup.

regards

Adrian

Re: SHACL & TDB

[Adrian Gschwend <ml...@netlabs.org>]

On 27.09.21 18:05, Andy Seaborne wrote:

Hi Andy,

> Other than the service name - where does it say in-memory only? The> validation API works on any graphs. The CLI tools do work in-memory.
Thanks for the clarification. I then over-interpreted the documentation
and draw that wrong conclusion. Good to hear!

> How many millions?
Right now still in the 1 digit range

> I've done some work on SHACL to split work out into inline streaming
> (e.g. anything checking objects which is often the bulk of shapes work +
> domain and range like things), entity wide items until later (e.g. track
> what might be affected cardinality) and things that are unknown (SPARQL)
> until the end. But not in the codebase.

yeah some things can work in streaming in SHACL, others can't.

> ShEx with its "shapes map" has a different approach - the shapes map can
> direct the validation so the app can break it up into chunks or
> calculate from changes. Might be interesting to use that concept sometime.

ah never had a look at that.

thanks

regards

Adrian

Re: SHACL & TDB

[Andy Seaborne <an...@apache.org>]

On 27/09/2021 16:28, Adrian Gschwend wrote:
> Hi group,
> 
> I recently started playing with SHACL validation in Jena, worked well
> with the cli for the few things I tested, thanks for the work!
> 
> What I don't really get is where SHACL support in Jena is heading. From
> what I understand in the documentation this is currently only usable in
> in-memory scenarios.

Other than the service name - where does it say in-memory only? The 
validation API works on any graphs. The CLI tools do work in-memory.

The Fuseki SHACL service works on any database.

How fast that is depends on the many factors.

> According to the doc the same is true for using
> SHACL in Fuseki.
> 
> I guess that this will work up to a certain size of the graph. Is there
> a plan/scenario where I could do the same on TDB indexes/backend?
> 
> The use-case I'm thinking about would be validating observations in RDF
> data cubes with SHACL. This could potentially be millions of
> observations so I guess there is a certain limit of what I could
> validate in the in-memory setup.

How many millions?

I've done some work on SHACL to split work out into inline streaming 
(e.g. anything checking objects which is often the bulk of shapes work + 
domain and range like things), entity wide items until later (e.g. track 
what might be affected cardinality) and things that are unknown (SPARQL) 
until the end. But not in the codebase.

ShEx with its "shapes map" has a different approach - the shapes map can 
direct the validation so the app can break it up into chunks or 
calculate from changes. Might be interesting to use that concept sometime.

     Andy

> 
> regards
> 
> Adrian
> 

Re: SHACL & TDB

[Florian Kleedorfer <fl...@austria.fm>]

Hi,

Cannot help with the question as far as jena is concerned. Fwiw, there is (limited) support for SHACL in RDF4J Sails, which is also supported by GraphDB - as you may know.





Am 27. September 2021 11:28:50 GMT-04:00 schrieb Adrian Gschwend <ml...@netlabs.org>:
>Hi group,
>
>I recently started playing with SHACL validation in Jena, worked well
>with the cli for the few things I tested, thanks for the work!
>
>What I don't really get is where SHACL support in Jena is heading. From
>what I understand in the documentation this is currently only usable in
>in-memory scenarios. According to the doc the same is true for using
>SHACL in Fuseki.
>
>I guess that this will work up to a certain size of the graph. Is there
>a plan/scenario where I could do the same on TDB indexes/backend?
>
>The use-case I'm thinking about would be validating observations in RDF
>data cubes with SHACL. This could potentially be millions of
>observations so I guess there is a certain limit of what I could
>validate in the in-memory setup.
>
>regards
>
>Adrian

-- 
Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.