You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Martynas Jusevičius <ma...@atomgraph.com> on 2020/05/19 12:03:50 UTC
SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Hi,
After upgrading our code as well as SPINRDF from 3.0.1 to
3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
[0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
With 3.0.1, no such internal validations are produced.
The models being validated use a basic RDFS inference (subclassing etc.)
I have tracked the source of the difference to query execution in
SPINConstraints.runQueryOnClass():
https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
Since SPINRDF code hasn't changed, only the Jena versions, could there
be differences in query execution? The specific query that produces
different results is this:
CONSTRUCT
{
_:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
<http://spinrdf.org/spin#ConstraintViolation> .
_:c0 <http://spinrdf.org/spin#violationRoot> ?this .
_:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
}
WHERE
{ ?this a ?TYPE_CLASS
{ FILTER ( bound(?minCount) && (
<http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
) }
UNION
{ FILTER ( bound(?maxCount) && (
<http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
) }
UNION
{ FILTER bound(?valueType)
?this ?predicate ?value
FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
}
}
The query string and initial bindings are the same with both versions,
for example:
( ?predicate, spin:body )( ?comment, "the body of the Template" )(
?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
Are there some obvious suspects here or do I need to create a
reproducible example?
Thanks,
Martynas
Re: SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Posted by Andy Seaborne <an...@apache.org>.
On 26/05/2020 12:34, Martynas Jusevičius wrote:
> To cite my own tweet from last year: "What I don't like about SHACL is
> that it mixes instance constructors with constraints. Which is nicely
> separated in SPIN. These are orthogonal matters IMO."
> Isn't that the case?
>
> And what is the maturity of the SHACL support? Does Jena have it built-in now?
Yes
>
>
> Re. VALUES, yes I guess there will be more complexity, but shouldn't
> the performance improve compared to the current SPINRDF implementation
> which loops all instances of the constrained class and executes a
> query for each of them?
> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L604
>
> On Tue, May 26, 2020 at 1:07 PM Andy Seaborne <an...@apache.org> wrote:
>>
>>
>>
>> On 24/05/2020 10:45, Martynas Jusevičius wrote:
>>> Andy,
>>>
>>> but Jena itself supports initial bindings in QueryExecutionFactory?
>>> https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/QueryExecutionFactory.html#create-java.lang.String-org.apache.jena.query.QuerySolution-
>>> Should that method be avoided or deprecated?
>>
>> It depends on the replacement. maybe the API is right even if the corner
>> cases of the effect changes.
>>
>>
>> Related https://afs.github.io/substitute.html
>> but not the way forward if we are to cover remote as well as local queries.
>>
>>> Any example of how that could be replaced with QueryTransformOps?
>>>
>>> What I'm considering is taking only the things we use (constraints and
>>> constructors) and implementing them using plain SPARQL, ripping out
>>> stuff like SPIN functions and rules.
>>
>> And SHACL?
>>
>> Because then micro-checks are fast (in Java) e.g. minCount, maxCount.
>> The SPARQL overhead would be significant andthe suggestion of VALUE
>> batching is added complexity.
>>
>>>
>>> I guess I would need to start with a simple test suite.
>>>
>>> On Sun, May 24, 2020 at 12:08 AM Andy Seaborne <an...@apache.org> wrote:
>>>>
>>>> Martynas,
>>>>
>>>> Two things:
>>>>
>>>> 1:: IIRC it is the queries that back various features that need to be
>>>> more stable. Depending on the internal details of ARQ isn't a good place
>>>> to be. Note that SpinDRF has it's own function and property function
>>>> evaluation subsystem with slightly different semantics.
>>>>
>>>> This is the problem you are encountering.
>>>>
>>>> 2:: The effect is initial binding isn't well defined.
>>>> Syntax rewriting is going to be more stable (and works remotely which
>>>> initial bindings can't).
>>>>
>>>> Andy
>>>>
>>>> On 22/05/2020 19:55, Martynas Jusevičius wrote:
>>>>> Andy,
>>>>>
>>>>> which part needs to be replaced with syntax rewriting -- SPIN
>>>>> functions or the initial bindings?
>>>>>
>>>>> Do you think it's possible to rewrite SPINRDF in a more SPARQL
>>>>> compliant and less ARQ-dependent way?
>>>>>
>>>>> On Thu, May 21, 2020 at 10:54 AM Andy Seaborne <an...@apache.org> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 20/05/2020 22:11, Martynas Jusevičius wrote:
>>>>>>> https://github.com/spinrdf/spinrdf/issues/22
>>>>>>>
>>>>>>> On Wed, May 20, 2020 at 10:57 PM Martynas Jusevičius
>>>>>>> <ma...@atomgraph.com> wrote:
>>>>>>>>
>>>>>>>> Andy,
>>>>>>>>
>>>>>>>> I was able to isolate a standalone example:
>>>>>>>> https://github.com/namedgraph/spinrdf-test
>>>>>>>>
>>>>>>>> With 3.0.1 it succeeds. Change Jena version to 3.16.0-SNAPSHOT by
>>>>>>>> uncommenting the other <version> in pom.xml and it will fail:
>>>>>>>> https://github.com/namedgraph/spinrdf-test/blob/master/pom.xml#L28
>>>>>>>>
>>>>>>>> I don't understand how or where spl:objectCount() and spl:instanceOf()
>>>>>>>> are executed, but i think it demonstrates clearly that some behaviour
>>>>>>>> has changed between those versions, even though SpinRDF code hasn't.
>>>>>>
>>>>>> Many SpinRDF functions are written with implicit assumptions about how
>>>>>> ARQ - not SPARQL - executes. They don't work when the optimizer is off,
>>>>>> reconfigured, or if the reference query engine is used.
>>>>>>
>>>>>> In addition, the external injection of initial bindings
>>>>>>
>>>>>> (Long term - that should be replaced with syntax rewriting of the query
>>>>>> - see QueryTransformOps)
>>>>>>
>>>>>> The result is that each version of SpinRDF needs to be tuned to the
>>>>>> version of Jena it uses.
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>>>>
>>>>>>>> Will post on https://github.com/spinrdf/spinrdf/issues as well.
>>>>>>>>
>>>>>>>>
>>>>>>>> Martynas
>>>>>>>>
>>>>>>>> On Wed, May 20, 2020 at 12:37 PM Martynas Jusevičius
>>>>>>>> <ma...@atomgraph.com> wrote:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I've narrowed down the differences to the counts produced by
>>>>>>>>> spl:objectCount() SPIN function.
>>>>>>>>>
>>>>>>>>> SELECT *
>>>>>>>>> WHERE
>>>>>>>>> { ?this a ?TYPE_CLASS
>>>>>>>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
>>>>>>>>> AS ?objCount)
>>>>>>>>> FILTER ( bound(?minCount) && ( ?objCount < ?minCount ) )
>>>>>>>>> }
>>>>>>>>> UNION
>>>>>>>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
>>>>>>>>> AS ?objCount)
>>>>>>>>> FILTER ( bound(?maxCount) && ( ?objCount > ?maxCount ) )
>>>>>>>>> }
>>>>>>>>> UNION
>>>>>>>>> { FILTER bound(?valueType)
>>>>>>>>> ?this ?predicate ?value
>>>>>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Bindings: ( ?predicate, spin:body )( ?comment, "the body of the
>>>>>>>>> Template" )( ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount,
>>>>>>>>> 1 )
>>>>>>>>>
>>>>>>>>> 3.15.0-SNAPSHOT
>>>>>>>>>
>>>>>>>>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>>>>>>>> | this
>>>>>>>>> | TYPE_CLASS |
>>>>>>>>> objCount | predicate | value |
>>>>>>>>> =================================================================================================================================================================================================
>>>>>>>>> | <http://spinrdf.org/spin#Templates>
>>>>>>>>> | <http://spinrdf.org/spin#Template> |
>>>>>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>>>>>> | <http://spinrdf.org/spin#ConstructTemplates>
>>>>>>>>> | <http://spinrdf.org/spin#Template> |
>>>>>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>>>>>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectUpdateTemplate>
>>>>>>>>> | <http://spinrdf.org/spin#Template> | 2 |
>>>>>>>>> <http://spinrdf.org/spin#body> | |
>>>>>>>>> | <http://spinrdf.org/spin#AskTemplates>
>>>>>>>>> | <http://spinrdf.org/spin#Template> |
>>>>>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>>>>>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectQueryTemplate>
>>>>>>>>> | <http://spinrdf.org/spin#Template> | 2 |
>>>>>>>>> <http://spinrdf.org/spin#body> | |
>>>>>>>>> | <http://spinrdf.org/spin#UpdateTemplates>
>>>>>>>>> | <http://spinrdf.org/spin#Template> |
>>>>>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>>>>>> | <http://spinrdf.org/spl#Argument>
>>>>>>>>> | <http://spinrdf.org/spin#Template> |
>>>>>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>>>>>> | <http://spinrdf.org/spin#SelectTemplates>
>>>>>>>>> | <http://spinrdf.org/spin#Template> |
>>>>>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>>>>>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> 3.0.1
>>>>>>>>>
>>>>>>>>> ----------------------------------------------------
>>>>>>>>> | this | TYPE_CLASS | objCount | predicate | value |
>>>>>>>>> ====================================================
>>>>>>>>> ----------------------------------------------------
>>>>>>>>>
>>>>>>>>> I'll try to create an example now.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, May 19, 2020 at 6:14 PM Andy Seaborne <an...@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>> That's a long jump.
>>>>>>>>>>
>>>>>>>>>> (and it's a SpinRDF question)
>>>>>>>>>>
>>>>>>>>>> >> Are there some obvious suspects here or do I need to create a
>>>>>>>>>> >> reproducible example?
>>>>>>>>>>
>>>>>>>>>> reproducible example (noting that the query has custom functions which
>>>>>>>>>> have names that suggest they are not proper "functions" (i.e. their
>>>>>>>>>> return is not a function of their arguments alone and they access the
>>>>>>>>>> data graph .. which might be a query).
>>>>>>>>>>
>>>>>>>>>> I can't see why it might now have answers when it didn't before.
>>>>>>>>>>
>>>>>>>>>> Andy
>>>>>>>>>>
>>>>>>>>>> On 19/05/2020 13:04, Martynas Jusevičius wrote:
>>>>>>>>>>> With 3.0.1, no such internal *violations* are produced.
>>>>>>>>>>>
>>>>>>>>>>> On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius
>>>>>>>>>>> <ma...@atomgraph.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> After upgrading our code as well as SPINRDF from 3.0.1 to
>>>>>>>>>>>> 3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
>>>>>>>>>>>>
>>>>>>>>>>>> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
>>>>>>>>>>>> are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
>>>>>>>>>>>> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
>>>>>>>>>>>> With 3.0.1, no such internal validations are produced.
>>>>>>>>>>>>
>>>>>>>>>>>> The models being validated use a basic RDFS inference (subclassing etc.)
>>>>>>>>>>>>
>>>>>>>>>>>> I have tracked the source of the difference to query execution in
>>>>>>>>>>>> SPINConstraints.runQueryOnClass():
>>>>>>>>>>>> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
>>>>>>>>>>>>
>>>>>>>>>>>> Since SPINRDF code hasn't changed, only the Jena versions, could there
>>>>>>>>>>>> be differences in query execution? The specific query that produces
>>>>>>>>>>>> different results is this:
>>>>>>>>>>>>
>>>>>>>>>>>> CONSTRUCT
>>>>>>>>>>>> {
>>>>>>>>>>>> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
>>>>>>>>>>>> <http://spinrdf.org/spin#ConstraintViolation> .
>>>>>>>>>>>> _:c0 <http://spinrdf.org/spin#violationRoot> ?this .
>>>>>>>>>>>> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
>>>>>>>>>>>> }
>>>>>>>>>>>> WHERE
>>>>>>>>>>>> { ?this a ?TYPE_CLASS
>>>>>>>>>>>> { FILTER ( bound(?minCount) && (
>>>>>>>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
>>>>>>>>>>>> ) }
>>>>>>>>>>>> UNION
>>>>>>>>>>>> { FILTER ( bound(?maxCount) && (
>>>>>>>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
>>>>>>>>>>>> ) }
>>>>>>>>>>>> UNION
>>>>>>>>>>>> { FILTER bound(?valueType)
>>>>>>>>>>>> ?this ?predicate ?value
>>>>>>>>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
>>>>>>>>>>>> }
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> The query string and initial bindings are the same with both versions,
>>>>>>>>>>>> for example:
>>>>>>>>>>>>
>>>>>>>>>>>> ( ?predicate, spin:body )( ?comment, "the body of the Template" )(
>>>>>>>>>>>> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
>>>>>>>>>>>>
>>>>>>>>>>>> Are there some obvious suspects here or do I need to create a
>>>>>>>>>>>> reproducible example?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>
>>>>>>>>>>>> Martynas
Re: SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Posted by Martynas Jusevičius <ma...@atomgraph.com>.
To cite my own tweet from last year: "What I don't like about SHACL is
that it mixes instance constructors with constraints. Which is nicely
separated in SPIN. These are orthogonal matters IMO."
Isn't that the case?
And what is the maturity of the SHACL support? Does Jena have it built-in now?
Re. VALUES, yes I guess there will be more complexity, but shouldn't
the performance improve compared to the current SPINRDF implementation
which loops all instances of the constrained class and executes a
query for each of them?
https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L604
On Tue, May 26, 2020 at 1:07 PM Andy Seaborne <an...@apache.org> wrote:
>
>
>
> On 24/05/2020 10:45, Martynas Jusevičius wrote:
> > Andy,
> >
> > but Jena itself supports initial bindings in QueryExecutionFactory?
> > https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/QueryExecutionFactory.html#create-java.lang.String-org.apache.jena.query.QuerySolution-
> > Should that method be avoided or deprecated?
>
> It depends on the replacement. maybe the API is right even if the corner
> cases of the effect changes.
>
>
> Related https://afs.github.io/substitute.html
> but not the way forward if we are to cover remote as well as local queries.
>
> > Any example of how that could be replaced with QueryTransformOps?
> >
> > What I'm considering is taking only the things we use (constraints and
> > constructors) and implementing them using plain SPARQL, ripping out
> > stuff like SPIN functions and rules.
>
> And SHACL?
>
> Because then micro-checks are fast (in Java) e.g. minCount, maxCount.
> The SPARQL overhead would be significant andthe suggestion of VALUE
> batching is added complexity.
>
> >
> > I guess I would need to start with a simple test suite.
> >
> > On Sun, May 24, 2020 at 12:08 AM Andy Seaborne <an...@apache.org> wrote:
> >>
> >> Martynas,
> >>
> >> Two things:
> >>
> >> 1:: IIRC it is the queries that back various features that need to be
> >> more stable. Depending on the internal details of ARQ isn't a good place
> >> to be. Note that SpinDRF has it's own function and property function
> >> evaluation subsystem with slightly different semantics.
> >>
> >> This is the problem you are encountering.
> >>
> >> 2:: The effect is initial binding isn't well defined.
> >> Syntax rewriting is going to be more stable (and works remotely which
> >> initial bindings can't).
> >>
> >> Andy
> >>
> >> On 22/05/2020 19:55, Martynas Jusevičius wrote:
> >>> Andy,
> >>>
> >>> which part needs to be replaced with syntax rewriting -- SPIN
> >>> functions or the initial bindings?
> >>>
> >>> Do you think it's possible to rewrite SPINRDF in a more SPARQL
> >>> compliant and less ARQ-dependent way?
> >>>
> >>> On Thu, May 21, 2020 at 10:54 AM Andy Seaborne <an...@apache.org> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On 20/05/2020 22:11, Martynas Jusevičius wrote:
> >>>>> https://github.com/spinrdf/spinrdf/issues/22
> >>>>>
> >>>>> On Wed, May 20, 2020 at 10:57 PM Martynas Jusevičius
> >>>>> <ma...@atomgraph.com> wrote:
> >>>>>>
> >>>>>> Andy,
> >>>>>>
> >>>>>> I was able to isolate a standalone example:
> >>>>>> https://github.com/namedgraph/spinrdf-test
> >>>>>>
> >>>>>> With 3.0.1 it succeeds. Change Jena version to 3.16.0-SNAPSHOT by
> >>>>>> uncommenting the other <version> in pom.xml and it will fail:
> >>>>>> https://github.com/namedgraph/spinrdf-test/blob/master/pom.xml#L28
> >>>>>>
> >>>>>> I don't understand how or where spl:objectCount() and spl:instanceOf()
> >>>>>> are executed, but i think it demonstrates clearly that some behaviour
> >>>>>> has changed between those versions, even though SpinRDF code hasn't.
> >>>>
> >>>> Many SpinRDF functions are written with implicit assumptions about how
> >>>> ARQ - not SPARQL - executes. They don't work when the optimizer is off,
> >>>> reconfigured, or if the reference query engine is used.
> >>>>
> >>>> In addition, the external injection of initial bindings
> >>>>
> >>>> (Long term - that should be replaced with syntax rewriting of the query
> >>>> - see QueryTransformOps)
> >>>>
> >>>> The result is that each version of SpinRDF needs to be tuned to the
> >>>> version of Jena it uses.
> >>>>
> >>>> Andy
> >>>>
> >>>>>>
> >>>>>> Will post on https://github.com/spinrdf/spinrdf/issues as well.
> >>>>>>
> >>>>>>
> >>>>>> Martynas
> >>>>>>
> >>>>>> On Wed, May 20, 2020 at 12:37 PM Martynas Jusevičius
> >>>>>> <ma...@atomgraph.com> wrote:
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I've narrowed down the differences to the counts produced by
> >>>>>>> spl:objectCount() SPIN function.
> >>>>>>>
> >>>>>>> SELECT *
> >>>>>>> WHERE
> >>>>>>> { ?this a ?TYPE_CLASS
> >>>>>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
> >>>>>>> AS ?objCount)
> >>>>>>> FILTER ( bound(?minCount) && ( ?objCount < ?minCount ) )
> >>>>>>> }
> >>>>>>> UNION
> >>>>>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
> >>>>>>> AS ?objCount)
> >>>>>>> FILTER ( bound(?maxCount) && ( ?objCount > ?maxCount ) )
> >>>>>>> }
> >>>>>>> UNION
> >>>>>>> { FILTER bound(?valueType)
> >>>>>>> ?this ?predicate ?value
> >>>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
> >>>>>>> }
> >>>>>>> }
> >>>>>>>
> >>>>>>> Bindings: ( ?predicate, spin:body )( ?comment, "the body of the
> >>>>>>> Template" )( ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount,
> >>>>>>> 1 )
> >>>>>>>
> >>>>>>> 3.15.0-SNAPSHOT
> >>>>>>>
> >>>>>>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >>>>>>> | this
> >>>>>>> | TYPE_CLASS |
> >>>>>>> objCount | predicate | value |
> >>>>>>> =================================================================================================================================================================================================
> >>>>>>> | <http://spinrdf.org/spin#Templates>
> >>>>>>> | <http://spinrdf.org/spin#Template> |
> >>>>>>> 2 | <http://spinrdf.org/spin#body> | |
> >>>>>>> | <http://spinrdf.org/spin#ConstructTemplates>
> >>>>>>> | <http://spinrdf.org/spin#Template> |
> >>>>>>> 2 | <http://spinrdf.org/spin#body> | |
> >>>>>>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectUpdateTemplate>
> >>>>>>> | <http://spinrdf.org/spin#Template> | 2 |
> >>>>>>> <http://spinrdf.org/spin#body> | |
> >>>>>>> | <http://spinrdf.org/spin#AskTemplates>
> >>>>>>> | <http://spinrdf.org/spin#Template> |
> >>>>>>> 2 | <http://spinrdf.org/spin#body> | |
> >>>>>>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectQueryTemplate>
> >>>>>>> | <http://spinrdf.org/spin#Template> | 2 |
> >>>>>>> <http://spinrdf.org/spin#body> | |
> >>>>>>> | <http://spinrdf.org/spin#UpdateTemplates>
> >>>>>>> | <http://spinrdf.org/spin#Template> |
> >>>>>>> 2 | <http://spinrdf.org/spin#body> | |
> >>>>>>> | <http://spinrdf.org/spl#Argument>
> >>>>>>> | <http://spinrdf.org/spin#Template> |
> >>>>>>> 2 | <http://spinrdf.org/spin#body> | |
> >>>>>>> | <http://spinrdf.org/spin#SelectTemplates>
> >>>>>>> | <http://spinrdf.org/spin#Template> |
> >>>>>>> 2 | <http://spinrdf.org/spin#body> | |
> >>>>>>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >>>>>>>
> >>>>>>> 3.0.1
> >>>>>>>
> >>>>>>> ----------------------------------------------------
> >>>>>>> | this | TYPE_CLASS | objCount | predicate | value |
> >>>>>>> ====================================================
> >>>>>>> ----------------------------------------------------
> >>>>>>>
> >>>>>>> I'll try to create an example now.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, May 19, 2020 at 6:14 PM Andy Seaborne <an...@apache.org> wrote:
> >>>>>>>>
> >>>>>>>> That's a long jump.
> >>>>>>>>
> >>>>>>>> (and it's a SpinRDF question)
> >>>>>>>>
> >>>>>>>> >> Are there some obvious suspects here or do I need to create a
> >>>>>>>> >> reproducible example?
> >>>>>>>>
> >>>>>>>> reproducible example (noting that the query has custom functions which
> >>>>>>>> have names that suggest they are not proper "functions" (i.e. their
> >>>>>>>> return is not a function of their arguments alone and they access the
> >>>>>>>> data graph .. which might be a query).
> >>>>>>>>
> >>>>>>>> I can't see why it might now have answers when it didn't before.
> >>>>>>>>
> >>>>>>>> Andy
> >>>>>>>>
> >>>>>>>> On 19/05/2020 13:04, Martynas Jusevičius wrote:
> >>>>>>>>> With 3.0.1, no such internal *violations* are produced.
> >>>>>>>>>
> >>>>>>>>> On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius
> >>>>>>>>> <ma...@atomgraph.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hi,
> >>>>>>>>>>
> >>>>>>>>>> After upgrading our code as well as SPINRDF from 3.0.1 to
> >>>>>>>>>> 3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
> >>>>>>>>>>
> >>>>>>>>>> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
> >>>>>>>>>> are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
> >>>>>>>>>> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
> >>>>>>>>>> With 3.0.1, no such internal validations are produced.
> >>>>>>>>>>
> >>>>>>>>>> The models being validated use a basic RDFS inference (subclassing etc.)
> >>>>>>>>>>
> >>>>>>>>>> I have tracked the source of the difference to query execution in
> >>>>>>>>>> SPINConstraints.runQueryOnClass():
> >>>>>>>>>> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
> >>>>>>>>>>
> >>>>>>>>>> Since SPINRDF code hasn't changed, only the Jena versions, could there
> >>>>>>>>>> be differences in query execution? The specific query that produces
> >>>>>>>>>> different results is this:
> >>>>>>>>>>
> >>>>>>>>>> CONSTRUCT
> >>>>>>>>>> {
> >>>>>>>>>> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
> >>>>>>>>>> <http://spinrdf.org/spin#ConstraintViolation> .
> >>>>>>>>>> _:c0 <http://spinrdf.org/spin#violationRoot> ?this .
> >>>>>>>>>> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
> >>>>>>>>>> }
> >>>>>>>>>> WHERE
> >>>>>>>>>> { ?this a ?TYPE_CLASS
> >>>>>>>>>> { FILTER ( bound(?minCount) && (
> >>>>>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
> >>>>>>>>>> ) }
> >>>>>>>>>> UNION
> >>>>>>>>>> { FILTER ( bound(?maxCount) && (
> >>>>>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
> >>>>>>>>>> ) }
> >>>>>>>>>> UNION
> >>>>>>>>>> { FILTER bound(?valueType)
> >>>>>>>>>> ?this ?predicate ?value
> >>>>>>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
> >>>>>>>>>> }
> >>>>>>>>>> }
> >>>>>>>>>>
> >>>>>>>>>> The query string and initial bindings are the same with both versions,
> >>>>>>>>>> for example:
> >>>>>>>>>>
> >>>>>>>>>> ( ?predicate, spin:body )( ?comment, "the body of the Template" )(
> >>>>>>>>>> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
> >>>>>>>>>>
> >>>>>>>>>> Are there some obvious suspects here or do I need to create a
> >>>>>>>>>> reproducible example?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>>
> >>>>>>>>>> Martynas
Re: SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Posted by Andy Seaborne <an...@apache.org>.
On 24/05/2020 10:45, Martynas Jusevičius wrote:
> Andy,
>
> but Jena itself supports initial bindings in QueryExecutionFactory?
> https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/QueryExecutionFactory.html#create-java.lang.String-org.apache.jena.query.QuerySolution-
> Should that method be avoided or deprecated?
It depends on the replacement. maybe the API is right even if the corner
cases of the effect changes.
Related https://afs.github.io/substitute.html
but not the way forward if we are to cover remote as well as local queries.
> Any example of how that could be replaced with QueryTransformOps?
>
> What I'm considering is taking only the things we use (constraints and
> constructors) and implementing them using plain SPARQL, ripping out
> stuff like SPIN functions and rules.
And SHACL?
Because then micro-checks are fast (in Java) e.g. minCount, maxCount.
The SPARQL overhead would be significant andthe suggestion of VALUE
batching is added complexity.
>
> I guess I would need to start with a simple test suite.
>
> On Sun, May 24, 2020 at 12:08 AM Andy Seaborne <an...@apache.org> wrote:
>>
>> Martynas,
>>
>> Two things:
>>
>> 1:: IIRC it is the queries that back various features that need to be
>> more stable. Depending on the internal details of ARQ isn't a good place
>> to be. Note that SpinDRF has it's own function and property function
>> evaluation subsystem with slightly different semantics.
>>
>> This is the problem you are encountering.
>>
>> 2:: The effect is initial binding isn't well defined.
>> Syntax rewriting is going to be more stable (and works remotely which
>> initial bindings can't).
>>
>> Andy
>>
>> On 22/05/2020 19:55, Martynas Jusevičius wrote:
>>> Andy,
>>>
>>> which part needs to be replaced with syntax rewriting -- SPIN
>>> functions or the initial bindings?
>>>
>>> Do you think it's possible to rewrite SPINRDF in a more SPARQL
>>> compliant and less ARQ-dependent way?
>>>
>>> On Thu, May 21, 2020 at 10:54 AM Andy Seaborne <an...@apache.org> wrote:
>>>>
>>>>
>>>>
>>>> On 20/05/2020 22:11, Martynas Jusevičius wrote:
>>>>> https://github.com/spinrdf/spinrdf/issues/22
>>>>>
>>>>> On Wed, May 20, 2020 at 10:57 PM Martynas Jusevičius
>>>>> <ma...@atomgraph.com> wrote:
>>>>>>
>>>>>> Andy,
>>>>>>
>>>>>> I was able to isolate a standalone example:
>>>>>> https://github.com/namedgraph/spinrdf-test
>>>>>>
>>>>>> With 3.0.1 it succeeds. Change Jena version to 3.16.0-SNAPSHOT by
>>>>>> uncommenting the other <version> in pom.xml and it will fail:
>>>>>> https://github.com/namedgraph/spinrdf-test/blob/master/pom.xml#L28
>>>>>>
>>>>>> I don't understand how or where spl:objectCount() and spl:instanceOf()
>>>>>> are executed, but i think it demonstrates clearly that some behaviour
>>>>>> has changed between those versions, even though SpinRDF code hasn't.
>>>>
>>>> Many SpinRDF functions are written with implicit assumptions about how
>>>> ARQ - not SPARQL - executes. They don't work when the optimizer is off,
>>>> reconfigured, or if the reference query engine is used.
>>>>
>>>> In addition, the external injection of initial bindings
>>>>
>>>> (Long term - that should be replaced with syntax rewriting of the query
>>>> - see QueryTransformOps)
>>>>
>>>> The result is that each version of SpinRDF needs to be tuned to the
>>>> version of Jena it uses.
>>>>
>>>> Andy
>>>>
>>>>>>
>>>>>> Will post on https://github.com/spinrdf/spinrdf/issues as well.
>>>>>>
>>>>>>
>>>>>> Martynas
>>>>>>
>>>>>> On Wed, May 20, 2020 at 12:37 PM Martynas Jusevičius
>>>>>> <ma...@atomgraph.com> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I've narrowed down the differences to the counts produced by
>>>>>>> spl:objectCount() SPIN function.
>>>>>>>
>>>>>>> SELECT *
>>>>>>> WHERE
>>>>>>> { ?this a ?TYPE_CLASS
>>>>>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
>>>>>>> AS ?objCount)
>>>>>>> FILTER ( bound(?minCount) && ( ?objCount < ?minCount ) )
>>>>>>> }
>>>>>>> UNION
>>>>>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
>>>>>>> AS ?objCount)
>>>>>>> FILTER ( bound(?maxCount) && ( ?objCount > ?maxCount ) )
>>>>>>> }
>>>>>>> UNION
>>>>>>> { FILTER bound(?valueType)
>>>>>>> ?this ?predicate ?value
>>>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> Bindings: ( ?predicate, spin:body )( ?comment, "the body of the
>>>>>>> Template" )( ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount,
>>>>>>> 1 )
>>>>>>>
>>>>>>> 3.15.0-SNAPSHOT
>>>>>>>
>>>>>>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>>>>>> | this
>>>>>>> | TYPE_CLASS |
>>>>>>> objCount | predicate | value |
>>>>>>> =================================================================================================================================================================================================
>>>>>>> | <http://spinrdf.org/spin#Templates>
>>>>>>> | <http://spinrdf.org/spin#Template> |
>>>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>>>> | <http://spinrdf.org/spin#ConstructTemplates>
>>>>>>> | <http://spinrdf.org/spin#Template> |
>>>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>>>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectUpdateTemplate>
>>>>>>> | <http://spinrdf.org/spin#Template> | 2 |
>>>>>>> <http://spinrdf.org/spin#body> | |
>>>>>>> | <http://spinrdf.org/spin#AskTemplates>
>>>>>>> | <http://spinrdf.org/spin#Template> |
>>>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>>>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectQueryTemplate>
>>>>>>> | <http://spinrdf.org/spin#Template> | 2 |
>>>>>>> <http://spinrdf.org/spin#body> | |
>>>>>>> | <http://spinrdf.org/spin#UpdateTemplates>
>>>>>>> | <http://spinrdf.org/spin#Template> |
>>>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>>>> | <http://spinrdf.org/spl#Argument>
>>>>>>> | <http://spinrdf.org/spin#Template> |
>>>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>>>> | <http://spinrdf.org/spin#SelectTemplates>
>>>>>>> | <http://spinrdf.org/spin#Template> |
>>>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>>>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>>>>>>
>>>>>>> 3.0.1
>>>>>>>
>>>>>>> ----------------------------------------------------
>>>>>>> | this | TYPE_CLASS | objCount | predicate | value |
>>>>>>> ====================================================
>>>>>>> ----------------------------------------------------
>>>>>>>
>>>>>>> I'll try to create an example now.
>>>>>>>
>>>>>>>
>>>>>>> On Tue, May 19, 2020 at 6:14 PM Andy Seaborne <an...@apache.org> wrote:
>>>>>>>>
>>>>>>>> That's a long jump.
>>>>>>>>
>>>>>>>> (and it's a SpinRDF question)
>>>>>>>>
>>>>>>>> >> Are there some obvious suspects here or do I need to create a
>>>>>>>> >> reproducible example?
>>>>>>>>
>>>>>>>> reproducible example (noting that the query has custom functions which
>>>>>>>> have names that suggest they are not proper "functions" (i.e. their
>>>>>>>> return is not a function of their arguments alone and they access the
>>>>>>>> data graph .. which might be a query).
>>>>>>>>
>>>>>>>> I can't see why it might now have answers when it didn't before.
>>>>>>>>
>>>>>>>> Andy
>>>>>>>>
>>>>>>>> On 19/05/2020 13:04, Martynas Jusevičius wrote:
>>>>>>>>> With 3.0.1, no such internal *violations* are produced.
>>>>>>>>>
>>>>>>>>> On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius
>>>>>>>>> <ma...@atomgraph.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> After upgrading our code as well as SPINRDF from 3.0.1 to
>>>>>>>>>> 3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
>>>>>>>>>>
>>>>>>>>>> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
>>>>>>>>>> are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
>>>>>>>>>> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
>>>>>>>>>> With 3.0.1, no such internal validations are produced.
>>>>>>>>>>
>>>>>>>>>> The models being validated use a basic RDFS inference (subclassing etc.)
>>>>>>>>>>
>>>>>>>>>> I have tracked the source of the difference to query execution in
>>>>>>>>>> SPINConstraints.runQueryOnClass():
>>>>>>>>>> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
>>>>>>>>>>
>>>>>>>>>> Since SPINRDF code hasn't changed, only the Jena versions, could there
>>>>>>>>>> be differences in query execution? The specific query that produces
>>>>>>>>>> different results is this:
>>>>>>>>>>
>>>>>>>>>> CONSTRUCT
>>>>>>>>>> {
>>>>>>>>>> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
>>>>>>>>>> <http://spinrdf.org/spin#ConstraintViolation> .
>>>>>>>>>> _:c0 <http://spinrdf.org/spin#violationRoot> ?this .
>>>>>>>>>> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
>>>>>>>>>> }
>>>>>>>>>> WHERE
>>>>>>>>>> { ?this a ?TYPE_CLASS
>>>>>>>>>> { FILTER ( bound(?minCount) && (
>>>>>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
>>>>>>>>>> ) }
>>>>>>>>>> UNION
>>>>>>>>>> { FILTER ( bound(?maxCount) && (
>>>>>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
>>>>>>>>>> ) }
>>>>>>>>>> UNION
>>>>>>>>>> { FILTER bound(?valueType)
>>>>>>>>>> ?this ?predicate ?value
>>>>>>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
>>>>>>>>>> }
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> The query string and initial bindings are the same with both versions,
>>>>>>>>>> for example:
>>>>>>>>>>
>>>>>>>>>> ( ?predicate, spin:body )( ?comment, "the body of the Template" )(
>>>>>>>>>> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
>>>>>>>>>>
>>>>>>>>>> Are there some obvious suspects here or do I need to create a
>>>>>>>>>> reproducible example?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Martynas
Re: SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Posted by Martynas Jusevičius <ma...@atomgraph.com>.
Andy,
but Jena itself supports initial bindings in QueryExecutionFactory?
https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/QueryExecutionFactory.html#create-java.lang.String-org.apache.jena.query.QuerySolution-
Should that method be avoided or deprecated?
Any example of how that could be replaced with QueryTransformOps?
What I'm considering is taking only the things we use (constraints and
constructors) and implementing them using plain SPARQL, ripping out
stuff like SPIN functions and rules.
I guess I would need to start with a simple test suite.
On Sun, May 24, 2020 at 12:08 AM Andy Seaborne <an...@apache.org> wrote:
>
> Martynas,
>
> Two things:
>
> 1:: IIRC it is the queries that back various features that need to be
> more stable. Depending on the internal details of ARQ isn't a good place
> to be. Note that SpinDRF has it's own function and property function
> evaluation subsystem with slightly different semantics.
>
> This is the problem you are encountering.
>
> 2:: The effect is initial binding isn't well defined.
> Syntax rewriting is going to be more stable (and works remotely which
> initial bindings can't).
>
> Andy
>
> On 22/05/2020 19:55, Martynas Jusevičius wrote:
> > Andy,
> >
> > which part needs to be replaced with syntax rewriting -- SPIN
> > functions or the initial bindings?
> >
> > Do you think it's possible to rewrite SPINRDF in a more SPARQL
> > compliant and less ARQ-dependent way?
> >
> > On Thu, May 21, 2020 at 10:54 AM Andy Seaborne <an...@apache.org> wrote:
> >>
> >>
> >>
> >> On 20/05/2020 22:11, Martynas Jusevičius wrote:
> >>> https://github.com/spinrdf/spinrdf/issues/22
> >>>
> >>> On Wed, May 20, 2020 at 10:57 PM Martynas Jusevičius
> >>> <ma...@atomgraph.com> wrote:
> >>>>
> >>>> Andy,
> >>>>
> >>>> I was able to isolate a standalone example:
> >>>> https://github.com/namedgraph/spinrdf-test
> >>>>
> >>>> With 3.0.1 it succeeds. Change Jena version to 3.16.0-SNAPSHOT by
> >>>> uncommenting the other <version> in pom.xml and it will fail:
> >>>> https://github.com/namedgraph/spinrdf-test/blob/master/pom.xml#L28
> >>>>
> >>>> I don't understand how or where spl:objectCount() and spl:instanceOf()
> >>>> are executed, but i think it demonstrates clearly that some behaviour
> >>>> has changed between those versions, even though SpinRDF code hasn't.
> >>
> >> Many SpinRDF functions are written with implicit assumptions about how
> >> ARQ - not SPARQL - executes. They don't work when the optimizer is off,
> >> reconfigured, or if the reference query engine is used.
> >>
> >> In addition, the external injection of initial bindings
> >>
> >> (Long term - that should be replaced with syntax rewriting of the query
> >> - see QueryTransformOps)
> >>
> >> The result is that each version of SpinRDF needs to be tuned to the
> >> version of Jena it uses.
> >>
> >> Andy
> >>
> >>>>
> >>>> Will post on https://github.com/spinrdf/spinrdf/issues as well.
> >>>>
> >>>>
> >>>> Martynas
> >>>>
> >>>> On Wed, May 20, 2020 at 12:37 PM Martynas Jusevičius
> >>>> <ma...@atomgraph.com> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I've narrowed down the differences to the counts produced by
> >>>>> spl:objectCount() SPIN function.
> >>>>>
> >>>>> SELECT *
> >>>>> WHERE
> >>>>> { ?this a ?TYPE_CLASS
> >>>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
> >>>>> AS ?objCount)
> >>>>> FILTER ( bound(?minCount) && ( ?objCount < ?minCount ) )
> >>>>> }
> >>>>> UNION
> >>>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
> >>>>> AS ?objCount)
> >>>>> FILTER ( bound(?maxCount) && ( ?objCount > ?maxCount ) )
> >>>>> }
> >>>>> UNION
> >>>>> { FILTER bound(?valueType)
> >>>>> ?this ?predicate ?value
> >>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
> >>>>> }
> >>>>> }
> >>>>>
> >>>>> Bindings: ( ?predicate, spin:body )( ?comment, "the body of the
> >>>>> Template" )( ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount,
> >>>>> 1 )
> >>>>>
> >>>>> 3.15.0-SNAPSHOT
> >>>>>
> >>>>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >>>>> | this
> >>>>> | TYPE_CLASS |
> >>>>> objCount | predicate | value |
> >>>>> =================================================================================================================================================================================================
> >>>>> | <http://spinrdf.org/spin#Templates>
> >>>>> | <http://spinrdf.org/spin#Template> |
> >>>>> 2 | <http://spinrdf.org/spin#body> | |
> >>>>> | <http://spinrdf.org/spin#ConstructTemplates>
> >>>>> | <http://spinrdf.org/spin#Template> |
> >>>>> 2 | <http://spinrdf.org/spin#body> | |
> >>>>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectUpdateTemplate>
> >>>>> | <http://spinrdf.org/spin#Template> | 2 |
> >>>>> <http://spinrdf.org/spin#body> | |
> >>>>> | <http://spinrdf.org/spin#AskTemplates>
> >>>>> | <http://spinrdf.org/spin#Template> |
> >>>>> 2 | <http://spinrdf.org/spin#body> | |
> >>>>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectQueryTemplate>
> >>>>> | <http://spinrdf.org/spin#Template> | 2 |
> >>>>> <http://spinrdf.org/spin#body> | |
> >>>>> | <http://spinrdf.org/spin#UpdateTemplates>
> >>>>> | <http://spinrdf.org/spin#Template> |
> >>>>> 2 | <http://spinrdf.org/spin#body> | |
> >>>>> | <http://spinrdf.org/spl#Argument>
> >>>>> | <http://spinrdf.org/spin#Template> |
> >>>>> 2 | <http://spinrdf.org/spin#body> | |
> >>>>> | <http://spinrdf.org/spin#SelectTemplates>
> >>>>> | <http://spinrdf.org/spin#Template> |
> >>>>> 2 | <http://spinrdf.org/spin#body> | |
> >>>>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >>>>>
> >>>>> 3.0.1
> >>>>>
> >>>>> ----------------------------------------------------
> >>>>> | this | TYPE_CLASS | objCount | predicate | value |
> >>>>> ====================================================
> >>>>> ----------------------------------------------------
> >>>>>
> >>>>> I'll try to create an example now.
> >>>>>
> >>>>>
> >>>>> On Tue, May 19, 2020 at 6:14 PM Andy Seaborne <an...@apache.org> wrote:
> >>>>>>
> >>>>>> That's a long jump.
> >>>>>>
> >>>>>> (and it's a SpinRDF question)
> >>>>>>
> >>>>>> >> Are there some obvious suspects here or do I need to create a
> >>>>>> >> reproducible example?
> >>>>>>
> >>>>>> reproducible example (noting that the query has custom functions which
> >>>>>> have names that suggest they are not proper "functions" (i.e. their
> >>>>>> return is not a function of their arguments alone and they access the
> >>>>>> data graph .. which might be a query).
> >>>>>>
> >>>>>> I can't see why it might now have answers when it didn't before.
> >>>>>>
> >>>>>> Andy
> >>>>>>
> >>>>>> On 19/05/2020 13:04, Martynas Jusevičius wrote:
> >>>>>>> With 3.0.1, no such internal *violations* are produced.
> >>>>>>>
> >>>>>>> On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius
> >>>>>>> <ma...@atomgraph.com> wrote:
> >>>>>>>>
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> After upgrading our code as well as SPINRDF from 3.0.1 to
> >>>>>>>> 3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
> >>>>>>>>
> >>>>>>>> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
> >>>>>>>> are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
> >>>>>>>> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
> >>>>>>>> With 3.0.1, no such internal validations are produced.
> >>>>>>>>
> >>>>>>>> The models being validated use a basic RDFS inference (subclassing etc.)
> >>>>>>>>
> >>>>>>>> I have tracked the source of the difference to query execution in
> >>>>>>>> SPINConstraints.runQueryOnClass():
> >>>>>>>> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
> >>>>>>>>
> >>>>>>>> Since SPINRDF code hasn't changed, only the Jena versions, could there
> >>>>>>>> be differences in query execution? The specific query that produces
> >>>>>>>> different results is this:
> >>>>>>>>
> >>>>>>>> CONSTRUCT
> >>>>>>>> {
> >>>>>>>> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
> >>>>>>>> <http://spinrdf.org/spin#ConstraintViolation> .
> >>>>>>>> _:c0 <http://spinrdf.org/spin#violationRoot> ?this .
> >>>>>>>> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
> >>>>>>>> }
> >>>>>>>> WHERE
> >>>>>>>> { ?this a ?TYPE_CLASS
> >>>>>>>> { FILTER ( bound(?minCount) && (
> >>>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
> >>>>>>>> ) }
> >>>>>>>> UNION
> >>>>>>>> { FILTER ( bound(?maxCount) && (
> >>>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
> >>>>>>>> ) }
> >>>>>>>> UNION
> >>>>>>>> { FILTER bound(?valueType)
> >>>>>>>> ?this ?predicate ?value
> >>>>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
> >>>>>>>> }
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>> The query string and initial bindings are the same with both versions,
> >>>>>>>> for example:
> >>>>>>>>
> >>>>>>>> ( ?predicate, spin:body )( ?comment, "the body of the Template" )(
> >>>>>>>> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
> >>>>>>>>
> >>>>>>>> Are there some obvious suspects here or do I need to create a
> >>>>>>>> reproducible example?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>>
> >>>>>>>> Martynas
Re: SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Posted by Andy Seaborne <an...@apache.org>.
Martynas,
Two things:
1:: IIRC it is the queries that back various features that need to be
more stable. Depending on the internal details of ARQ isn't a good place
to be. Note that SpinDRF has it's own function and property function
evaluation subsystem with slightly different semantics.
This is the problem you are encountering.
2:: The effect is initial binding isn't well defined.
Syntax rewriting is going to be more stable (and works remotely which
initial bindings can't).
Andy
On 22/05/2020 19:55, Martynas Jusevičius wrote:
> Andy,
>
> which part needs to be replaced with syntax rewriting -- SPIN
> functions or the initial bindings?
>
> Do you think it's possible to rewrite SPINRDF in a more SPARQL
> compliant and less ARQ-dependent way?
>
> On Thu, May 21, 2020 at 10:54 AM Andy Seaborne <an...@apache.org> wrote:
>>
>>
>>
>> On 20/05/2020 22:11, Martynas Jusevičius wrote:
>>> https://github.com/spinrdf/spinrdf/issues/22
>>>
>>> On Wed, May 20, 2020 at 10:57 PM Martynas Jusevičius
>>> <ma...@atomgraph.com> wrote:
>>>>
>>>> Andy,
>>>>
>>>> I was able to isolate a standalone example:
>>>> https://github.com/namedgraph/spinrdf-test
>>>>
>>>> With 3.0.1 it succeeds. Change Jena version to 3.16.0-SNAPSHOT by
>>>> uncommenting the other <version> in pom.xml and it will fail:
>>>> https://github.com/namedgraph/spinrdf-test/blob/master/pom.xml#L28
>>>>
>>>> I don't understand how or where spl:objectCount() and spl:instanceOf()
>>>> are executed, but i think it demonstrates clearly that some behaviour
>>>> has changed between those versions, even though SpinRDF code hasn't.
>>
>> Many SpinRDF functions are written with implicit assumptions about how
>> ARQ - not SPARQL - executes. They don't work when the optimizer is off,
>> reconfigured, or if the reference query engine is used.
>>
>> In addition, the external injection of initial bindings
>>
>> (Long term - that should be replaced with syntax rewriting of the query
>> - see QueryTransformOps)
>>
>> The result is that each version of SpinRDF needs to be tuned to the
>> version of Jena it uses.
>>
>> Andy
>>
>>>>
>>>> Will post on https://github.com/spinrdf/spinrdf/issues as well.
>>>>
>>>>
>>>> Martynas
>>>>
>>>> On Wed, May 20, 2020 at 12:37 PM Martynas Jusevičius
>>>> <ma...@atomgraph.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I've narrowed down the differences to the counts produced by
>>>>> spl:objectCount() SPIN function.
>>>>>
>>>>> SELECT *
>>>>> WHERE
>>>>> { ?this a ?TYPE_CLASS
>>>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
>>>>> AS ?objCount)
>>>>> FILTER ( bound(?minCount) && ( ?objCount < ?minCount ) )
>>>>> }
>>>>> UNION
>>>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
>>>>> AS ?objCount)
>>>>> FILTER ( bound(?maxCount) && ( ?objCount > ?maxCount ) )
>>>>> }
>>>>> UNION
>>>>> { FILTER bound(?valueType)
>>>>> ?this ?predicate ?value
>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
>>>>> }
>>>>> }
>>>>>
>>>>> Bindings: ( ?predicate, spin:body )( ?comment, "the body of the
>>>>> Template" )( ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount,
>>>>> 1 )
>>>>>
>>>>> 3.15.0-SNAPSHOT
>>>>>
>>>>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>>>> | this
>>>>> | TYPE_CLASS |
>>>>> objCount | predicate | value |
>>>>> =================================================================================================================================================================================================
>>>>> | <http://spinrdf.org/spin#Templates>
>>>>> | <http://spinrdf.org/spin#Template> |
>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>> | <http://spinrdf.org/spin#ConstructTemplates>
>>>>> | <http://spinrdf.org/spin#Template> |
>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectUpdateTemplate>
>>>>> | <http://spinrdf.org/spin#Template> | 2 |
>>>>> <http://spinrdf.org/spin#body> | |
>>>>> | <http://spinrdf.org/spin#AskTemplates>
>>>>> | <http://spinrdf.org/spin#Template> |
>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectQueryTemplate>
>>>>> | <http://spinrdf.org/spin#Template> | 2 |
>>>>> <http://spinrdf.org/spin#body> | |
>>>>> | <http://spinrdf.org/spin#UpdateTemplates>
>>>>> | <http://spinrdf.org/spin#Template> |
>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>> | <http://spinrdf.org/spl#Argument>
>>>>> | <http://spinrdf.org/spin#Template> |
>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>> | <http://spinrdf.org/spin#SelectTemplates>
>>>>> | <http://spinrdf.org/spin#Template> |
>>>>> 2 | <http://spinrdf.org/spin#body> | |
>>>>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>>>>
>>>>> 3.0.1
>>>>>
>>>>> ----------------------------------------------------
>>>>> | this | TYPE_CLASS | objCount | predicate | value |
>>>>> ====================================================
>>>>> ----------------------------------------------------
>>>>>
>>>>> I'll try to create an example now.
>>>>>
>>>>>
>>>>> On Tue, May 19, 2020 at 6:14 PM Andy Seaborne <an...@apache.org> wrote:
>>>>>>
>>>>>> That's a long jump.
>>>>>>
>>>>>> (and it's a SpinRDF question)
>>>>>>
>>>>>> >> Are there some obvious suspects here or do I need to create a
>>>>>> >> reproducible example?
>>>>>>
>>>>>> reproducible example (noting that the query has custom functions which
>>>>>> have names that suggest they are not proper "functions" (i.e. their
>>>>>> return is not a function of their arguments alone and they access the
>>>>>> data graph .. which might be a query).
>>>>>>
>>>>>> I can't see why it might now have answers when it didn't before.
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>> On 19/05/2020 13:04, Martynas Jusevičius wrote:
>>>>>>> With 3.0.1, no such internal *violations* are produced.
>>>>>>>
>>>>>>> On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius
>>>>>>> <ma...@atomgraph.com> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> After upgrading our code as well as SPINRDF from 3.0.1 to
>>>>>>>> 3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
>>>>>>>>
>>>>>>>> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
>>>>>>>> are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
>>>>>>>> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
>>>>>>>> With 3.0.1, no such internal validations are produced.
>>>>>>>>
>>>>>>>> The models being validated use a basic RDFS inference (subclassing etc.)
>>>>>>>>
>>>>>>>> I have tracked the source of the difference to query execution in
>>>>>>>> SPINConstraints.runQueryOnClass():
>>>>>>>> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
>>>>>>>>
>>>>>>>> Since SPINRDF code hasn't changed, only the Jena versions, could there
>>>>>>>> be differences in query execution? The specific query that produces
>>>>>>>> different results is this:
>>>>>>>>
>>>>>>>> CONSTRUCT
>>>>>>>> {
>>>>>>>> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
>>>>>>>> <http://spinrdf.org/spin#ConstraintViolation> .
>>>>>>>> _:c0 <http://spinrdf.org/spin#violationRoot> ?this .
>>>>>>>> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
>>>>>>>> }
>>>>>>>> WHERE
>>>>>>>> { ?this a ?TYPE_CLASS
>>>>>>>> { FILTER ( bound(?minCount) && (
>>>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
>>>>>>>> ) }
>>>>>>>> UNION
>>>>>>>> { FILTER ( bound(?maxCount) && (
>>>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
>>>>>>>> ) }
>>>>>>>> UNION
>>>>>>>> { FILTER bound(?valueType)
>>>>>>>> ?this ?predicate ?value
>>>>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> The query string and initial bindings are the same with both versions,
>>>>>>>> for example:
>>>>>>>>
>>>>>>>> ( ?predicate, spin:body )( ?comment, "the body of the Template" )(
>>>>>>>> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
>>>>>>>>
>>>>>>>> Are there some obvious suspects here or do I need to create a
>>>>>>>> reproducible example?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Martynas
Re: SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Posted by Martynas Jusevičius <ma...@atomgraph.com>.
Andy,
which part needs to be replaced with syntax rewriting -- SPIN
functions or the initial bindings?
Do you think it's possible to rewrite SPINRDF in a more SPARQL
compliant and less ARQ-dependent way?
On Thu, May 21, 2020 at 10:54 AM Andy Seaborne <an...@apache.org> wrote:
>
>
>
> On 20/05/2020 22:11, Martynas Jusevičius wrote:
> > https://github.com/spinrdf/spinrdf/issues/22
> >
> > On Wed, May 20, 2020 at 10:57 PM Martynas Jusevičius
> > <ma...@atomgraph.com> wrote:
> >>
> >> Andy,
> >>
> >> I was able to isolate a standalone example:
> >> https://github.com/namedgraph/spinrdf-test
> >>
> >> With 3.0.1 it succeeds. Change Jena version to 3.16.0-SNAPSHOT by
> >> uncommenting the other <version> in pom.xml and it will fail:
> >> https://github.com/namedgraph/spinrdf-test/blob/master/pom.xml#L28
> >>
> >> I don't understand how or where spl:objectCount() and spl:instanceOf()
> >> are executed, but i think it demonstrates clearly that some behaviour
> >> has changed between those versions, even though SpinRDF code hasn't.
>
> Many SpinRDF functions are written with implicit assumptions about how
> ARQ - not SPARQL - executes. They don't work when the optimizer is off,
> reconfigured, or if the reference query engine is used.
>
> In addition, the external injection of initial bindings
>
> (Long term - that should be replaced with syntax rewriting of the query
> - see QueryTransformOps)
>
> The result is that each version of SpinRDF needs to be tuned to the
> version of Jena it uses.
>
> Andy
>
> >>
> >> Will post on https://github.com/spinrdf/spinrdf/issues as well.
> >>
> >>
> >> Martynas
> >>
> >> On Wed, May 20, 2020 at 12:37 PM Martynas Jusevičius
> >> <ma...@atomgraph.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I've narrowed down the differences to the counts produced by
> >>> spl:objectCount() SPIN function.
> >>>
> >>> SELECT *
> >>> WHERE
> >>> { ?this a ?TYPE_CLASS
> >>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
> >>> AS ?objCount)
> >>> FILTER ( bound(?minCount) && ( ?objCount < ?minCount ) )
> >>> }
> >>> UNION
> >>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
> >>> AS ?objCount)
> >>> FILTER ( bound(?maxCount) && ( ?objCount > ?maxCount ) )
> >>> }
> >>> UNION
> >>> { FILTER bound(?valueType)
> >>> ?this ?predicate ?value
> >>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
> >>> }
> >>> }
> >>>
> >>> Bindings: ( ?predicate, spin:body )( ?comment, "the body of the
> >>> Template" )( ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount,
> >>> 1 )
> >>>
> >>> 3.15.0-SNAPSHOT
> >>>
> >>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >>> | this
> >>> | TYPE_CLASS |
> >>> objCount | predicate | value |
> >>> =================================================================================================================================================================================================
> >>> | <http://spinrdf.org/spin#Templates>
> >>> | <http://spinrdf.org/spin#Template> |
> >>> 2 | <http://spinrdf.org/spin#body> | |
> >>> | <http://spinrdf.org/spin#ConstructTemplates>
> >>> | <http://spinrdf.org/spin#Template> |
> >>> 2 | <http://spinrdf.org/spin#body> | |
> >>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectUpdateTemplate>
> >>> | <http://spinrdf.org/spin#Template> | 2 |
> >>> <http://spinrdf.org/spin#body> | |
> >>> | <http://spinrdf.org/spin#AskTemplates>
> >>> | <http://spinrdf.org/spin#Template> |
> >>> 2 | <http://spinrdf.org/spin#body> | |
> >>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectQueryTemplate>
> >>> | <http://spinrdf.org/spin#Template> | 2 |
> >>> <http://spinrdf.org/spin#body> | |
> >>> | <http://spinrdf.org/spin#UpdateTemplates>
> >>> | <http://spinrdf.org/spin#Template> |
> >>> 2 | <http://spinrdf.org/spin#body> | |
> >>> | <http://spinrdf.org/spl#Argument>
> >>> | <http://spinrdf.org/spin#Template> |
> >>> 2 | <http://spinrdf.org/spin#body> | |
> >>> | <http://spinrdf.org/spin#SelectTemplates>
> >>> | <http://spinrdf.org/spin#Template> |
> >>> 2 | <http://spinrdf.org/spin#body> | |
> >>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >>>
> >>> 3.0.1
> >>>
> >>> ----------------------------------------------------
> >>> | this | TYPE_CLASS | objCount | predicate | value |
> >>> ====================================================
> >>> ----------------------------------------------------
> >>>
> >>> I'll try to create an example now.
> >>>
> >>>
> >>> On Tue, May 19, 2020 at 6:14 PM Andy Seaborne <an...@apache.org> wrote:
> >>>>
> >>>> That's a long jump.
> >>>>
> >>>> (and it's a SpinRDF question)
> >>>>
> >>>> >> Are there some obvious suspects here or do I need to create a
> >>>> >> reproducible example?
> >>>>
> >>>> reproducible example (noting that the query has custom functions which
> >>>> have names that suggest they are not proper "functions" (i.e. their
> >>>> return is not a function of their arguments alone and they access the
> >>>> data graph .. which might be a query).
> >>>>
> >>>> I can't see why it might now have answers when it didn't before.
> >>>>
> >>>> Andy
> >>>>
> >>>> On 19/05/2020 13:04, Martynas Jusevičius wrote:
> >>>>> With 3.0.1, no such internal *violations* are produced.
> >>>>>
> >>>>> On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius
> >>>>> <ma...@atomgraph.com> wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> After upgrading our code as well as SPINRDF from 3.0.1 to
> >>>>>> 3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
> >>>>>>
> >>>>>> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
> >>>>>> are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
> >>>>>> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
> >>>>>> With 3.0.1, no such internal validations are produced.
> >>>>>>
> >>>>>> The models being validated use a basic RDFS inference (subclassing etc.)
> >>>>>>
> >>>>>> I have tracked the source of the difference to query execution in
> >>>>>> SPINConstraints.runQueryOnClass():
> >>>>>> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
> >>>>>>
> >>>>>> Since SPINRDF code hasn't changed, only the Jena versions, could there
> >>>>>> be differences in query execution? The specific query that produces
> >>>>>> different results is this:
> >>>>>>
> >>>>>> CONSTRUCT
> >>>>>> {
> >>>>>> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
> >>>>>> <http://spinrdf.org/spin#ConstraintViolation> .
> >>>>>> _:c0 <http://spinrdf.org/spin#violationRoot> ?this .
> >>>>>> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
> >>>>>> }
> >>>>>> WHERE
> >>>>>> { ?this a ?TYPE_CLASS
> >>>>>> { FILTER ( bound(?minCount) && (
> >>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
> >>>>>> ) }
> >>>>>> UNION
> >>>>>> { FILTER ( bound(?maxCount) && (
> >>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
> >>>>>> ) }
> >>>>>> UNION
> >>>>>> { FILTER bound(?valueType)
> >>>>>> ?this ?predicate ?value
> >>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
> >>>>>> }
> >>>>>> }
> >>>>>>
> >>>>>> The query string and initial bindings are the same with both versions,
> >>>>>> for example:
> >>>>>>
> >>>>>> ( ?predicate, spin:body )( ?comment, "the body of the Template" )(
> >>>>>> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
> >>>>>>
> >>>>>> Are there some obvious suspects here or do I need to create a
> >>>>>> reproducible example?
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Martynas
Re: SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Posted by Andy Seaborne <an...@apache.org>.
On 20/05/2020 22:11, Martynas Jusevičius wrote:
> https://github.com/spinrdf/spinrdf/issues/22
>
> On Wed, May 20, 2020 at 10:57 PM Martynas Jusevičius
> <ma...@atomgraph.com> wrote:
>>
>> Andy,
>>
>> I was able to isolate a standalone example:
>> https://github.com/namedgraph/spinrdf-test
>>
>> With 3.0.1 it succeeds. Change Jena version to 3.16.0-SNAPSHOT by
>> uncommenting the other <version> in pom.xml and it will fail:
>> https://github.com/namedgraph/spinrdf-test/blob/master/pom.xml#L28
>>
>> I don't understand how or where spl:objectCount() and spl:instanceOf()
>> are executed, but i think it demonstrates clearly that some behaviour
>> has changed between those versions, even though SpinRDF code hasn't.
Many SpinRDF functions are written with implicit assumptions about how
ARQ - not SPARQL - executes. They don't work when the optimizer is off,
reconfigured, or if the reference query engine is used.
In addition, the external injection of initial bindings
(Long term - that should be replaced with syntax rewriting of the query
- see QueryTransformOps)
The result is that each version of SpinRDF needs to be tuned to the
version of Jena it uses.
Andy
>>
>> Will post on https://github.com/spinrdf/spinrdf/issues as well.
>>
>>
>> Martynas
>>
>> On Wed, May 20, 2020 at 12:37 PM Martynas Jusevičius
>> <ma...@atomgraph.com> wrote:
>>>
>>> Hi,
>>>
>>> I've narrowed down the differences to the counts produced by
>>> spl:objectCount() SPIN function.
>>>
>>> SELECT *
>>> WHERE
>>> { ?this a ?TYPE_CLASS
>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
>>> AS ?objCount)
>>> FILTER ( bound(?minCount) && ( ?objCount < ?minCount ) )
>>> }
>>> UNION
>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
>>> AS ?objCount)
>>> FILTER ( bound(?maxCount) && ( ?objCount > ?maxCount ) )
>>> }
>>> UNION
>>> { FILTER bound(?valueType)
>>> ?this ?predicate ?value
>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
>>> }
>>> }
>>>
>>> Bindings: ( ?predicate, spin:body )( ?comment, "the body of the
>>> Template" )( ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount,
>>> 1 )
>>>
>>> 3.15.0-SNAPSHOT
>>>
>>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>> | this
>>> | TYPE_CLASS |
>>> objCount | predicate | value |
>>> =================================================================================================================================================================================================
>>> | <http://spinrdf.org/spin#Templates>
>>> | <http://spinrdf.org/spin#Template> |
>>> 2 | <http://spinrdf.org/spin#body> | |
>>> | <http://spinrdf.org/spin#ConstructTemplates>
>>> | <http://spinrdf.org/spin#Template> |
>>> 2 | <http://spinrdf.org/spin#body> | |
>>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectUpdateTemplate>
>>> | <http://spinrdf.org/spin#Template> | 2 |
>>> <http://spinrdf.org/spin#body> | |
>>> | <http://spinrdf.org/spin#AskTemplates>
>>> | <http://spinrdf.org/spin#Template> |
>>> 2 | <http://spinrdf.org/spin#body> | |
>>> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectQueryTemplate>
>>> | <http://spinrdf.org/spin#Template> | 2 |
>>> <http://spinrdf.org/spin#body> | |
>>> | <http://spinrdf.org/spin#UpdateTemplates>
>>> | <http://spinrdf.org/spin#Template> |
>>> 2 | <http://spinrdf.org/spin#body> | |
>>> | <http://spinrdf.org/spl#Argument>
>>> | <http://spinrdf.org/spin#Template> |
>>> 2 | <http://spinrdf.org/spin#body> | |
>>> | <http://spinrdf.org/spin#SelectTemplates>
>>> | <http://spinrdf.org/spin#Template> |
>>> 2 | <http://spinrdf.org/spin#body> | |
>>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>>
>>> 3.0.1
>>>
>>> ----------------------------------------------------
>>> | this | TYPE_CLASS | objCount | predicate | value |
>>> ====================================================
>>> ----------------------------------------------------
>>>
>>> I'll try to create an example now.
>>>
>>>
>>> On Tue, May 19, 2020 at 6:14 PM Andy Seaborne <an...@apache.org> wrote:
>>>>
>>>> That's a long jump.
>>>>
>>>> (and it's a SpinRDF question)
>>>>
>>>> >> Are there some obvious suspects here or do I need to create a
>>>> >> reproducible example?
>>>>
>>>> reproducible example (noting that the query has custom functions which
>>>> have names that suggest they are not proper "functions" (i.e. their
>>>> return is not a function of their arguments alone and they access the
>>>> data graph .. which might be a query).
>>>>
>>>> I can't see why it might now have answers when it didn't before.
>>>>
>>>> Andy
>>>>
>>>> On 19/05/2020 13:04, Martynas Jusevičius wrote:
>>>>> With 3.0.1, no such internal *violations* are produced.
>>>>>
>>>>> On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius
>>>>> <ma...@atomgraph.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> After upgrading our code as well as SPINRDF from 3.0.1 to
>>>>>> 3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
>>>>>>
>>>>>> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
>>>>>> are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
>>>>>> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
>>>>>> With 3.0.1, no such internal validations are produced.
>>>>>>
>>>>>> The models being validated use a basic RDFS inference (subclassing etc.)
>>>>>>
>>>>>> I have tracked the source of the difference to query execution in
>>>>>> SPINConstraints.runQueryOnClass():
>>>>>> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
>>>>>>
>>>>>> Since SPINRDF code hasn't changed, only the Jena versions, could there
>>>>>> be differences in query execution? The specific query that produces
>>>>>> different results is this:
>>>>>>
>>>>>> CONSTRUCT
>>>>>> {
>>>>>> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
>>>>>> <http://spinrdf.org/spin#ConstraintViolation> .
>>>>>> _:c0 <http://spinrdf.org/spin#violationRoot> ?this .
>>>>>> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
>>>>>> }
>>>>>> WHERE
>>>>>> { ?this a ?TYPE_CLASS
>>>>>> { FILTER ( bound(?minCount) && (
>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
>>>>>> ) }
>>>>>> UNION
>>>>>> { FILTER ( bound(?maxCount) && (
>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
>>>>>> ) }
>>>>>> UNION
>>>>>> { FILTER bound(?valueType)
>>>>>> ?this ?predicate ?value
>>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> The query string and initial bindings are the same with both versions,
>>>>>> for example:
>>>>>>
>>>>>> ( ?predicate, spin:body )( ?comment, "the body of the Template" )(
>>>>>> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
>>>>>>
>>>>>> Are there some obvious suspects here or do I need to create a
>>>>>> reproducible example?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Martynas
Re: SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Posted by Martynas Jusevičius <ma...@atomgraph.com>.
https://github.com/spinrdf/spinrdf/issues/22
On Wed, May 20, 2020 at 10:57 PM Martynas Jusevičius
<ma...@atomgraph.com> wrote:
>
> Andy,
>
> I was able to isolate a standalone example:
> https://github.com/namedgraph/spinrdf-test
>
> With 3.0.1 it succeeds. Change Jena version to 3.16.0-SNAPSHOT by
> uncommenting the other <version> in pom.xml and it will fail:
> https://github.com/namedgraph/spinrdf-test/blob/master/pom.xml#L28
>
> I don't understand how or where spl:objectCount() and spl:instanceOf()
> are executed, but i think it demonstrates clearly that some behaviour
> has changed between those versions, even though SpinRDF code hasn't.
>
> Will post on https://github.com/spinrdf/spinrdf/issues as well.
>
>
> Martynas
>
> On Wed, May 20, 2020 at 12:37 PM Martynas Jusevičius
> <ma...@atomgraph.com> wrote:
> >
> > Hi,
> >
> > I've narrowed down the differences to the counts produced by
> > spl:objectCount() SPIN function.
> >
> > SELECT *
> > WHERE
> > { ?this a ?TYPE_CLASS
> > { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
> > AS ?objCount)
> > FILTER ( bound(?minCount) && ( ?objCount < ?minCount ) )
> > }
> > UNION
> > { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
> > AS ?objCount)
> > FILTER ( bound(?maxCount) && ( ?objCount > ?maxCount ) )
> > }
> > UNION
> > { FILTER bound(?valueType)
> > ?this ?predicate ?value
> > FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
> > }
> > }
> >
> > Bindings: ( ?predicate, spin:body )( ?comment, "the body of the
> > Template" )( ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount,
> > 1 )
> >
> > 3.15.0-SNAPSHOT
> >
> > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > | this
> > | TYPE_CLASS |
> > objCount | predicate | value |
> > =================================================================================================================================================================================================
> > | <http://spinrdf.org/spin#Templates>
> > | <http://spinrdf.org/spin#Template> |
> > 2 | <http://spinrdf.org/spin#body> | |
> > | <http://spinrdf.org/spin#ConstructTemplates>
> > | <http://spinrdf.org/spin#Template> |
> > 2 | <http://spinrdf.org/spin#body> | |
> > | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectUpdateTemplate>
> > | <http://spinrdf.org/spin#Template> | 2 |
> > <http://spinrdf.org/spin#body> | |
> > | <http://spinrdf.org/spin#AskTemplates>
> > | <http://spinrdf.org/spin#Template> |
> > 2 | <http://spinrdf.org/spin#body> | |
> > | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectQueryTemplate>
> > | <http://spinrdf.org/spin#Template> | 2 |
> > <http://spinrdf.org/spin#body> | |
> > | <http://spinrdf.org/spin#UpdateTemplates>
> > | <http://spinrdf.org/spin#Template> |
> > 2 | <http://spinrdf.org/spin#body> | |
> > | <http://spinrdf.org/spl#Argument>
> > | <http://spinrdf.org/spin#Template> |
> > 2 | <http://spinrdf.org/spin#body> | |
> > | <http://spinrdf.org/spin#SelectTemplates>
> > | <http://spinrdf.org/spin#Template> |
> > 2 | <http://spinrdf.org/spin#body> | |
> > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >
> > 3.0.1
> >
> > ----------------------------------------------------
> > | this | TYPE_CLASS | objCount | predicate | value |
> > ====================================================
> > ----------------------------------------------------
> >
> > I'll try to create an example now.
> >
> >
> > On Tue, May 19, 2020 at 6:14 PM Andy Seaborne <an...@apache.org> wrote:
> > >
> > > That's a long jump.
> > >
> > > (and it's a SpinRDF question)
> > >
> > > >> Are there some obvious suspects here or do I need to create a
> > > >> reproducible example?
> > >
> > > reproducible example (noting that the query has custom functions which
> > > have names that suggest they are not proper "functions" (i.e. their
> > > return is not a function of their arguments alone and they access the
> > > data graph .. which might be a query).
> > >
> > > I can't see why it might now have answers when it didn't before.
> > >
> > > Andy
> > >
> > > On 19/05/2020 13:04, Martynas Jusevičius wrote:
> > > > With 3.0.1, no such internal *violations* are produced.
> > > >
> > > > On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius
> > > > <ma...@atomgraph.com> wrote:
> > > >>
> > > >> Hi,
> > > >>
> > > >> After upgrading our code as well as SPINRDF from 3.0.1 to
> > > >> 3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
> > > >>
> > > >> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
> > > >> are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
> > > >> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
> > > >> With 3.0.1, no such internal validations are produced.
> > > >>
> > > >> The models being validated use a basic RDFS inference (subclassing etc.)
> > > >>
> > > >> I have tracked the source of the difference to query execution in
> > > >> SPINConstraints.runQueryOnClass():
> > > >> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
> > > >>
> > > >> Since SPINRDF code hasn't changed, only the Jena versions, could there
> > > >> be differences in query execution? The specific query that produces
> > > >> different results is this:
> > > >>
> > > >> CONSTRUCT
> > > >> {
> > > >> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
> > > >> <http://spinrdf.org/spin#ConstraintViolation> .
> > > >> _:c0 <http://spinrdf.org/spin#violationRoot> ?this .
> > > >> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
> > > >> }
> > > >> WHERE
> > > >> { ?this a ?TYPE_CLASS
> > > >> { FILTER ( bound(?minCount) && (
> > > >> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
> > > >> ) }
> > > >> UNION
> > > >> { FILTER ( bound(?maxCount) && (
> > > >> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
> > > >> ) }
> > > >> UNION
> > > >> { FILTER bound(?valueType)
> > > >> ?this ?predicate ?value
> > > >> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
> > > >> }
> > > >> }
> > > >>
> > > >> The query string and initial bindings are the same with both versions,
> > > >> for example:
> > > >>
> > > >> ( ?predicate, spin:body )( ?comment, "the body of the Template" )(
> > > >> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
> > > >>
> > > >> Are there some obvious suspects here or do I need to create a
> > > >> reproducible example?
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Martynas
Re: SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Posted by Martynas Jusevičius <ma...@atomgraph.com>.
Andy,
I was able to isolate a standalone example:
https://github.com/namedgraph/spinrdf-test
With 3.0.1 it succeeds. Change Jena version to 3.16.0-SNAPSHOT by
uncommenting the other <version> in pom.xml and it will fail:
https://github.com/namedgraph/spinrdf-test/blob/master/pom.xml#L28
I don't understand how or where spl:objectCount() and spl:instanceOf()
are executed, but i think it demonstrates clearly that some behaviour
has changed between those versions, even though SpinRDF code hasn't.
Will post on https://github.com/spinrdf/spinrdf/issues as well.
Martynas
On Wed, May 20, 2020 at 12:37 PM Martynas Jusevičius
<ma...@atomgraph.com> wrote:
>
> Hi,
>
> I've narrowed down the differences to the counts produced by
> spl:objectCount() SPIN function.
>
> SELECT *
> WHERE
> { ?this a ?TYPE_CLASS
> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
> AS ?objCount)
> FILTER ( bound(?minCount) && ( ?objCount < ?minCount ) )
> }
> UNION
> { BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
> AS ?objCount)
> FILTER ( bound(?maxCount) && ( ?objCount > ?maxCount ) )
> }
> UNION
> { FILTER bound(?valueType)
> ?this ?predicate ?value
> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
> }
> }
>
> Bindings: ( ?predicate, spin:body )( ?comment, "the body of the
> Template" )( ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount,
> 1 )
>
> 3.15.0-SNAPSHOT
>
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> | this
> | TYPE_CLASS |
> objCount | predicate | value |
> =================================================================================================================================================================================================
> | <http://spinrdf.org/spin#Templates>
> | <http://spinrdf.org/spin#Template> |
> 2 | <http://spinrdf.org/spin#body> | |
> | <http://spinrdf.org/spin#ConstructTemplates>
> | <http://spinrdf.org/spin#Template> |
> 2 | <http://spinrdf.org/spin#body> | |
> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectUpdateTemplate>
> | <http://spinrdf.org/spin#Template> | 2 |
> <http://spinrdf.org/spin#body> | |
> | <http://spinrdf.org/spin#AskTemplates>
> | <http://spinrdf.org/spin#Template> |
> 2 | <http://spinrdf.org/spin#body> | |
> | <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectQueryTemplate>
> | <http://spinrdf.org/spin#Template> | 2 |
> <http://spinrdf.org/spin#body> | |
> | <http://spinrdf.org/spin#UpdateTemplates>
> | <http://spinrdf.org/spin#Template> |
> 2 | <http://spinrdf.org/spin#body> | |
> | <http://spinrdf.org/spl#Argument>
> | <http://spinrdf.org/spin#Template> |
> 2 | <http://spinrdf.org/spin#body> | |
> | <http://spinrdf.org/spin#SelectTemplates>
> | <http://spinrdf.org/spin#Template> |
> 2 | <http://spinrdf.org/spin#body> | |
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> 3.0.1
>
> ----------------------------------------------------
> | this | TYPE_CLASS | objCount | predicate | value |
> ====================================================
> ----------------------------------------------------
>
> I'll try to create an example now.
>
>
> On Tue, May 19, 2020 at 6:14 PM Andy Seaborne <an...@apache.org> wrote:
> >
> > That's a long jump.
> >
> > (and it's a SpinRDF question)
> >
> > >> Are there some obvious suspects here or do I need to create a
> > >> reproducible example?
> >
> > reproducible example (noting that the query has custom functions which
> > have names that suggest they are not proper "functions" (i.e. their
> > return is not a function of their arguments alone and they access the
> > data graph .. which might be a query).
> >
> > I can't see why it might now have answers when it didn't before.
> >
> > Andy
> >
> > On 19/05/2020 13:04, Martynas Jusevičius wrote:
> > > With 3.0.1, no such internal *violations* are produced.
> > >
> > > On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius
> > > <ma...@atomgraph.com> wrote:
> > >>
> > >> Hi,
> > >>
> > >> After upgrading our code as well as SPINRDF from 3.0.1 to
> > >> 3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
> > >>
> > >> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
> > >> are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
> > >> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
> > >> With 3.0.1, no such internal validations are produced.
> > >>
> > >> The models being validated use a basic RDFS inference (subclassing etc.)
> > >>
> > >> I have tracked the source of the difference to query execution in
> > >> SPINConstraints.runQueryOnClass():
> > >> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
> > >>
> > >> Since SPINRDF code hasn't changed, only the Jena versions, could there
> > >> be differences in query execution? The specific query that produces
> > >> different results is this:
> > >>
> > >> CONSTRUCT
> > >> {
> > >> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
> > >> <http://spinrdf.org/spin#ConstraintViolation> .
> > >> _:c0 <http://spinrdf.org/spin#violationRoot> ?this .
> > >> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
> > >> }
> > >> WHERE
> > >> { ?this a ?TYPE_CLASS
> > >> { FILTER ( bound(?minCount) && (
> > >> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
> > >> ) }
> > >> UNION
> > >> { FILTER ( bound(?maxCount) && (
> > >> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
> > >> ) }
> > >> UNION
> > >> { FILTER bound(?valueType)
> > >> ?this ?predicate ?value
> > >> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
> > >> }
> > >> }
> > >>
> > >> The query string and initial bindings are the same with both versions,
> > >> for example:
> > >>
> > >> ( ?predicate, spin:body )( ?comment, "the body of the Template" )(
> > >> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
> > >>
> > >> Are there some obvious suspects here or do I need to create a
> > >> reproducible example?
> > >>
> > >> Thanks,
> > >>
> > >> Martynas
Re: SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Posted by Martynas Jusevičius <ma...@atomgraph.com>.
Hi,
I've narrowed down the differences to the counts produced by
spl:objectCount() SPIN function.
SELECT *
WHERE
{ ?this a ?TYPE_CLASS
{ BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
AS ?objCount)
FILTER ( bound(?minCount) && ( ?objCount < ?minCount ) )
}
UNION
{ BIND(<http://spinrdf.org/spl#objectCount>(?this, ?predicate)
AS ?objCount)
FILTER ( bound(?maxCount) && ( ?objCount > ?maxCount ) )
}
UNION
{ FILTER bound(?valueType)
?this ?predicate ?value
FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
}
}
Bindings: ( ?predicate, spin:body )( ?comment, "the body of the
Template" )( ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount,
1 )
3.15.0-SNAPSHOT
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| this
| TYPE_CLASS |
objCount | predicate | value |
=================================================================================================================================================================================================
| <http://spinrdf.org/spin#Templates>
| <http://spinrdf.org/spin#Template> |
2 | <http://spinrdf.org/spin#body> | |
| <http://spinrdf.org/spin#ConstructTemplates>
| <http://spinrdf.org/spin#Template> |
2 | <http://spinrdf.org/spin#body> | |
| <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectUpdateTemplate>
| <http://spinrdf.org/spin#Template> | 2 |
<http://spinrdf.org/spin#body> | |
| <http://spinrdf.org/spin#AskTemplates>
| <http://spinrdf.org/spin#Template> |
2 | <http://spinrdf.org/spin#body> | |
| <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectQueryTemplate>
| <http://spinrdf.org/spin#Template> | 2 |
<http://spinrdf.org/spin#body> | |
| <http://spinrdf.org/spin#UpdateTemplates>
| <http://spinrdf.org/spin#Template> |
2 | <http://spinrdf.org/spin#body> | |
| <http://spinrdf.org/spl#Argument>
| <http://spinrdf.org/spin#Template> |
2 | <http://spinrdf.org/spin#body> | |
| <http://spinrdf.org/spin#SelectTemplates>
| <http://spinrdf.org/spin#Template> |
2 | <http://spinrdf.org/spin#body> | |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
3.0.1
----------------------------------------------------
| this | TYPE_CLASS | objCount | predicate | value |
====================================================
----------------------------------------------------
I'll try to create an example now.
On Tue, May 19, 2020 at 6:14 PM Andy Seaborne <an...@apache.org> wrote:
>
> That's a long jump.
>
> (and it's a SpinRDF question)
>
> >> Are there some obvious suspects here or do I need to create a
> >> reproducible example?
>
> reproducible example (noting that the query has custom functions which
> have names that suggest they are not proper "functions" (i.e. their
> return is not a function of their arguments alone and they access the
> data graph .. which might be a query).
>
> I can't see why it might now have answers when it didn't before.
>
> Andy
>
> On 19/05/2020 13:04, Martynas Jusevičius wrote:
> > With 3.0.1, no such internal *violations* are produced.
> >
> > On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius
> > <ma...@atomgraph.com> wrote:
> >>
> >> Hi,
> >>
> >> After upgrading our code as well as SPINRDF from 3.0.1 to
> >> 3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
> >>
> >> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
> >> are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
> >> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
> >> With 3.0.1, no such internal validations are produced.
> >>
> >> The models being validated use a basic RDFS inference (subclassing etc.)
> >>
> >> I have tracked the source of the difference to query execution in
> >> SPINConstraints.runQueryOnClass():
> >> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
> >>
> >> Since SPINRDF code hasn't changed, only the Jena versions, could there
> >> be differences in query execution? The specific query that produces
> >> different results is this:
> >>
> >> CONSTRUCT
> >> {
> >> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
> >> <http://spinrdf.org/spin#ConstraintViolation> .
> >> _:c0 <http://spinrdf.org/spin#violationRoot> ?this .
> >> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
> >> }
> >> WHERE
> >> { ?this a ?TYPE_CLASS
> >> { FILTER ( bound(?minCount) && (
> >> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
> >> ) }
> >> UNION
> >> { FILTER ( bound(?maxCount) && (
> >> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
> >> ) }
> >> UNION
> >> { FILTER bound(?valueType)
> >> ?this ?predicate ?value
> >> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
> >> }
> >> }
> >>
> >> The query string and initial bindings are the same with both versions,
> >> for example:
> >>
> >> ( ?predicate, spin:body )( ?comment, "the body of the Template" )(
> >> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
> >>
> >> Are there some obvious suspects here or do I need to create a
> >> reproducible example?
> >>
> >> Thanks,
> >>
> >> Martynas
Re: SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Posted by Andy Seaborne <an...@apache.org>.
That's a long jump.
(and it's a SpinRDF question)
>> Are there some obvious suspects here or do I need to create a
>> reproducible example?
reproducible example (noting that the query has custom functions which
have names that suggest they are not proper "functions" (i.e. their
return is not a function of their arguments alone and they access the
data graph .. which might be a query).
I can't see why it might now have answers when it didn't before.
Andy
On 19/05/2020 13:04, Martynas Jusevičius wrote:
> With 3.0.1, no such internal *violations* are produced.
>
> On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius
> <ma...@atomgraph.com> wrote:
>>
>> Hi,
>>
>> After upgrading our code as well as SPINRDF from 3.0.1 to
>> 3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
>>
>> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
>> are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
>> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
>> With 3.0.1, no such internal validations are produced.
>>
>> The models being validated use a basic RDFS inference (subclassing etc.)
>>
>> I have tracked the source of the difference to query execution in
>> SPINConstraints.runQueryOnClass():
>> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
>>
>> Since SPINRDF code hasn't changed, only the Jena versions, could there
>> be differences in query execution? The specific query that produces
>> different results is this:
>>
>> CONSTRUCT
>> {
>> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
>> <http://spinrdf.org/spin#ConstraintViolation> .
>> _:c0 <http://spinrdf.org/spin#violationRoot> ?this .
>> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
>> }
>> WHERE
>> { ?this a ?TYPE_CLASS
>> { FILTER ( bound(?minCount) && (
>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
>> ) }
>> UNION
>> { FILTER ( bound(?maxCount) && (
>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
>> ) }
>> UNION
>> { FILTER bound(?valueType)
>> ?this ?predicate ?value
>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
>> }
>> }
>>
>> The query string and initial bindings are the same with both versions,
>> for example:
>>
>> ( ?predicate, spin:body )( ?comment, "the body of the Template" )(
>> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
>>
>> Are there some obvious suspects here or do I need to create a
>> reproducible example?
>>
>> Thanks,
>>
>> Martynas
Re: SPIN constraint validation differs on 3.0.1 and 3.16.0-SNAPSHOT
Posted by Martynas Jusevičius <ma...@atomgraph.com>.
With 3.0.1, no such internal *violations* are produced.
On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius
<ma...@atomgraph.com> wrote:
>
> Hi,
>
> After upgrading our code as well as SPINRDF from 3.0.1 to
> 3.16.0-SNAPSHOT, some constraint violation-related tests are failing.
>
> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint violations
> are coming from the SPIN vocabulary iself, e.g. "Attribute spin:body :
> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc.
> With 3.0.1, no such internal validations are produced.
>
> The models being validated use a basic RDFS inference (subclassing etc.)
>
> I have tracked the source of the difference to query execution in
> SPINConstraints.runQueryOnClass():
> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614
>
> Since SPINRDF code hasn't changed, only the Jena versions, could there
> be differences in query execution? The specific query that produces
> different results is this:
>
> CONSTRUCT
> {
> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
> <http://spinrdf.org/spin#ConstraintViolation> .
> _:c0 <http://spinrdf.org/spin#violationRoot> ?this .
> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate .
> }
> WHERE
> { ?this a ?TYPE_CLASS
> { FILTER ( bound(?minCount) && (
> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < ?minCount )
> ) }
> UNION
> { FILTER ( bound(?maxCount) && (
> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > ?maxCount )
> ) }
> UNION
> { FILTER bound(?valueType)
> ?this ?predicate ?value
> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, ?valueType) )
> }
> }
>
> The query string and initial bindings are the same with both versions,
> for example:
>
> ( ?predicate, spin:body )( ?comment, "the body of the Template" )(
> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 )
>
> Are there some obvious suspects here or do I need to create a
> reproducible example?
>
> Thanks,
>
> Martynas