Reasoners

[Claude Warren <cl...@xenei.com>]

I am digging into reasoners for the first time in a long time and have a
couple of questions.

What I am trying to do is the following:

Assume data:

<x> <my:sameValue> <y>
<y> <my:value> 5

I want a rule that will produce

<x> <my:value> 5

but in addition if I change [<y> <my:value> 5] to [<y> <my:value> 10] I
want the reasoner to change [<x> <my:value> 5] to [<x> <my:value> 10]

So after first reasoning data are:

<x> <my:sameValue> <y>
<y> <my:value> 5
<x> <my:value> 5

and after changing <y> <my:value> 5 the data are:

<x> <my:sameValue> <y>
<y> <my:value> 10
<x> <my:value> 10

Note that the <x> <my:value> 5 has been deleted.

Is this possible?

In general do rules recalculate the results when the data changes?--

Thx,
Claude
<http://www.linkedin.com/in/claudewarren>

Re: Reasoners

[Dave Reynolds <da...@gmail.com>]

On 10/01/17 19:31, Paul Houle wrote:
> On Mon, Jan 9, 2017, at 05:31 PM, Dave Reynolds wrote:
>
>>
>> Great that this works for you but the design centre for Jena Rules was
>> deductive closure not event-condition-action (ECA) or reactive rules.
>
>       One of the things I think about is why production rules haven't
>       had a bigger impact in the industry and one of the big problems is
>       a lack of unity.  Some of that is as simple as there is no "C
>       language" for rules,  but part of the reason it is is that people
>       are not thinking from the viewpoint that I might want to apply (at
>       least some of) the same rules to deductive closure, ECA, SMT
>       solving,  backwards chaining reasoning,  and other modes of
>       execution.  I see a need to separate "rules" from "schemes" that
>       define how the rules are applied.

This is possibly getting off topic for jena-dev but can't resist 
pointing out the RIF-CORE is a subset of both RIF-BLD (i.e. backward) or 
RIF-PRD (i.e. forward production) and so is executable by either sort of 
engine.

Indeed monotonic non-hybrid JenaRules can be run by either the forward 
or backward engine.

The cost of this is that the shared subset is small and "pure" so 
doesn't support all the negation and non-monotonic actions you might want.

>     Associated with that some issues I have are:
>
> * the structure of "common sense knowledge" is "this is true unless..."
> and there are many ways to implement this in various inference engines
> that aren't compatible.  Most of them involve manipulation of the order
> in which rules fire in time.  Like the many ways to map relational
> structures to objects with Java-style inheritence,  none of them are
> entirely satisfying.

The frame problem has been a known hard problem for quite a few decades 
:) Though I do seem to remember that at around the time RIF was 
finishing a Darpa funded consortium developed a rather sophisticated 
rule system that had a really good model of default knowledge. Can't 
recall the details and haven't tracked its progress.

> * a "road less traveled" is the subdivision of the knowledge base into
> subsets (like the named graphs in RDF) which can also be used to control
> the sequence of events (and implement overriding,  the "deletion" of
> facts by not copying them to a target graph, and other fun things.)  In
> the case of Jena Rules this would look like an expansion of the matching
> patterns to quads which I can't imagine would be that difficult

In principle a quad version of the jena forward engine would indeed be 
doable. Though the jena forward engine is a primitive beast and there's 
design choices in there I would not make again.

>> Or if one wanted something more like normal forward inference rules but
>> with more control over the non-monotonic aspects of handling retractions
>> then that would take you towards a full production rules engine and thus
>> drools. I suspect it would make more sense to just build a jena/drools
>> bridge than reinvent that wheel. There's a *lot* of years of engineering
>> in the drools RETE+++ implementation.
>
> It's an interesting idea.
>
> The manual for the Jena Rules Engine complains about error handling for
> Jena Rules but the situation with error handling in Drools is much
> worse.  Inspired by Inform7 I started writing a "Colossal Cave" sort of
> game in Drools and did not get that far because (i) it took me far too
> long to figure out why I was getting syntax errors,  and (ii) no way I
> could expect a junior programmer,  never mind a subject matter expert,
> to do the same.

> In comparison I don't find it painful at all to deal with syntax errors
> in Jena Rules,  I might have to look at the Java source code sometimes,
> but that is not like understanding what goes wrong between Drools and
> ECJ.

Interesting, especially given how primitive the syntax and error 
handling for Jena rules is.

> It seems like it ought to be possible though to put jena objects
> directly into Drools and write some kind of code generator that turns
> java rules + annotations into Drools rules.

Yes, should be possible and would allow people to use Drools as an 
engine for Jena rules and to experiment with extensions that made use of 
the richer drools language.

None of this is anything I personally have time to look at but seems 
tractable.

Dave

Re: Reasoners

[Paul Houle <pa...@ontology2.com>]

On Mon, Jan 9, 2017, at 05:31 PM, Dave Reynolds wrote:

> 
> Great that this works for you but the design centre for Jena Rules was 
> deductive closure not event-condition-action (ECA) or reactive rules.

      One of the things I think about is why production rules haven't
      had a bigger impact in the industry and one of the big problems is
      a lack of unity.  Some of that is as simple as there is no "C
      language" for rules,  but part of the reason it is is that people
      are not thinking from the viewpoint that I might want to apply (at
      least some of) the same rules to deductive closure, ECA, SMT
      solving,  backwards chaining reasoning,  and other modes of
      execution.  I see a need to separate "rules" from "schemes" that
      define how the rules are applied.

    Associated with that some issues I have are:

* the structure of "common sense knowledge" is "this is true unless..."
and there are many ways to implement this in various inference engines
that aren't compatible.  Most of them involve manipulation of the order
in which rules fire in time.  Like the many ways to map relational
structures to objects with Java-style inheritence,  none of them are
entirely satisfying.
* a "road less traveled" is the subdivision of the knowledge base into
subsets (like the named graphs in RDF) which can also be used to control
the sequence of events (and implement overriding,  the "deletion" of
facts by not copying them to a target graph, and other fun things.)  In
the case of Jena Rules this would look like an expansion of the matching
patterns to quads which I can't imagine would be that difficult


> 
> If one wanted primarily ECA type rules for event processing then you 
> would look at the various Complex Event Processing designs and see what 
> it would take to build one. Commercially there's lots (Oracle, SAP, 
> Tibco et al all have one) not sure what there is that's open source 
> other than Drools Fusion and I'm not sure how that relates to the rest 
> of Drools.

Drools Fusion is a collection of features that are part of Drools
Expert.  Part of that is support for the Allen Algebra,  and part of it
is support for "garbage collecting" events once they can no longer cause
rules to fire,  and the rest of it is the underlying rule engine's
support for truth maintainance, negation,  etc.

> Or if one wanted something more like normal forward inference rules but 
> with more control over the non-monotonic aspects of handling retractions 
> then that would take you towards a full production rules engine and thus 
> drools. I suspect it would make more sense to just build a jena/drools 
> bridge than reinvent that wheel. There's a *lot* of years of engineering 
> in the drools RETE+++ implementation.

It's an interesting idea.

The manual for the Jena Rules Engine complains about error handling for
Jena Rules but the situation with error handling in Drools is much
worse.  Inspired by Inform7 I started writing a "Colossal Cave" sort of
game in Drools and did not get that far because (i) it took me far too
long to figure out why I was getting syntax errors,  and (ii) no way I
could expect a junior programmer,  never mind a subject matter expert, 
to do the same.

In comparison I don't find it painful at all to deal with syntax errors
in Jena Rules,  I might have to look at the Java source code sometimes, 
but that is not like understanding what goes wrong between Drools and
ECJ.

It seems like it ought to be possible though to put jena objects
directly into Drools and write some kind of code generator that turns
java rules + annotations into Drools rules.

Re: Reasoners

[Dave Reynolds <da...@gmail.com>]

Hi Paul,

On 09/01/17 20:23, Paul Houle wrote:
> In vanilla RDF,  you do not change 5 to 10,  you either delete the 5 and
> add the 10 or you add the 10 then delete the five!  Maybe it all happens
> in a transaction which might make the immediate states invisible to the
> outside world.
>
> I use the Jena Rules engine with user defined functions that create side
> effects and it is a lot of fun:  I write RDF -> X conversion systems
> this way,  and also have an asynchronous execution engine.  In both of
> these cases I conciously or unconciously make everything a latch so
> deletes do not happen,  but it could be a problem if I want to use
> deletion,  I would not want to run the side effecting methods again if I
> can at all help it.

Great that this works for you but the design centre for Jena Rules was 
deductive closure not event-condition-action (ECA) or reactive rules.

> For the most part I *do not* apply inference rules to a transactional or
> archival database but rather I pull in one or more records and maybe
> some data tables and do some reasoning to answer a question about some
> topic and then use the answer or put it back in some database.

Sounds fine.

> I know Drools has better support for deletions,  so does the RDF
> database from Ontotext.
>
> What would be involved in supporting this in Jena?

The short answer is "a complete redesign".

The slightly longer answer is "it depends on what 'better support for 
deletions' means.

If one wanted primarily ECA type rules for event processing then you 
would look at the various Complex Event Processing designs and see what 
it would take to build one. Commercially there's lots (Oracle, SAP, 
Tibco et al all have one) not sure what there is that's open source 
other than Drools Fusion and I'm not sure how that relates to the rest 
of Drools.

Or if one wanted something more like normal forward inference rules but 
with more control over the non-monotonic aspects of handling retractions 
then that would take you towards a full production rules engine and thus 
drools. I suspect it would make more sense to just build a jena/drools 
bridge than reinvent that wheel. There's a *lot* of years of engineering 
in the drools RETE+++ implementation.

On the other hand, you want pure monotonic deductive closure but wanted 
more efficient handling of deletes then that would be a whole different 
ballgame and would be quite different to either of the above options.

If you stuck with a mostly-forward-inference setting then I could 
imagine there's some ways to draw a boundary around the set of 
deductions that might have been resulted from the deleted triples, 
retract the things within that boundary, and then restart the forward 
closure from there.

There was sort of a start on that in the Jena forward rules engine many 
years ago when we added hooks for reference counting. However, you can't 
just use reference counting on its own because there are lots of 
different duplicate paths to derive a given triple which are never 
normally explored, plus cycles!

It might be tempting to think of some sort of generic truth maintenance 
system for tracking what deductions are derived from what starting 
points in order to work out the things you have to delete. However, my 
hazy memory of the hey day of TMS was that the storage and maintenance 
costs always blew up so badly you were worse off than just recomputing 
when needed.

So while I'm sure a more efficient incremental handling of delete is 
possible it's a non trivial task and I wouldn't start from here.

Dave

> -- Paul Houle paul.houle@ontology2.com On Mon, Jan 9, 2017, at 02:57 PM,
> Dave Reynolds wrote:
>> Hi Claude,
>>
>> On 09/01/17 10:55, Claude Warren wrote:
>>> I am digging into reasoners for the first time in a long time and have a
>>> couple of questions.
>>>
>>> What I am trying to do is the following:
>>>
>>> Assume data:
>>>
>>> <x> <my:sameValue> <y>
>>> <y> <my:value> 5
>>>
>>> I want a rule that will produce
>>>
>>> <x> <my:value> 5
>> So something like:
>>
>>     (?x my:sameValue ?y) (?y my:value ?v) -> (?x my:value ?v)
>>
>>> but in addition if I change [<y> <my:value> 5] to [<y> <my:value> 10] I
>>> want the reasoner to change [<x> <my:value> 5] to [<x> <my:value> 10]
>>>
>>> So after first reasoning data are:
>>>
>>> <x> <my:sameValue> <y>
>>> <y> <my:value> 5
>>> <x> <my:value> 5
>>>
>>> and after changing <y> <my:value> 5 the data are:
>>>
>>> <x> <my:sameValue> <y>
>>> <y> <my:value> 10
>>> <x> <my:value> 10
>>>
>>> Note that the <x> <my:value> 5 has been deleted.
>>>
>>> Is this possible?
>> Sure, it would work that way so long as the [<y> <my:value> 5] is only
>> present by virtue of the inference, not in the base graph (see below).
>>
>>> In general do rules recalculate the results when the data changes?--
>> Yes but deletes are not incremental, and the details differ between the
>> reasoners.
>>
>> The forward reasoners keeps a graph of partial bindings and if new
>> triples are added the additional inferred triples are computed
>> incrementally. If a triple is removed then the graph is marked unclean
>> (the "prepared" flag is reset) so at the next query (or next explicit
>> prepare() call) the inference will be restarted from scratch.
>>
>> So in your case, starting with a base model of:
>>
>>    <x> <my:sameValue> <y>
>>    <y> <my:value> 5
>>
>> then the above rule would generate a deduction of:
>>
>>    <x> <my:value> 5
>>
>> If you then delete:
>>
>>    <y> <my:value> 5
>>
>> and add:
>>
>>    <y> <my:value> 10
>>
>> Then, at the next query, the forward reasoner will throw away its
>> deductions graph and internal rete network and start over, it would then
>> generate:
>>
>>    <y> <my:value> 5
>>
>> However, if your base graph had:
>>
>>    <x> <my:sameValue> <y>
>>    <y> <my:value> 5
>>    <x> <my:value> 42
>>
>> then the rules won't remove that third triple so you would end up with:
>>
>>    <x> <my:sameValue> <y>
>>    <y> <my:value> 5
>>    <x> <my:value> 42
>>    <x> <my:value> 5
>>
>> For the backward rules then, unless you use tabling to cache partial
>> goals, the inference is done on demand at query time anyway so you'll
>> get the same answers as above and the work will be done again at each
>> query whatever you've done to the graph in the meantime.
>>
>> Dave
>>

Re: Reasoners

[Paul Houle <pa...@ontology2.com>]

In vanilla RDF,  you do not change 5 to 10,  you either delete the 5 and
add the 10 or you add the 10 then delete the five!  Maybe it all happens
in a transaction which might make the immediate states invisible to the
outside world.

I use the Jena Rules engine with user defined functions that create side
effects and it is a lot of fun:  I write RDF -> X conversion systems
this way,  and also have an asynchronous execution engine.  In both of
these cases I conciously or unconciously make everything a latch so
deletes do not happen,  but it could be a problem if I want to use
deletion,  I would not want to run the side effecting methods again if I
can at all help it.

For the most part I *do not* apply inference rules to a transactional or
archival database but rather I pull in one or more records and maybe
some data tables and do some reasoning to answer a question about some
topic and then use the answer or put it back in some database.  

I know Drools has better support for deletions,  so does the RDF
database from Ontotext.

What would be involved in supporting this in Jena?



-- 
  Paul Houle
  paul.houle@ontology2.com

On Mon, Jan 9, 2017, at 02:57 PM, Dave Reynolds wrote:
> Hi Claude,
> 
> On 09/01/17 10:55, Claude Warren wrote:
> > I am digging into reasoners for the first time in a long time and have a
> > couple of questions.
> >
> > What I am trying to do is the following:
> >
> > Assume data:
> >
> > <x> <my:sameValue> <y>
> > <y> <my:value> 5
> >
> > I want a rule that will produce
> >
> > <x> <my:value> 5
> 
> So something like:
> 
>     (?x my:sameValue ?y) (?y my:value ?v) -> (?x my:value ?v)
> 
> > but in addition if I change [<y> <my:value> 5] to [<y> <my:value> 10] I
> > want the reasoner to change [<x> <my:value> 5] to [<x> <my:value> 10]
> >
> > So after first reasoning data are:
> >
> > <x> <my:sameValue> <y>
> > <y> <my:value> 5
> > <x> <my:value> 5
> >
> > and after changing <y> <my:value> 5 the data are:
> >
> > <x> <my:sameValue> <y>
> > <y> <my:value> 10
> > <x> <my:value> 10
> >
> > Note that the <x> <my:value> 5 has been deleted.
> >
> > Is this possible?
> 
> Sure, it would work that way so long as the [<y> <my:value> 5] is only 
> present by virtue of the inference, not in the base graph (see below).
> 
> > In general do rules recalculate the results when the data changes?--
> 
> Yes but deletes are not incremental, and the details differ between the 
> reasoners.
> 
> The forward reasoners keeps a graph of partial bindings and if new 
> triples are added the additional inferred triples are computed 
> incrementally. If a triple is removed then the graph is marked unclean 
> (the "prepared" flag is reset) so at the next query (or next explicit 
> prepare() call) the inference will be restarted from scratch.
> 
> So in your case, starting with a base model of:
> 
>    <x> <my:sameValue> <y>
>    <y> <my:value> 5
> 
> then the above rule would generate a deduction of:
> 
>    <x> <my:value> 5
> 
> If you then delete:
> 
>    <y> <my:value> 5
> 
> and add:
> 
>    <y> <my:value> 10
> 
> Then, at the next query, the forward reasoner will throw away its 
> deductions graph and internal rete network and start over, it would then 
> generate:
> 
>    <y> <my:value> 5
> 
> However, if your base graph had:
> 
>    <x> <my:sameValue> <y>
>    <y> <my:value> 5
>    <x> <my:value> 42
> 
> then the rules won't remove that third triple so you would end up with:
> 
>    <x> <my:sameValue> <y>
>    <y> <my:value> 5
>    <x> <my:value> 42
>    <x> <my:value> 5
> 
> For the backward rules then, unless you use tabling to cache partial 
> goals, the inference is done on demand at query time anyway so you'll 
> get the same answers as above and the work will be done again at each 
> query whatever you've done to the graph in the meantime.
> 
> Dave
> 

Re: Reasoners

[Dave Reynolds <da...@gmail.com>]

Hi Claude,

On 09/01/17 10:55, Claude Warren wrote:
> I am digging into reasoners for the first time in a long time and have a
> couple of questions.
>
> What I am trying to do is the following:
>
> Assume data:
>
> <x> <my:sameValue> <y>
> <y> <my:value> 5
>
> I want a rule that will produce
>
> <x> <my:value> 5

So something like:

    (?x my:sameValue ?y) (?y my:value ?v) -> (?x my:value ?v)

> but in addition if I change [<y> <my:value> 5] to [<y> <my:value> 10] I
> want the reasoner to change [<x> <my:value> 5] to [<x> <my:value> 10]
>
> So after first reasoning data are:
>
> <x> <my:sameValue> <y>
> <y> <my:value> 5
> <x> <my:value> 5
>
> and after changing <y> <my:value> 5 the data are:
>
> <x> <my:sameValue> <y>
> <y> <my:value> 10
> <x> <my:value> 10
>
> Note that the <x> <my:value> 5 has been deleted.
>
> Is this possible?

Sure, it would work that way so long as the [<y> <my:value> 5] is only 
present by virtue of the inference, not in the base graph (see below).

> In general do rules recalculate the results when the data changes?--

Yes but deletes are not incremental, and the details differ between the 
reasoners.

The forward reasoners keeps a graph of partial bindings and if new 
triples are added the additional inferred triples are computed 
incrementally. If a triple is removed then the graph is marked unclean 
(the "prepared" flag is reset) so at the next query (or next explicit 
prepare() call) the inference will be restarted from scratch.

So in your case, starting with a base model of:

   <x> <my:sameValue> <y>
   <y> <my:value> 5

then the above rule would generate a deduction of:

   <x> <my:value> 5

If you then delete:

   <y> <my:value> 5

and add:

   <y> <my:value> 10

Then, at the next query, the forward reasoner will throw away its 
deductions graph and internal rete network and start over, it would then 
generate:

   <y> <my:value> 5

However, if your base graph had:

   <x> <my:sameValue> <y>
   <y> <my:value> 5
   <x> <my:value> 42

then the rules won't remove that third triple so you would end up with:

   <x> <my:sameValue> <y>
   <y> <my:value> 5
   <x> <my:value> 42
   <x> <my:value> 5

For the backward rules then, unless you use tabling to cache partial 
goals, the inference is done on demand at query time anyway so you'll 
get the same answers as above and the work will be done again at each 
query whatever you've done to the graph in the meantime.

Dave