[GitHub] jena pull request: Lucene index synchro on triple deletion (jena-t...

[osma <gi...@git.apache.org>]

Github user osma commented on the pull request:

    https://github.com/apache/jena/pull/53#issuecomment-98649858
  
    > Indeed, both of them will be dropped (however, can be solved with multi language proposal)
    
    I'm curious, how would the multi language proposal help with this problem?
    
    > I don't think that storing the original literal in non-debug mode is a good way to follow. It will result in a huge index and will affect performance.
    > Instead of that, storing a smaller identifier generated from a hash of a key (like uri+literal+language) will surely be a better solution.
    
    I agree. A hash with sufficient uniqueness guarantees could work here. I think it should incorporate all parts of the quad (subject, predicate, literal value + language, named graph). This way, the entries in the Lucene index would correspond exactly to quads. When a quad gets deleted, the corresponding entry can be uniquely identified via the hash and deleted from the Lucene index as well.
    
    But I'm not sure how this would interact with the recent work done by Chris Dollin (@ehedgehog) on cross field conjunctive queries (pull request #42 and [JENA-686](https://issues.apache.org/jira/browse/JENA-686)). In that work, the Lucene index is modified so that entries ("documents") in the Lucene index do not correspond to triples/quads, but instead there is one entry per subject resource (which may incorporate information from several triples/quads), allowing more efficient conjunctive queries on the Lucene index level. How would you deal with deletions in such a scenario?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---