[jira] [Commented] (JENA-686) Enable configuration of custom doc producers

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/JENA-686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526456#comment-14526456 ] 

ASF GitHub Bot commented on JENA-686:
-------------------------------------

Github user osma commented on the pull request:

    https://github.com/apache/jena/pull/53#issuecomment-98649858
  
    > Indeed, both of them will be dropped (however, can be solved with multi language proposal)
    
    I'm curious, how would the multi language proposal help with this problem?
    
    > I don't think that storing the original literal in non-debug mode is a good way to follow. It will result in a huge index and will affect performance.
    > Instead of that, storing a smaller identifier generated from a hash of a key (like uri+literal+language) will surely be a better solution.
    
    I agree. A hash with sufficient uniqueness guarantees could work here. I think it should incorporate all parts of the quad (subject, predicate, literal value + language, named graph). This way, the entries in the Lucene index would correspond exactly to quads. When a quad gets deleted, the corresponding entry can be uniquely identified via the hash and deleted from the Lucene index as well.
    
    But I'm not sure how this would interact with the recent work done by Chris Dollin (@ehedgehog) on cross field conjunctive queries (pull request #42 and [JENA-686](https://issues.apache.org/jira/browse/JENA-686)). In that work, the Lucene index is modified so that entries ("documents") in the Lucene index do not correspond to triples/quads, but instead there is one entry per subject resource (which may incorporate information from several triples/quads), allowing more efficient conjunctive queries on the Lucene index level. How would you deal with deletions in such a scenario?


> Enable configuration of custom doc producers 
> ---------------------------------------------
>
>                 Key: JENA-686
>                 URL: https://issues.apache.org/jira/browse/JENA-686
>             Project: Apache Jena
>          Issue Type: Improvement
>          Components: Text
>    Affects Versions: Jena 2.11.1, Fuseki 1.0.1
>            Reporter: Brian McBride
>            Assignee: christopher james dollin
>         Attachments: TestDatasetWithBatchProducer.java
>
>
> We have a project where we are doing text search on addresses and wish to do jena text queries like "city:liverpool AND street:green".  These queries return no results, whilst queries like "street:green AND street:lane" work fine.
> The reason is that jena text indexes each property in a separate Lucene document, so there is no Lucene document matching city:liverpool AND street:green, there are two documents, one for each property.
> Given the scale of our data, we really want to do the conjunctive query in Lucene and not two separate queries and then a filter in SPARQL.
> I will attach a test case from an attempt to solve this for us to illustrate.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)