[jira] [Commented] (SOLR-3946) Support delta import in SolrEntityProcessor

["James Dyer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476237#comment-13476237 ] 

James Dyer commented on SOLR-3946:
----------------------------------

It might be difficult to make "command=delta-import" work with anything other than SqlEntityProcessor, as it seems to be designed around SQL and RDBMS concepts.  However, you might be able to do deltas with SolrEntityProcessor using "command=full-import&clean=false".  Then, parameterize SolrEntityProcessor's "query" and/or "fq" parameters to retrieve just the documents that were added or changed since the last sync.  Of course deletes are going to be a problem, and you might need to invent some multiple-step process to find a way to do these. 

Given that you can do incremental updates on your index using "command=full-import&clean=false", and that the Delta Update is unsupported (indeed often cannot be supported) for anything other than Sql, I wonder if "command=delta-update" could just be removed entirely from DIH.  As DIH is slipping more and more towards death, it might someday be necessary to amputate the sickest parts to save the patient...
                
> Support delta import in SolrEntityProcessor
> -------------------------------------------
>
>                 Key: SOLR-3946
>                 URL: https://issues.apache.org/jira/browse/SOLR-3946
>             Project: Solr
>          Issue Type: Improvement
>          Components: clients - java
>    Affects Versions: 4.0
>            Reporter: yuanyun.cn
>            Priority: Minor
>              Labels: SolrEntityProcessor, deltaimport
>             Fix For: 4.1
>
>
> SolrEntityProcessor is very useful to copy a part of index from central solr to another solr server based on some query.
> But its function is quite limited, doesn't support delta import, which is a quite useful feature, for example:
> One central solr server stores index of all docs, in the index we record information such as owner, last_modified and etc. Then create a local cache solr server in client side which only contains index of docs created by this user, so user can search his/her docs even when there is no internet connection. After the first full import to copy index of doc created by this user in last several weeks (or month), we want to update index in client's local solr server consistently from the central server.
> But now, we can't do this, as SolrEntityProcessor doesn't support delta-import - which already supports in SqlEntityProcessor: using deltaQuery, deltaImportQuery to do delta-import, using deletedPkQuery to remove deleted index when do delta-import.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org