RE: CachedSqlEntityProcessor with delta-import

["Mohan, Sowmya" <So...@icf.com>]

Thanks. We did implement the delete by query on another core and thought of giving the delta import a try here. Looks like differential via full index and deletes using delete by id/query is the way to go. 

-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com] 
Sent: Tuesday, October 25, 2016 12:31 PM
To: solr-user <so...@lucene.apache.org>
Subject: Re: CachedSqlEntityProcessor with delta-import

Why not use delete by id rather than query? It'll be more efficient....

Probably not a big deal though.

On Tue, Oct 25, 2016 at 1:47 AM, Aniket Khare <an...@gmail.com> wrote:
> Hi Sowmya,
>
> I my case I have implemeneted the data indexing suggested by James and 
> for deleting the reords I have created my own data indexing job which 
> will call the delete API periodically by passing the list of unique Id.
> https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+I
> ndex+Handlers
>
> http://localhost:8983/solr/update?stream.body=
> <delete><query>id:1234</query></delete>&commit=true
>
> Thanks,
> Aniket S. Khare
>
> On Tue, Oct 25, 2016 at 1:32 AM, Mohan, Sowmya <So...@icf.com> wrote:
>
>> Thanks James. That's what I was using before. But I also wanted to 
>> perform deletes using deletedPkQuery and hence switched to delta 
>> imports. The problem with using deletedPkQuery with the full import 
>> is that dataimporter.last_index_time is no longer accurate.
>>
>> Below is an example of my deletedPkQuery. If run the full-import for 
>> a differential index, that would update the last index time. Running 
>> the delta import to remove the deleted records then wouldn't do 
>> anything since nothing changed since the last index time.
>>
>>
>>  deletedPkQuery="SELECT id
>>                         FROM content
>>                         WHERE active = 1 AND lastUpdate > 
>> '${dataimporter.last_index_time}'"
>>
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: Dyer, James [mailto:James.Dyer@ingramcontent.com]
>> Sent: Friday, October 21, 2016 4:23 PM
>> To: solr-user@lucene.apache.org
>> Subject: RE: CachedSqlEntityProcessor with delta-import
>>
>> Sowmya,
>>
>> My memory is that the cache feature does not work with Delta Imports.  
>> In fact, I believe that nearly all DIH features except straight JDBC 
>> imports do not work with Delta Imports.  My advice is to not use the 
>> Delta Import feature at all as the same result can (often 
>> more-efficiently) be accomplished following the approach outlined here:
>> https://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport
>>
>> James Dyer
>> Ingram Content Group
>>
>> -----Original Message-----
>> From: Mohan, Sowmya [mailto:Sowmya.Mohan@icf.com]
>> Sent: Tuesday, October 18, 2016 10:07 AM
>> To: solr-user@lucene.apache.org
>> Subject: CachedSqlEntityProcessor with delta-import
>>
>> Good morning,
>>
>> Can CachedSqlEntityProcessor be used with delta-import? In my setup 
>> when running a delta-import with CachedSqlEntityProcessor, the child 
>> entity values are not correctly updated for the parent record. I am on Solr 4.3.
>> Has anyone experienced this and if so how to resolve it?
>>
>> Thanks,
>> Sowmya.
>>
>>
>
>
> --
> Regards,
>
> Aniket S. Khare