You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Thomas Lustig <tm...@gmail.com> on 2018/05/27 18:47:43 UTC

delta-update alternative on filechanges when using FileListEntityProcessor

I configured a DataImportHandler using a FileListEntityProcessor to import
files from a folder.
This setup works really great, but i do not now how i should handle changes
on the filesystem (e.g. files added, deleted,...)
Should I always do a "full-import"? As far as i read "delta-import" is only
supported by SqlEntityProcessor.
Is there a best practise, that is recommended?
Thanks in advance for helping me

Br
Tom

Re: delta-update alternative on filechanges when using FileListEntityProcessor

Posted by Alexandre Rafalovitch <ar...@gmail.com>.
The best practice is not to use DIH in production. It is great for several
rounds of prototyping but then things get messy and uneven as you found.
The delete logic is always extra messy.

So, this may be a good point to switch to an external client and implement
the monitoring logic there.

Regards,
    Alex
P.s. or you could reindex everything periodically in a separate collection
and swap it into production. No delete logic.....

On Sun, May 27, 2018, 2:48 PM Thomas Lustig, <tm...@gmail.com> wrote:

> I configured a DataImportHandler using a FileListEntityProcessor to import
> files from a folder.
> This setup works really great, but i do not now how i should handle changes
> on the filesystem (e.g. files added, deleted,...)
> Should I always do a "full-import"? As far as i read "delta-import" is only
> supported by SqlEntityProcessor.
> Is there a best practise, that is recommended?
> Thanks in advance for helping me
>
> Br
> Tom
>