[jira] [Created] (SOLR-7952) Change DeltaImport from HashSet to LinkedHashSet.

["Pablo Lozano (JIRA)" <ji...@apache.org>]

Pablo Lozano created SOLR-7952:
----------------------------------

             Summary: Change DeltaImport from HashSet to LinkedHashSet.
                 Key: SOLR-7952
                 URL: https://issues.apache.org/jira/browse/SOLR-7952
             Project: Solr
          Issue Type: Improvement
          Components: contrib - DataImportHandler
    Affects Versions: 5.2.1
            Reporter: Pablo Lozano
            Priority: Minor


This is only a minor modification which on some cases might be useful for certain custom DataSources or ImportHandlers.

The way my imports work is by fetching on batches, So I need to store those batches on a disk cache for a certain time as they are not required on the mean time.

And also use some lazy loading as my batches are not initialized by my custom iterators until the time they are iterated for the first time,

My issue comes from that the order in which I pass the ids of my documents to the ImporHandler during the "FIND_DELTA" step is not the same order they are tried to be fetch during the DELTA_DUMP step. It causes my batches to be initialized when only one of them could be done at a time.

What I would like is to simply change the HashSet used on the "collectDelta" method to a LinkedHashSet. This would help as we would obtain a predictable order of documents.

This may be a very specific case but the change is simple and shouldn't impact on anything.

The second option would be to create a "deltaImportQuery" like that would work like:" select * from table where last_modified > '${dih.last_index_time}'".

I can issue the patch for this.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org