DeltaImport records not commited using DIH.

[Anandha Ranganathan <an...@gmail.com>]

Hi,


1) When I run full-import it works fine and commits all the records. The
document count matches table and dataImport.properties is updated with
last_index timestamp.
2)  After some time I ran the delta import and it is giving enough
information but it is not adding the new record into the index.

 I am including my config and log information. Could anyone help me to fix
this.

*Data-config*

<dataConfig>
  <document>

    <entity name="cps_dataset" pk="PP_ID"
      dataSource="xxx_xxx_dataset"
      query="     select * from tableA"
           deltaQuery="select PP_ID from tableA where update_date &gt;
TO_DATE('${dataimporter.last_index_time}', 'yyyy/mm/dd hh24:mi:ss')"
           deltaImportQuery="  select * from tableA where
PP_ID='${dataImporter.delta.PP_ID}'"       >

        <field column="PP_ID" name="id"/>
        <field column="CONTACT_ID" name="conactId"/>
        <field column="orgId" name="orgId"/>

        </entity>

  </document>
</dataConfig>



*DeltaImport Status message.*

<lst name="statusMessages">
<str name="Time Elapsed">0:0:3.391</str>
<str name="Total Requests made to DataSource">15</str>
<str name="Total Rows Fetched">77</str>
<str name="Total Documents Processed">0</str>
<str name="Total Documents Skipped">0</str>
<str name="Delta Dump started">2011-06-02 16:58:23</str>
<str name="Identifying Delta">2011-06-02 16:58:23</str>
<str name="Deltas Obtained">2011-06-02 16:58:24</str>
<str name="Building documents">2011-06-02 16:58:24</str>
<str name="Total Changed Documents">77</str>
</lst>


*Log info*
Jun 2, 2011 3:59:20 PM org.apache.solr.handler.dataimport.DocBuilder execute
INFO: Time taken = 0:0:15.641
Jun 2, 2011 4:01:02 PM org.apache.solr.handler.dataimport.DataImporter
doDeltaImport
INFO: Starting Delta Import
Jun 2, 2011 4:01:02 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select
params={command=delta-import&clean=false&qt=/dataimport&commit=true}
status=0 QTime=0
Jun 2, 2011 4:01:02 PM org.apache.solr.handler.dataimport.SolrWriter
readIndexerProperties
INFO: Read dataimport.properties
Jun 2, 2011 4:01:02 PM org.apache.solr.handler.dataimport.DocBuilder doDelta
INFO: Starting delta collection.
Jun 2, 2011 4:01:02 PM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Running ModifiedRowKey() for Entity: cps_dataset
Jun 2, 2011 4:01:02 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
call
INFO: Creating a connection for entity cps_dataset with URL:
jdbc:oracle:thin:@lnxdb-stg-abcd.com:1521:xxxxSTG
Jun 2, 2011 4:01:03 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
call
INFO: Time taken for getConnection(): 484
Jun 2, 2011 4:01:03 PM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Completed ModifiedRowKey for Entity: xxx_xxx_cps_dataset rows obtained
: 77
Jun 2, 2011 4:01:03 PM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Completed DeletedRowKey for Entity: xxx_xxx_cps_dataset rows obtained
: 0
Jun 2, 2011 4:01:03 PM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Completed parentDeltaQuery for Entity: xxx_xxx_dataset
Jun 2, 2011 4:01:18 PM org.apache.solr.handler.dataimport.DocBuilder doDelta
INFO: Delta Import completed successfully
Jun 2, 2011 4:01:18 PM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {} 0 0
Jun 2, 2011 4:01:18 PM org.apache.solr.handler.dataimport.DocBuilder execute
INFO: Time taken = 0:0:15.625