You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jorg Heymans <jo...@gmail.com> on 2010/02/09 10:45:48 UTC

DIH: delta-import not working

Hi,

I am having problems getting the delta-import to work for my schema.
Following what i have found in the list, jira and the wiki below
configuration should just work but it doesn't.

<dataConfig>
  <dataSource name="ora" driver="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:@....." user="...." password="...."/>
  <dataSource name="orablob" type="FieldStreamDataSource" />
  <document name="mydocuments">
    <entity dataSource="ora" name="attachment" pk="id" query="select id,
bytes from attachment where application = 'MYAPP'"
      deltaImportQuery="select id, bytes from attachment where application =
'MYAPP' and id = '${dataimporter.attachment.id}'"
      deltaQuery="select id from attachment where application = 'MYAPP' and
modified_on &gt; to_date('${dataimporter.attachment.last_index_time}',
'yyyy-mm-dd hh24:mi:ss')">
      <field column="id" name="attachmentId" />
      <entity dataSource="orablob" processor="TikaEntityProcessor"
url="bytes" dataField="attachment.bytes">
        <field column="text" name="attachmentContents"/>
      </entity>
    </entity>
  </document>
</dataConfig>

The sql generated in the deltaquery is correct, the timestamp is passed
correctly. When i execute that query manually in the DB it returns the pk of
the rows that were added. However no documents are added to the index. What
am i missing here ?? I'm using a build snapshot from 03/02.


Thanks
Jorg

Re: DIH: delta-import not working

Posted by Jorg Heymans <jo...@gmail.com>.
indeed that made it work. Looking back at the documentation, it's all there
but one needs to read every single line with care :-)

2010/2/9 Noble Paul നോബിള്‍ नोब्ळ् <no...@corp.aol.com>

> try this
>
> deltaImportQuery="select id, bytes from attachment where application =
>  'MYAPP' and id = '${dataimporter.delta.id}'"
>
> be aware that the names are case sensitive . if the id comes as 'ID'
> this will not work
>
>
>
> On Tue, Feb 9, 2010 at 3:15 PM, Jorg Heymans <jo...@gmail.com>
> wrote:
> > Hi,
> >
> > I am having problems getting the delta-import to work for my schema.
> > Following what i have found in the list, jira and the wiki below
> > configuration should just work but it doesn't.
> >
> > <dataConfig>
> >  <dataSource name="ora" driver="oracle.jdbc.OracleDriver"
> > url="jdbc:oracle:thin:@....." user="...." password="...."/>
> >  <dataSource name="orablob" type="FieldStreamDataSource" />
> >  <document name="mydocuments">
> >    <entity dataSource="ora" name="attachment" pk="id" query="select id,
> > bytes from attachment where application = 'MYAPP'"
> >      deltaImportQuery="select id, bytes from attachment where application
> =
> > 'MYAPP' and id = '${dataimporter.attachment.id}'"
> >      deltaQuery="select id from attachment where application = 'MYAPP'
> and
> > modified_on &gt; to_date('${dataimporter.attachment.last_index_time}',
> > 'yyyy-mm-dd hh24:mi:ss')">
> >      <field column="id" name="attachmentId" />
> >      <entity dataSource="orablob" processor="TikaEntityProcessor"
> > url="bytes" dataField="attachment.bytes">
> >        <field column="text" name="attachmentContents"/>
> >      </entity>
> >    </entity>
> >  </document>
> > </dataConfig>
> >
> > The sql generated in the deltaquery is correct, the timestamp is passed
> > correctly. When i execute that query manually in the DB it returns the pk
> of
> > the rows that were added. However no documents are added to the index.
> What
> > am i missing here ?? I'm using a build snapshot from 03/02.
> >
> >
> > Thanks
> > Jorg
> >
>
>
>
> --
> -----------------------------------------------------
> Noble Paul | Systems Architect| AOL | http://aol.com
>

Re: DIH: delta-import not working

Posted by Noble Paul നോബിള്‍ नोब्ळ् <no...@corp.aol.com>.
try this

deltaImportQuery="select id, bytes from attachment where application =
 'MYAPP' and id = '${dataimporter.delta.id}'"

be aware that the names are case sensitive . if the id comes as 'ID'
this will not work



On Tue, Feb 9, 2010 at 3:15 PM, Jorg Heymans <jo...@gmail.com> wrote:
> Hi,
>
> I am having problems getting the delta-import to work for my schema.
> Following what i have found in the list, jira and the wiki below
> configuration should just work but it doesn't.
>
> <dataConfig>
>  <dataSource name="ora" driver="oracle.jdbc.OracleDriver"
> url="jdbc:oracle:thin:@....." user="...." password="...."/>
>  <dataSource name="orablob" type="FieldStreamDataSource" />
>  <document name="mydocuments">
>    <entity dataSource="ora" name="attachment" pk="id" query="select id,
> bytes from attachment where application = 'MYAPP'"
>      deltaImportQuery="select id, bytes from attachment where application =
> 'MYAPP' and id = '${dataimporter.attachment.id}'"
>      deltaQuery="select id from attachment where application = 'MYAPP' and
> modified_on &gt; to_date('${dataimporter.attachment.last_index_time}',
> 'yyyy-mm-dd hh24:mi:ss')">
>      <field column="id" name="attachmentId" />
>      <entity dataSource="orablob" processor="TikaEntityProcessor"
> url="bytes" dataField="attachment.bytes">
>        <field column="text" name="attachmentContents"/>
>      </entity>
>    </entity>
>  </document>
> </dataConfig>
>
> The sql generated in the deltaquery is correct, the timestamp is passed
> correctly. When i execute that query manually in the DB it returns the pk of
> the rows that were added. However no documents are added to the index. What
> am i missing here ?? I'm using a build snapshot from 03/02.
>
>
> Thanks
> Jorg
>



-- 
-----------------------------------------------------
Noble Paul | Systems Architect| AOL | http://aol.com