You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Souvik Chakraborty <so...@nrifintech.com> on 2013/12/23 04:39:24 UTC

Importing from Multiple tables using Solr DIH

Hi all,

 

I have the below configuration which is working absolutely fine.

 

 

Data-config.xml:

 

<?xml version="1.0" encoding="UTF-8" ?>

<dataConfig>

<dataSource name="db" driver="oracle.jdbc.driver.OracleDriver"
url="jdbc:oracle:thin:@//172.16.29.92:1521/d11gr21" user="eig_nbl_dev"
password="eig_nbl_dev"/>

<dataSource name="dastream" type="FieldStreamDataSource" />

<document>

  <entity 

      name="messages" pk="BLOB_PK" transformer='DateFormatTransformer'

      query="select * from BLOB_TEST"

                  deltaQuery="select BLOB_PK from BLOB_TEST where
to_char(last_modified,'YYYY-MM-DD HH24:MI:SS') &gt;
'${dataimporter.last_index_time}' "

      dataSource="db">

                <field column ="BLOB_PK" name ="id" />

                <field column="last_modified"  dateTimeFormat="YYYY-MM-DD
HH24:MI:SS" locale="en" />

    <entity 

        name="message" 

        dataSource="dastream"

        processor="TikaEntityProcessor"

        url="message"

        dataField="messages.MESSAGE"

        format="text">

                                

        <field column="text" name="mxMsg" blob="true" />

      </entity>

    </entity> 

 </document>

</dataConfig>

 

Now my requirement is that I have a similar table BLOB_TEST1 and have the
same fields as that of BLOB_TEST.

I wish to index it the same way as I have done it for BLOB_TEST.

Can't figure out how to accomplish that.

Any help would be highly appreciated.

 

-Souvik

Re: Importing from Multiple tables using Solr DIH

Posted by Ahmet Arslan <io...@yahoo.com>.
Hi

There are several ways to do it. One way is to create to two entities at the same level. Use its name to call it. 

Request : command=full-import&entity=messages_test

<document>

  <entity name="messages_test" query="select * from BLOB_TEST"

   ...
  </entity>
  
  <entity name="messages_test1" query="select * from BLOB_TEST1"
   ....
  </entity>
</document>

 

Other way : Make table name variable in data-config.xml.  And change/set it (BLOB_TEST or BLOB_TEST1) from request parameters.

<entity name="messages_test" query="select * from '${dataimporter.request.myTableName}'"


Request : command=full-import&myTableName=BLOB_TEST1


http://wiki.apache.org/solr/DataImportHandler#Accessing_request_parameters





On Monday, December 23, 2013 5:40 AM, Souvik Chakraborty <so...@nrifintech.com> wrote:
Hi all,



I have the below configuration which is working absolutely fine.





Data-config.xml:



<?xml version="1.0" encoding="UTF-8" ?>

<dataConfig>

<dataSource name="db" driver="oracle.jdbc.driver.OracleDriver"
url="jdbc:oracle:thin:@//172.16.29.92:1521/d11gr21" user="eig_nbl_dev"
password="eig_nbl_dev"/>

<dataSource name="dastream" type="FieldStreamDataSource" />

<document>

  <entity 

      name="messages" pk="BLOB_PK" transformer='DateFormatTransformer'

      query="select * from BLOB_TEST"

                  deltaQuery="select BLOB_PK from BLOB_TEST where
to_char(last_modified,'YYYY-MM-DD HH24:MI:SS') &gt;
'${dataimporter.last_index_time}' "

      dataSource="db">

                <field column ="BLOB_PK" name ="id" />

                <field column="last_modified"  dateTimeFormat="YYYY-MM-DD
HH24:MI:SS" locale="en" />

    <entity 

        name="message" 

        dataSource="dastream"

        processor="TikaEntityProcessor"

        url="message"

        dataField="messages.MESSAGE"

        format="text">

                                

        <field column="text" name="mxMsg" blob="true" />

      </entity>

    </entity> 

</document>

</dataConfig>



Now my requirement is that I have a similar table BLOB_TEST1 and have the
same fields as that of BLOB_TEST.

I wish to index it the same way as I have done it for BLOB_TEST.

Can't figure out how to accomplish that.

Any help would be highly appreciated.



-Souvik