data import / delta question

[Tim Heckman <th...@gmail.com>]

My data-import-config.xml has a parent entity and a child entity. The
data is coming from rdbms's.

I'm trying to make use of the delta-import feature where a change in
the child entity can be used to regenerate the entire document.

The child entity is on a different database (and a different server)
from the parent entity, so the child's parentDeltaQuery cannot
reference the table of the parent entity the way that the example on
the wiki does, because it's bound to the database connection for the
child entity's data (as far as I can tell).

http://wiki.apache.org/solr/DataImportHandler#Using_delta-import_command


I have tried extracting the parent's ID's from the child table in the
parentDeltaQuery, thinking that these id's would be fed into the
parent's deltaImportQuery, but this doesn't seem to work, either.

Should this work? If not, any suggestions how to work around it?

thanks,
Tim

Re: data import / delta question

[Tim Heckman <th...@gmail.com>]

Thanks, Erick. I was starting to think I may have to go the SolrJ route.

Here's a simplified version of my DIH config showing what I'm trying to do.


<dataConfig>
    <dataSource name="PROD" type="JdbcDataSource"
      driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"
      url="jdbc:sqlserver://PRODSERVER:1433;databaseName=XXXX"
      />

    <dataSource name="IMG" type="JdbcDataSource"
      driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"
      url="jdbc:sqlserver://IMGDBSERVER:1433;databaseName=XXXX"
      />

  <document name="fullProduct">

        <entity name="product" datasouce="PROD" pk="upc"
            query=" [ big select goes here ] "

            deltaImportQuery=" [big select goes here]
            where product.upc = '${dataimporter.delta.upc}' " >

                <entity name="coverimage" dataSource="IMG" pk="upc, imageid"
                    query=" select top 1  pm.imageid
                        FROM 	productImageLink as pm
                        WHERE 	pm.ean = ${product.upc}
                        AND pm.isPrimary = 1"

                    deltaQuery=" select upc, imageid
                        from productImageLink pm with (nolock)
                        where pm.isprimary = 1
                        and upddtime > '${dataimporter.last_index_time}' "

                    parentDeltaQuery=" select distinct upc
                        from productImageLink pm with (nolock)
                        where pm.isprimary = 1
                        and upddtime > '${dataimporter.last_index_time}' " />
        </entity>
    </document>
</dataConfig>



On Mon, Oct 11, 2010 at 4:25 PM, Erick Erickson <er...@gmail.com> wrote:
> Without seeing your DIH config, it's really hard to say much of anything.
>
> You can gain finer control over edge cases by writing a Java
> app that uses SolrJ if necessary.
>
> HTH
> Erick
>
> On Mon, Oct 11, 2010 at 3:27 PM, Tim Heckman <th...@gmail.com> wrote:
>
>> My data-import-config.xml has a parent entity and a child entity. The
>> data is coming from rdbms's.
>>
>> I'm trying to make use of the delta-import feature where a change in
>> the child entity can be used to regenerate the entire document.
>>
>> The child entity is on a different database (and a different server)
>> from the parent entity, so the child's parentDeltaQuery cannot
>> reference the table of the parent entity the way that the example on
>> the wiki does, because it's bound to the database connection for the
>> child entity's data (as far as I can tell).
>>
>> http://wiki.apache.org/solr/DataImportHandler#Using_delta-import_command
>>
>>
>> I have tried extracting the parent's ID's from the child table in the
>> parentDeltaQuery, thinking that these id's would be fed into the
>> parent's deltaImportQuery, but this doesn't seem to work, either.
>>
>> Should this work? If not, any suggestions how to work around it?
>>
>> thanks,
>> Tim
>>
>

Re: data import / delta question

[Erick Erickson <er...@gmail.com>]

Without seeing your DIH config, it's really hard to say much of anything.

You can gain finer control over edge cases by writing a Java
app that uses SolrJ if necessary.

HTH
Erick

On Mon, Oct 11, 2010 at 3:27 PM, Tim Heckman <th...@gmail.com> wrote:

> My data-import-config.xml has a parent entity and a child entity. The
> data is coming from rdbms's.
>
> I'm trying to make use of the delta-import feature where a change in
> the child entity can be used to regenerate the entire document.
>
> The child entity is on a different database (and a different server)
> from the parent entity, so the child's parentDeltaQuery cannot
> reference the table of the parent entity the way that the example on
> the wiki does, because it's bound to the database connection for the
> child entity's data (as far as I can tell).
>
> http://wiki.apache.org/solr/DataImportHandler#Using_delta-import_command
>
>
> I have tried extracting the parent's ID's from the child table in the
> parentDeltaQuery, thinking that these id's would be fed into the
> parent's deltaImportQuery, but this doesn't seem to work, either.
>
> Should this work? If not, any suggestions how to work around it?
>
> thanks,
> Tim
>