You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jon Drukman <jd...@gmail.com> on 2010/02/01 21:15:29 UTC
DataImportHandler delta-import confusion
First, let me just say that DataImportHandler is fantastic. It got my
old mysql-php-xml index rebuild process down from 30 hours to 6 minutes.
I'm trying to use the delta-import functionality now but failing miserably.
Here's my entity tag: (some SELECT statements reduced to increase
readability)
<entity name="moment"
query="select ..."
deltaQuery="select moment_id from moments where date_modified >
'${dataimporter.last_index_time}'"
deltaImportQuery="select [bunch of stuff]
WHERE m.moment_id = '${dataimporter.delta.MOMENTID}'"
pk="MOMENTID"
transformer="TemplateTransformer">
When I look at the MySQL query log I see the date modified query running
fine and returning 3 rows. The deltaImportQuery, however, does not have
the proper primary key in the where clause. It's just blank. I also
tried changing it to ${moment.MOMENTID}.
I don't really get the relation between the pk field and the
${dataimport.delta.whatever} stuff.
Help please!
-jsd-
Re: DataImportHandler delta-import confusion
Posted by Noble Paul നോബിള് नोब्ळ् <no...@corp.aol.com>.
Please do not hijack a thread. http://people.apache.org/~hossman/#threadhijack
On Tue, Feb 2, 2010 at 11:32 PM, Leann Pereira
<le...@1sourcestaffing.com> wrote:
> Hi Paul,
>
> Can you take me off this distribution list?
>
> Thanks,
>
> Leann
>
> ________________________________________
> From: noble.paul@gmail.com [noble.paul@gmail.com] On Behalf Of Noble Paul നോബിള് नोब्ळ् [noble.paul@corp.aol.com]
> Sent: Tuesday, February 02, 2010 2:12 AM
> To: solr-user@lucene.apache.org
> Subject: Re: DataImportHandler delta-import confusion
>
> try
> deltaImportQuery="select [bunch of stuff]
> WHERE m.moment_id = '${dataimporter.delta.moment_id}'"
>
> The key has to be same and in the same case
>
> On Tue, Feb 2, 2010 at 1:45 AM, Jon Drukman <jd...@gmail.com> wrote:
>> First, let me just say that DataImportHandler is fantastic. It got my old
>> mysql-php-xml index rebuild process down from 30 hours to 6 minutes.
>>
>> I'm trying to use the delta-import functionality now but failing miserably.
>>
>> Here's my entity tag: (some SELECT statements reduced to increase
>> readability)
>>
>> <entity name="moment"
>> query="select ..."
>>
>> deltaQuery="select moment_id from moments where date_modified >
>> '${dataimporter.last_index_time}'"
>>
>> deltaImportQuery="select [bunch of stuff]
>> WHERE m.moment_id = '${dataimporter.delta.MOMENTID}'"
>>
>> pk="MOMENTID"
>>
>> transformer="TemplateTransformer">
>>
>> When I look at the MySQL query log I see the date modified query running
>> fine and returning 3 rows. The deltaImportQuery, however, does not have the
>> proper primary key in the where clause. It's just blank. I also tried
>> changing it to ${moment.MOMENTID}.
>>
>> I don't really get the relation between the pk field and the
>> ${dataimport.delta.whatever} stuff.
>>
>> Help please!
>> -jsd-
>>
>>
>>
>
>
>
> --
> -----------------------------------------------------
> Noble Paul | Systems Architect| AOL | http://aol.com
--
-----------------------------------------------------
Noble Paul | Systems Architect| AOL | http://aol.com
RE: DataImportHandler delta-import confusion
Posted by Leann Pereira <le...@1sourcestaffing.com>.
Hi Paul,
Can you take me off this distribution list?
Thanks,
Leann
________________________________________
From: noble.paul@gmail.com [noble.paul@gmail.com] On Behalf Of Noble Paul നോബിള് नोब्ळ् [noble.paul@corp.aol.com]
Sent: Tuesday, February 02, 2010 2:12 AM
To: solr-user@lucene.apache.org
Subject: Re: DataImportHandler delta-import confusion
try
deltaImportQuery="select [bunch of stuff]
WHERE m.moment_id = '${dataimporter.delta.moment_id}'"
The key has to be same and in the same case
On Tue, Feb 2, 2010 at 1:45 AM, Jon Drukman <jd...@gmail.com> wrote:
> First, let me just say that DataImportHandler is fantastic. It got my old
> mysql-php-xml index rebuild process down from 30 hours to 6 minutes.
>
> I'm trying to use the delta-import functionality now but failing miserably.
>
> Here's my entity tag: (some SELECT statements reduced to increase
> readability)
>
> <entity name="moment"
> query="select ..."
>
> deltaQuery="select moment_id from moments where date_modified >
> '${dataimporter.last_index_time}'"
>
> deltaImportQuery="select [bunch of stuff]
> WHERE m.moment_id = '${dataimporter.delta.MOMENTID}'"
>
> pk="MOMENTID"
>
> transformer="TemplateTransformer">
>
> When I look at the MySQL query log I see the date modified query running
> fine and returning 3 rows. The deltaImportQuery, however, does not have the
> proper primary key in the where clause. It's just blank. I also tried
> changing it to ${moment.MOMENTID}.
>
> I don't really get the relation between the pk field and the
> ${dataimport.delta.whatever} stuff.
>
> Help please!
> -jsd-
>
>
>
--
-----------------------------------------------------
Noble Paul | Systems Architect| AOL | http://aol.com
Re: DataImportHandler delta-import confusion
Posted by Noble Paul നോബിള് नोब्ळ् <no...@corp.aol.com>.
try
deltaImportQuery="select [bunch of stuff]
WHERE m.moment_id = '${dataimporter.delta.moment_id}'"
The key has to be same and in the same case
On Tue, Feb 2, 2010 at 1:45 AM, Jon Drukman <jd...@gmail.com> wrote:
> First, let me just say that DataImportHandler is fantastic. It got my old
> mysql-php-xml index rebuild process down from 30 hours to 6 minutes.
>
> I'm trying to use the delta-import functionality now but failing miserably.
>
> Here's my entity tag: (some SELECT statements reduced to increase
> readability)
>
> <entity name="moment"
> query="select ..."
>
> deltaQuery="select moment_id from moments where date_modified >
> '${dataimporter.last_index_time}'"
>
> deltaImportQuery="select [bunch of stuff]
> WHERE m.moment_id = '${dataimporter.delta.MOMENTID}'"
>
> pk="MOMENTID"
>
> transformer="TemplateTransformer">
>
> When I look at the MySQL query log I see the date modified query running
> fine and returning 3 rows. The deltaImportQuery, however, does not have the
> proper primary key in the where clause. It's just blank. I also tried
> changing it to ${moment.MOMENTID}.
>
> I don't really get the relation between the pk field and the
> ${dataimport.delta.whatever} stuff.
>
> Help please!
> -jsd-
>
>
>
--
-----------------------------------------------------
Noble Paul | Systems Architect| AOL | http://aol.com