Re: Performance issue in indexing the data with DIH when using subqueries

[Shalin Shekhar Mangar <sh...@gmail.com>]

On Tue, Feb 23, 2010 at 1:01 AM, JavaGuy84 <bb...@gmail.com> wrote:

>
> Hi,
>
> I am facing a performace issue when I am trying to index the data using
> DIH.. I have a model as below
>
> Tables
> --------
> Object
> ObjectProperty
> ObjectRelationship
>
>
> Object --> ObjectProperty ---- one to Many Relationship
> Object --> ObjectRelationship ---- one to Many Relationship
>
> We need to get the object and its related properties / relationship in a
> single document.
>
> So as of now I have a outer query in DIH config which loops over the
> objectid and for each objectid my inner query retreives the data from
> objectrelationship/objectproperty table.
>
> The performace seems to be very bad (took 4+ minutes to index 4000 rows /
> 590 documents) and I am trying to figure out a way to improve the
> performance.
>
> It would be great if someone can give me a suggestion on how to overcome /
> work around to this problem.
>
>
Have you tried using CachedSqlEntityProcessor?

See http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor

-- 
Regards,
Shalin Shekhar Mangar.

Re: Performance issue in indexing the data with DIH when using subqueries

[JavaGuy84 <bb...@gmail.com>]

Thanks a lot Shalin.. This resolve my issue :).


Thanks,
Barani

Shalin Shekhar Mangar wrote:
> 
> On Tue, Feb 23, 2010 at 1:01 AM, JavaGuy84 <bb...@gmail.com> wrote:
> 
>>
>> Hi,
>>
>> I am facing a performace issue when I am trying to index the data using
>> DIH.. I have a model as below
>>
>> Tables
>> --------
>> Object
>> ObjectProperty
>> ObjectRelationship
>>
>>
>> Object --> ObjectProperty ---- one to Many Relationship
>> Object --> ObjectRelationship ---- one to Many Relationship
>>
>> We need to get the object and its related properties / relationship in a
>> single document.
>>
>> So as of now I have a outer query in DIH config which loops over the
>> objectid and for each objectid my inner query retreives the data from
>> objectrelationship/objectproperty table.
>>
>> The performace seems to be very bad (took 4+ minutes to index 4000 rows /
>> 590 documents) and I am trying to figure out a way to improve the
>> performance.
>>
>> It would be great if someone can give me a suggestion on how to overcome
>> /
>> work around to this problem.
>>
>>
> Have you tried using CachedSqlEntityProcessor?
> 
> See http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor
> 
> -- 
> Regards,
> Shalin Shekhar Mangar.
> 
> 

-- 
View this message in context: http://old.nabble.com/Performance-issue-in-indexing-the-data-with-DIH-when-using-subqueries-tp27692967p27714484.html
Sent from the Solr - User mailing list archive at Nabble.com.