Performance problem with DIH in solr 3.3

[Pravin Agrawal <Pr...@persistent.co.in>]

Hi All,

I am using Delta import handler(solr 3.3) to index data from my database (using 19 tables)
  Total Number of solr documents that get created from these 19 table is 444
  Total number of request send to data source during clean full import is 91083.

 My problem is that, DIH makes too many calls and puts load on my database.
  1. Can we batch these calls ?
  2. Can we use view instead? If yes can I get some examples to use view with DIH
  3. What kind of locks SOLR DIH acquire while querying DB?

Note: we are using both Full-import and delta-import handler.

Thanks in advance
Pravin Agrawal

DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

RE: Performance problem with DIH in solr 3.3

["Dyer, James" <Ja...@ingrambook.com>]

See this page for an alternate way to use DIH for Delta updates that does not generate n+1 Selects:  http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport

James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311

-----Original Message-----
From: Pravin Agrawal [mailto:Pravin_Agrawal@persistent.co.in] 
Sent: Monday, April 23, 2012 5:51 AM
To: solr-user@lucene.apache.org
Subject: Performance problem with DIH in solr 3.3

Hi All,

I am using Delta import handler(solr 3.3) to index data from my database (using 19 tables)
  Total Number of solr documents that get created from these 19 table is 444
  Total number of request send to data source during clean full import is 91083.

 My problem is that, DIH makes too many calls and puts load on my database.
  1. Can we batch these calls ?
  2. Can we use view instead? If yes can I get some examples to use view with DIH
  3. What kind of locks SOLR DIH acquire while querying DB?

Note: we are using both Full-import and delta-import handler.

Thanks in advance
Pravin Agrawal

DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.