Solr read timeout

[Tod <li...@gmail.com>]

I'm using perl to indirectly call the solr ExtractingRequestHandler to 
stream remote documents into a solr index instance.  Every 100 URL's I 
process I do a commit.  I've got about 30K documents to be indexed.  I'm 
using a stock, out of the box version of solr 1.4.1 with the necessary 
schema changes for the fields I'm indexing.

I seem to be running into performance problems about 40 documents in.  I 
start getting Failed: 500 read timeouts that last about 4 minutes each 
slowing processing down to a crawl.  I've tried a later version of tika 
(0.8) and that didn't seem to help.  I'm also not sure it's the problem.

Given I'm using a pretty much unaltered version of Solr could it be my 
problem?  I'm running everything under a typical Tomcat install on a 
Linux VM.  I understand there are performance tweaks I can make to the 
Solr config but would like to focus them first on resolving this problem 
rather than blanket tweaking the entire config.

Is there anything in particular I should look at?  Can I provide any 
more information?


Thanks - Tod