remote multiSearching not scaling well

["Haines, Ronald C. (LNG-DAY)" <Ro...@lexisnexis.com>]

I'm hoping I'm doing something wrong, because I've been impressed with
Lucene so far.  The basic problem I'm seeing is that when I run the same
search several times against box A (with 1 RemoteSearchable), I see X
for an average search response time.  When I run the same search several
times against a different box, say box B (also with 1 RemoteSearchable),
I see Y for an average response time.

 

NOTE: the search I am using does have a date range in it and I am not
using any Filters.

 

However, when I then run the same search several times against both
boxes, I'm seeing an average response time that is much larger than the
larger or X or Y.

 

For example,

Searching against only box A, average response time = 3707 ms, 683847
docs found

Searching against only box B, average response time = 1095 ms, 93566
docs found

 

Searching against box A and B, 777413 docs found

average response time for box A = 8528 ms

average response time for box B = 2357 ms

 

As you can see the response times on each box suffers a lot when I
search them at the same time.

 

Details: I have a configuration that has a ParallelMultiSearcher at the
highest level.  In this example, it would hold 2 RemoteSearchables.
Each RemoteSearchable is running on a Wintel box and is holding a
ParallelMultiSearcher of 12 IndexSearchers.

  

So, in the above example, I am searching a total of 24 indices, 12 on
each box.

 

Do you have any suggestions as to where I can start digging ?  Thanks
for any help.

 

Ron.