Solr becomes unresponsive while merging

[Serdar Sahin <se...@hebux.com>]

Hi,

We have been using Solr for last three years without any problem.

We have recently migrated our servers to a new datacenter, and our
servers have been upgraded as well. We also used this opportunity to
upgrade the software we use in our infrastructure. As a result of
this, Solr is upgraded to version 3.6.

However, we have been experiencing issues while merging. We have about
130.000 documents, and average page count is something about 80 pages.
We are running a delta indexing every hour, and about 20.000-30.000
documents get re-indexed. The reason is that we have about 80k unique
visitors, and we need to re-index deleted, added and viewed documents
(for view count). We are considering to create a new indexing format
to separate viewed documents, and run it daily to reduce number of
documents for the hourly operation. However it is not going to solve
the problem, it will just make things better. (We have workarounds
right now to prevent the issue, but no real solution)

We have no problems while indexing. However, when the merging starts,
we simply lose the connection to the server, and solr becomes
unresponsive for 7-12 minutes. Average load in the server also
increases.

So far;

- We tried changing compound file format
- We tried TieredMergePolicy, however we haven't changed
maxMergeAtOnce and segmentsPerTier. If they are going to help us, what
should be the ideal value?
- We switched back to LogByteSizeMergePolicy since it was the default
value in our old configuration but it didn't help either.

What do you suggest? How can we prevent the unresponsiveness during
merging. The reason I am saying merging is the problem is that we
experience the issue right after indexing, and it takes only 7-12
minutes. Could be something internal I do not know though. However,
the new server is quite powerful, way powerful than old server. So, it
was a bit of surprise since we haven't had any issues in the old
setup.

Thanks,