You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by Apache Wiki <wi...@apache.org> on 2013/09/11 10:03:42 UTC

[Solr Wiki] Update of "SolrPerformanceProblems" by TokeEskildsen

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The "SolrPerformanceProblems" page has been changed by TokeEskildsen:
https://wiki.apache.org/solr/SolrPerformanceProblems?action=diff&rev1=15&rev2=16

Comment:
Updated the cache- and the SSD-sections with more information on SSDs. Shamelessly inserted link to own blog entry on SSD vs. RAM

  
  === OS Disk Cache ===
  
- Solr relies on [[http://en.wikipedia.org/wiki/Page_cache|disk caching]] for good performance.  See [[http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html|Uwe's blog entry]] for a lot of good Lucene/Solr specific information.
+ For index updates, Solr relies on fast bulk reads and writes. For search, fast random reads are essential. One way to satisfy these requirements is to ensure that a large [[http://en.wikipedia.org/wiki/Page_cache|disk cache]] is available, another is it use Solid State Drives (see the SSD section for details). See [[http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html|Uwe's blog entry]] for a lot of good Lucene/Solr specific information.
  
  In a nutshell, you want to have enough memory available in the OS disk cache so that the important parts of your index, or ideally your entire index, will fit into the cache.  Let's say that you have a Solr index size of 8GB.  If your OS, Solr's Java heap, and all other running programs require 4GB of memory, then an ideal memory size for that server is at least 12GB.  The exact minimum requirements are highly variable and depend on things like your schema, your index contents, and your queries.
  
@@ -86, +86 @@

  
  Often SSD will be touted as a replacement for RAM used as disk cache.  This is both true and untrue.  Despite the incredible speed of SSD, RAM (the OS disk cache) is still significantly faster, and RAM still plays a big role in the performance of SSD-based systems.  You probably don't need as much RAM with SSD as you do with spinning disks, but you can't eliminate the requirement.  With spinning disks you need between 50 and 100 percent of your index size as cache.  With SSD, that might be 25 to 50 percent, less if your index is very small.
  
- One potential problem with SSD is that they require operating system TRIM support for good long-term performance.  For single disks, TRIM is usually well supported, but if you want to add any kind of hardware RAID (and most software RAID as well), TRIM support disappears.  At the time of this writing the author does not know about any mainstream RAID controllers that have operating system TRIM support, and so far has not heard about any exotic RAID controllers with the feature.  One way to make this less of a problem with Solr is to put your OS and Solr itself on a RAID of regular disks, and put your index data on a lone SSD.  If the SSD fails, your redundant server(s) will still be there to handle requests.
+ Note that SSDs are still a young technology and that the amount of independent Solr-oriented performance tests is very limited. One such test indicates that a disk cache of only 10% index size might be enough for high search performance with SSDs. See [[https://sbdevel.wordpress.com/2013/06/06/memory-is-overrated/|Memory is overrated]].
+ 
+ One potential problem with SSD is that they require operating system TRIM support for good long-term performance.  For single disks, TRIM is usually well supported, but if you want to add any kind of hardware RAID (and most software RAID as well), TRIM support disappears.   At the time of this writing, it seems that only Intel supports a solution and that is [[http://www.anandtech.com/show/6161/intel-brings-trim-to-raid0-ssd-arrays-on-7series-motherboards-we-test-it|limited to Windows 7 and RAID 0]].  One way to make this less of a problem with Solr is to put your OS and Solr itself on a RAID of regular disks, and put your index data on a lone SSD.  If the SSD fails, your redundant server(s) will still be there to handle requests.
  
  == Slow startup ==