You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Huy Le <hu...@springpartners.com> on 2012/02/13 17:47:24 UTC
New segment file created too often
Hi,
I am using solr 3.5. I seeing solr keeps creating new segment files (<1MB
files) so often that it triggers segment merge about every one minute. I
search the news archive, but could not find any info on this issue. I am
indexing about 10 docs of less 2KB each every second. Commit is called
after adding each document. Relevant config params are:
<mergeFactor>10</mergeFactor>
<ramBufferSizeMB>1024</ramBufferSizeMB>
<maxMergeDocs>2147483647</maxMergeDocs>
What might be triggering this frequent new segment files creation? Thanks!
Huy
--
Huy Le
Spring Partners, Inc.
http://springpadit.com
Re: New segment file created too often
Posted by Li Li <fa...@gmail.com>.
as for issue two, I have read the codes of DirectUpdateHandler2 of 2.9.1(I
think 3.x is the same)
it's commit method will call IndexWriter.close, which will flush memory
data structure to a new segment.
Logically, commit should flush. Otherwise, if you feed some data and they
are still in memory, then you commit, you expected your data are
consistent. if commit don't flush and tell you that your commit is
successful. and then the machine is power down. Then what your feed data is
lost.
so to keep commit semantically correct, it should do flush first.
As for the second question, I am now not familiar with the codes of 3.x and
4.x. maybe other guys could help
On Tue, Feb 14, 2012 at 3:24 AM, Huy Le <hu...@springpartners.com> wrote:
> Hi,
>
> This turned out to be a two-issue question. Issue one is that segment
> files get merged too frequent. Issue two is that new segment file is
> created on commit call.
>
> Looks like issue one is the behavior of tieredMergePolicy. This
> environment is newly setup, and the segment files are small. And
> mergeFactor is set at 10. Because the segment files are small, they are
> all of a similar file size, this trigger merge call by tier merge policy
> when there are 10 segments. So this is not an issue; it works as designed.
>
> Issue two, however, I don't understand why new segment is created per
> commit call? Does this have anything to do with replication setup I have
> (<str name="replicateAfter">commit</str>)?
>
> Huy
>
>
> On Mon, Feb 13, 2012 at 1:18 PM, Huy Le <hu...@springpartners.com> wrote:
>
> > I do have the indexDefault and mainIndex sections configured with the
> > desired values. As trying to get to the root cause of this issue, I have
> > the application disable commit calls, and segment files no longer that
> > created/merged so often. I then manually call commit (curl '
> > http://localhost:8983/solr/sbr01/update?commit=true'), segment files get
> > merged immediately. This suggest somethings to do with commits.
> >
> > Here is content of the config file:
> >
> >
> > <?xml version="1.0" encoding="UTF-8" ?>
> > <!--
> > Licensed to the Apache Software Foundation (ASF) under one or more
> > contributor license agreements. See the NOTICE file distributed with
> > this work for additional information regarding copyright ownership.
> > The ASF licenses this file to You under the Apache License, Version 2.0
> > (the "License"); you may not use this file except in compliance with
> > the License. You may obtain a copy of the License at
> >
> > http://www.apache.org/licenses/LICENSE-2.0
> >
> > Unless required by applicable law or agreed to in writing, software
> > distributed under the License is distributed on an "AS IS" BASIS,
> > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> > See the License for the specific language governing permissions and
> > limitations under the License.
> > -->
> >
> > <!--
> > For more details about configurations options that may appear in
> > this file, see http://wiki.apache.org/solr/SolrConfigXml.
> > -->
> > <config>
> > <!-- In all configuration below, a prefix of "solr." for class names
> > is an alias that causes solr to search appropriate packages,
> > including org.apache.solr.(search|update|request|core|analysis)
> >
> > You may also specify a fully qualified Java classname if you
> > have your own custom plugins.
> > -->
> >
> > <!-- Set this to 'false' if you want solr to continue working after
> > it has encountered an severe configuration error. In a
> > production environment, you may want solr to keep working even
> > if one handler is mis-configured.
> >
> > You may also set this to false using by setting the system
> > property:
> >
> > -Dsolr.abortOnConfigurationError=false
> > -->
> >
> >
> <abortOnConfigurationError>${solr.abortOnConfigurationError:true}</abortOnConfigurationError>
> >
> > <!-- Controls what version of Lucene various components of Solr
> > adhere to. Generally, you want to use the latest version to
> > get all bug fixes and improvements. It is highly recommended
> > that you fully re-index after changing this setting as it can
> > affect both how text is indexed and queried.
> > -->
> > <luceneMatchVersion>LUCENE_35</luceneMatchVersion>
> >
> > <!-- lib directives can be used to instruct Solr to load an Jars
> > identified and use them to resolve any "plugins" specified in
> > your solrconfig.xml or schema.xml (ie: Analyzers, Request
> > Handlers, etc...).
> >
> > All directories and paths are resolved relative to the
> > instanceDir.
> >
> > If a "./lib" directory exists in your instanceDir, all files
> > found in it are included as if you had used the following
> > syntax...
> >
> > <lib dir="./lib" />
> > -->
> >
> > <!-- A dir option by itself adds any files found in the directory to
> > the classpath, this is useful for including all jars in a
> > directory.
> > -->
> > <lib dir="../../contrib/extraction/lib" />
> > <lib dir="../../contrib/clustering/lib/" />
> > <lib dir="../../contrib/velocity/lib" />
> >
> > <!-- When a regex is specified in addition to a directory, only the
> > files in that directory which completely match the regex
> > (anchored on both ends) will be included.
> > -->
> > <lib dir="../../dist/" regex="apache-solr-cell-\d.*\.jar" />
> > <lib dir="../../dist/" regex="apache-solr-clustering-\d.*\.jar" />
> > <lib dir="../../dist/" regex="apache-solr-dataimporthandler-\d.*\.jar"
> />
> > <lib dir="../../dist/" regex="apache-solr-langid-\d.*\.jar" />
> > <lib dir="../../dist/" regex="apache-solr-velocity-\d.*\.jar" />
> >
> > <!-- If a dir option (with or without a regex) is used and nothing
> > is found that matches, it will be ignored
> > -->
> > <lib dir="../../contrib/clustering/lib/" />
> > <lib dir="/total/crap/dir/ignored" />
> >
> > <!-- an exact path can be used to specify a specific file. This
> > will cause a serious error to be logged if it can't be loaded.
> > -->
> > <!--
> > <lib path="../a-jar-that-does-not-exist.jar" />
> > -->
> >
> > <!-- Data Directory
> >
> > Used to specify an alternate directory to hold all index data
> > other than the default ./data under the Solr home. If
> > replication is in use, this should match the replication
> > configuration.
> > -->
> > <dataDir>${solr.data.dir:/u01/solr/sbr01/data}</dataDir>
> >
> >
> > <!-- The DirectoryFactory to use for indexes.
> >
> > solr.StandardDirectoryFactory, the default, is filesystem
> > based and tries to pick the best implementation for the current
> > JVM and platform. One can force a particular implementation
> > via solr.MMapDirectoryFactory, solr.NIOFSDirectoryFactory, or
> > solr.SimpleFSDirectoryFactory.
> >
> > solr.RAMDirectoryFactory is memory based, not
> > persistent, and doesn't work with replication.
> > -->
> > <directoryFactory name="DirectoryFactory"
> >
> > class="${solr.directoryFactory:solr.StandardDirectoryFactory}"/>
> >
> >
> > <!-- Index Defaults
> >
> > Values here affect all index writers and act as a default
> > unless overridden.
> >
> > WARNING: See also the <mainIndex> section below for parameters
> > that overfor Solr's main Lucene index.
> > -->
> > <indexDefaults>
> >
> > <useCompoundFile>true</useCompoundFile>
> >
> >
> > <mergeFactor>10</mergeFactor>
> > <!-- Sets the amount of RAM that may be used by Lucene indexing
> > for buffering added documents and deletions before they are
> > flushed to the Directory. -->
> > <ramBufferSizeMB>1024</ramBufferSizeMB>
> >
> > <!-- If both ramBufferSizeMB and maxBufferedDocs is set, then
> > Lucene will flush based on whichever limit is hit first.
> > -->
> > <!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
> > <!-- springpad configure -->
> > <maxMergeDocs>2147483647</maxMergeDocs>
> >
> > <!-- end springpad configure -->
> >
> >
> > <maxFieldLength>10000</maxFieldLength>
> > <writeLockTimeout>1000</writeLockTimeout>
> >
> > <!-- Expert: Merge Policy
> >
> > The Merge Policy in Lucene controls how merging is handled by
> > Lucene. The default in Solr 3.3 is TieredMergePolicy.
> >
> > The default in 2.3 was the LogByteSizeMergePolicy,
> > previous versions used LogDocMergePolicy.
> >
> > LogByteSizeMergePolicy chooses segments to merge based on
> > their size. The Lucene 2.2 default, LogDocMergePolicy chose
> > when to merge based on number of documents
> >
> > Other implementations of MergePolicy must have a no-argument
> > constructor
> > -->
> > <!--
> > <mergePolicy class="org.apache.lucene.index.TieredMergePolicy"/>
> > -->
> >
> > <!-- Expert: Merge Scheduler
> >
> > The Merge Scheduler in Lucene controls how merges are
> > performed. The ConcurrentMergeScheduler (Lucene 2.3 default)
> > can perform merges in the background using separate threads.
> > The SerialMergeScheduler (Lucene 2.2 default) does not.
> > -->
> > <!--
> > <mergeScheduler
> > class="org.apache.lucene.index.ConcurrentMergeScheduler"/>
> > -->
> >
> > <!-- LockFactory
> >
> > This option specifies which Lucene LockFactory implementation
> > to use.
> >
> > single = SingleInstanceLockFactory - suggested for a
> > read-only index or when there is no possibility of
> > another process trying to modify the index.
> > native = NativeFSLockFactory - uses OS native file locking.
> > Do not use when multiple solr webapps in the same
> > JVM are attempting to share a single index.
> > simple = SimpleFSLockFactory - uses a plain file for locking
> >
> > (For backwards compatibility with Solr 1.2, 'simple' is the
> > default if not specified.)
> >
> > More details on the nuances of each LockFactory...
> > http://wiki.apache.org/lucene-java/AvailableLockFactories
> > -->
> > <lockType>native</lockType>
> >
> > <!-- Expert: Controls how often Lucene loads terms into memory
> > Default is 128 and is likely good for most everyone.
> > -->
> >
> > <!-- <termIndexInterval>256</termIndexInterval> -->
> > </indexDefaults>
> >
> > <!-- Main Index
> >
> > Values here override the values in the <indexDefaults> section
> > for the main on disk index.
> > -->
> > <mainIndex>
> >
> > <maxMergeDocs>2147483647</maxMergeDocs>
> >
> > <useCompoundFile>true</useCompoundFile>
> > <ramBufferSizeMB>1024</ramBufferSizeMB>
> > <mergeFactor>30</mergeFactor>
> >
> > <!-- Unlock On Startup
> >
> > If true, unlock any held write or commit locks on startup.
> > This defeats the locking mechanism that allows multiple
> > processes to safely access a lucene index, and should be used
> > with care.
> >
> > This is not needed if lock type is 'none' or 'single'
> > -->
> > <unlockOnStartup>false</unlockOnStartup>
> >
> > <!-- If true, IndexReaders will be reopened (often more efficient)
> > instead of closed and then opened.
> > -->
> > <reopenReaders>true</reopenReaders>
> >
> > <!-- Commit Deletion Policy
> >
> > Custom deletion policies can specified here. The class must
> > implement org.apache.lucene.index.IndexDeletionPolicy.
> >
> >
> >
> http://lucene.apache.org/java/2_9_1/api/all/org/apache/lucene/index/IndexDeletionPolicy.html
> >
> > The standard Solr IndexDeletionPolicy implementation supports
> > deleting index commit points on number of commits, age of
> > commit point and optimized status.
> >
> > The latest commit point should always be preserved regardless
> > of the criteria.
> > -->
> > <deletionPolicy class="solr.SolrDeletionPolicy">
> > <!-- The number of commit points to be kept -->
> > <str name="maxCommitsToKeep">1</str>
> > <!-- The number of optimized commit points to be kept -->
> > <str name="maxOptimizedCommitsToKeep">0</str>
> > <!--
> > Delete all commit points once they have reached the given age.
> > Supports DateMathParser syntax e.g.
> > -->
> > <!--
> > <str name="maxCommitAge">30MINUTES</str>
> > <str name="maxCommitAge">1DAY</str>
> > -->
> > </deletionPolicy>
> >
> > <!-- Lucene Infostream
> >
> > To aid in advanced debugging, Lucene provides an "InfoStream"
> > of detailed information when indexing.
> >
> > Setting The value to true will instruct the underlying Lucene
> > IndexWriter to write its debugging info the specified file
> > -->
> > <infoStream file="INFOSTREAM.txt">false</infoStream>
> >
> > </mainIndex>
> >
> > <!-- JMX
> >
> > This example enables JMX if and only if an existing MBeanServer
> > is found, use this if you want to configure JMX through JVM
> > parameters. Remove this to disable exposing Solr configuration
> > and statistics to JMX.
> >
> > For more details see http://wiki.apache.org/solr/SolrJmx
> > -->
> > <jmx />
> > <!-- If you want to connect to a particular server, specify the
> > agentId
> > -->
> > <!-- <jmx agentId="myAgent" /> -->
> > <!-- If you want to start a new MBeanServer, specify the serviceUrl -->
> > <!-- <jmx
> > serviceUrl="service:jmx:rmi:///jndi/rmi://localhost:9999/solr"/>
> > -->
> >
> > <!-- The default high-performance update handler -->
> > <updateHandler class="solr.DirectUpdateHandler2">
> >
> > <!-- AutoCommit
> >
> > Perform a <commit/> automatically under certain conditions.
> > Instead of enabling autoCommit, consider using "commitWithin"
> > when adding documents.
> >
> > http://wiki.apache.org/solr/UpdateXmlMessages
> >
> > maxDocs - Maximum number of documents to add since the last
> > commit before automatically triggering a new commit.
> >
> > maxTime - Maximum amount of time that is allowed to pass
> > since a document was added before automaticly
> > triggering a new commit.
> > -->
> > <autoCommit>
> > <maxDocs>10000</maxDocs>
> > <maxTime>300000</maxTime>
> > </autoCommit>
> >
> > <!-- Update Related Event Listeners
> >
> > Various IndexWriter related events can trigger Listeners to
> > take actions.
> >
> > postCommit - fired after every commit or optimize command
> > postOptimize - fired after every optimize command
> > -->
> > <!-- The RunExecutableListener executes an external command from a
> > hook such as postCommit or postOptimize.
> >
> > exe - the name of the executable to run
> > dir - dir to use as the current working directory. (default=".")
> > wait - the calling thread waits until the executable returns.
> > (default="true")
> > args - the arguments to pass to the program. (default is none)
> > env - environment variables to set. (default is none)
> > -->
> > <!-- This example shows how RunExecutableListener could be used
> > with the script based replication...
> > http://wiki.apache.org/solr/CollectionDistribution
> > -->
> > <!--
> > <listener event="postCommit" class="solr.RunExecutableListener">
> > <str name="exe">solr/bin/snapshooter</str>
> > <str name="dir">.</str>
> > <bool name="wait">true</bool>
> > <arr name="args"> <str>arg1</str> <str>arg2</str> </arr>
> > <arr name="env"> <str>MYVAR=val1</str> </arr>
> > </listener>
> > -->
> > </updateHandler>
> >
> > <!-- IndexReaderFactory
> >
> > Use the following format to specify a custom IndexReaderFactory,
> > which allows for alternate IndexReader implementations.
> >
> > ** Experimental Feature **
> >
> > Please note - Using a custom IndexReaderFactory may prevent
> > certain other features from working. The API to
> > IndexReaderFactory may change without warning or may even be
> > removed from future releases if the problems cannot be
> > resolved.
> >
> >
> > ** Features that may not work with custom IndexReaderFactory **
> >
> > The ReplicationHandler assumes a disk-resident index. Using a
> > custom IndexReader implementation may cause incompatibility
> > with ReplicationHandler and may cause replication to not work
> > correctly. See SOLR-1366 for details.
> >
> > -->
> > <!--
> > <indexReaderFactory name="IndexReaderFactory" class="package.class">
> > <str name="someArg">Some Value</str>
> > </indexReaderFactory >
> > -->
> > <!-- By explicitly declaring the Factory, the termIndexDivisor can
> > be specified.
> > -->
> > <!--
> > <indexReaderFactory name="IndexReaderFactory"
> > class="solr.StandardIndexReaderFactory">
> > <int name="setTermIndexDivisor">12</int>
> > </indexReaderFactory >
> > -->
> >
> >
> > <query>
> > <!-- Max Boolean Clauses
> >
> > Maximum number of clauses in each BooleanQuery, an exception
> > is thrown if exceeded.
> >
> > ** WARNING **
> >
> > This option actually modifies a global Lucene property that
> > will affect all SolrCores. If multiple solrconfig.xml files
> > disagree on this property, the value at any given moment will
> > be based on the last SolrCore to be initialized.
> >
> > -->
> > <maxBooleanClauses>1024</maxBooleanClauses>
> >
> >
> > <!-- Solr Internal Query Caches
> >
> > There are two implementations of cache available for Solr,
> > LRUCache, based on a synchronized LinkedHashMap, and
> > FastLRUCache, based on a ConcurrentHashMap.
> >
> > FastLRUCache has faster gets and slower puts in single
> > threaded operation and thus is generally faster than LRUCache
> > when the hit ratio of the cache is high (> 75%), and may be
> > faster under other scenarios on multi-cpu systems.
> > -->
> >
> > <!-- Filter Cache
> >
> > Cache used by SolrIndexSearcher for filters (DocSets),
> > unordered sets of *all* documents that match a query. When a
> > new searcher is opened, its caches may be prepopulated or
> > "autowarmed" using data from caches in the old searcher.
> > autowarmCount is the number of items to prepopulate. For
> > LRUCache, the autowarmed items will be the most recently
> > accessed items.
> >
> > Parameters:
> > class - the SolrCache implementation LRUCache or
> > (LRUCache or FastLRUCache)
> > size - the maximum number of entries in the cache
> > initialSize - the initial capacity (number of entries) of
> > the cache. (see java.util.HashMap)
> > autowarmCount - the number of entries to prepopulate from
> > and old cache.
> > -->
> > <filterCache class="solr.FastLRUCache"
> > size="20000"
> > initialSize="1000"
> > autowarmCount="0"/>
> >
> > <!-- Query Result Cache
> >
> > Caches results of searches - ordered lists of document ids
> > (DocList) based on a query, a sort, and the range of documents
> > requested.
> > -->
> > <queryResultCache class="solr.LRUCache"
> > size="10000"
> > initialSize="1000"
> > autowarmCount="0"/>
> >
> > <!-- Document Cache
> >
> > Caches Lucene Document objects (the stored fields for each
> > document). Since Lucene internal document ids are transient,
> > this cache will not be autowarmed.
> > -->
> > <documentCache class="solr.LRUCache"
> > size="15000"
> > initialSize="1000"
> > autowarmCount="0"/>
> >
> > <!-- Field Value Cache
> >
> > Cache used to hold field values that are quickly accessible
> > by document id. The fieldValueCache is created by default
> > even if not configured here.
> > -->
> > <!--
> > <fieldValueCache class="solr.FastLRUCache"
> > size="512"
> > autowarmCount="128"
> > showItems="32" />
> > -->
> >
> > <!-- Custom Cache
> >
> > Example of a generic cache. These caches may be accessed by
> > name through SolrIndexSearcher.getCache(),cacheLookup(), and
> > cacheInsert(). The purpose is to enable easy caching of
> > user/application level data. The regenerator argument should
> > be specified as an implementation of solr.CacheRegenerator
> > if autowarming is desired.
> > -->
> > <!--
> > <cache name="myUserCache"
> > class="solr.LRUCache"
> > size="4096"
> > initialSize="1024"
> > autowarmCount="1024"
> > regenerator="com.mycompany.MyRegenerator"
> > />
> > -->
> >
> >
> > <!-- Lazy Field Loading
> >
> > If true, stored fields that are not requested will be loaded
> > lazily. This can result in a significant speed improvement
> > if the usual case is to not load all stored fields,
> > especially if the skipped fields are large compressed text
> > fields.
> > -->
> > <enableLazyFieldLoading>true</enableLazyFieldLoading>
> >
> > <!-- Use Filter For Sorted Query
> >
> > A possible optimization that attempts to use a filter to
> > satisfy a search. If the requested sort does not include
> > score, then the filterCache will be checked for a filter
> > matching the query. If found, the filter will be used as the
> > source of document ids, and then the sort will be applied to
> > that.
> >
> > For most situations, this will not be useful unless you
> > frequently get the same search repeatedly with different sort
> > options, and none of them ever use "score"
> > -->
> > <!--
> > <useFilterForSortedQuery>true</useFilterForSortedQuery>
> > -->
> >
> > <!-- Result Window Size
> >
> > An optimization for use with the queryResultCache. When a search
> > is requested, a superset of the requested number of document ids
> > are collected. For example, if a search for a particular query
> > requests matching documents 10 through 19, and queryWindowSize is
> > 50,
> > then documents 0 through 49 will be collected and cached. Any
> > further
> > requests in that range can be satisfied via the cache.
> > -->
> > <queryResultWindowSize>50</queryResultWindowSize>
> >
> > <!-- Maximum number of documents to cache for any entry in the
> > queryResultCache.
> > -->
> > <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
> >
> > <!-- Query Related Event Listeners
> >
> > Various IndexSearcher related events can trigger Listeners to
> > take actions.
> >
> > newSearcher - fired whenever a new searcher is being prepared
> > and there is a current searcher handling requests (aka
> > registered). It can be used to prime certain caches to
> > prevent long request times for certain requests.
> >
> > firstSearcher - fired whenever a new searcher is being
> > prepared but there is no current registered searcher to handle
> > requests or to gain autowarming data from.
> >
> >
> > -->
> > <!-- QuerySenderListener takes an array of NamedList and executes a
> > local query request for each NamedList in sequence.
> > -->
> > <listener event="newSearcher" class="solr.QuerySenderListener">
> > <arr name="queries">
> > <!--
> > <lst><str name="q">solr</str><str name="sort">price
> > asc</str></lst>
> > <lst><str name="q">rocks</str><str name="sort">weight
> > asc</str></lst>
> > -->
> > </arr>
> > </listener>
> > <listener event="firstSearcher" class="solr.QuerySenderListener">
> > <arr name="queries">
> > <lst>
> > <str name="q">static firstSearcher warming in
> > solrconfig.xml</str>
> > </lst>
> > </arr>
> > </listener>
> >
> > <!-- Use Cold Searcher
> >
> > If a search request comes in and there is no current
> > registered searcher, then immediately register the still
> > warming searcher and use it. If "false" then all requests
> > will block until the first searcher is done warming.
> > -->
> > <useColdSearcher>true</useColdSearcher>
> >
> > <!-- Max Warming Searchers
> >
> > Maximum number of searchers that may be warming in the
> > background concurrently. An error is returned if this limit
> > is exceeded.
> >
> > Recommend values of 1-2 for read-only slaves, higher for
> > masters w/o cache warming.
> > -->
> > <maxWarmingSearchers>100</maxWarmingSearchers>
> >
> > </query>
> >
> >
> > <!-- Request Dispatcher
> >
> > This section contains instructions for how the SolrDispatchFilter
> > should behave when processing requests for this SolrCore.
> >
> > handleSelect affects the behavior of requests such as
> /select?qt=XXX
> >
> > handleSelect="true" will cause the SolrDispatchFilter to process
> > the request and will result in consistent error handling and
> > formatting for all types of requests.
> >
> > handleSelect="false" will cause the SolrDispatchFilter to
> > ignore "/select" requests and fallback to using the legacy
> > SolrServlet and it's Solr 1.1 style error formatting
> > -->
> > <requestDispatcher handleSelect="true" >
> > <!-- Request Parsing
> >
> > These settings indicate how Solr Requests may be parsed, and
> > what restrictions may be placed on the ContentStreams from
> > those requests
> >
> > enableRemoteStreaming - enables use of the stream.file
> > and stream.url parameters for specifying remote streams.
> >
> > multipartUploadLimitInKB - specifies the max size of
> > Multipart File Uploads that Solr will allow in a Request.
> >
> > *** WARNING ***
> > The settings below authorize Solr to fetch remote files, You
> > should make sure your system has some authentication before
> > using enableRemoteStreaming="true"
> >
> > -->
> > <requestParsers enableRemoteStreaming="true"
> > multipartUploadLimitInKB="2048000" />
> >
> > <!-- HTTP Caching
> >
> > Set HTTP caching related parameters (for proxy caches and
> > clients).
> >
> > The options below instruct Solr not to output any HTTP Caching
> > related headers
> > -->
> > <httpCaching never304="true" />
> > <!-- If you include a <cacheControl> directive, it will be used to
> > generate a Cache-Control header (as well as an Expires header
> > if the value contains "max-age=")
> >
> > By default, no Cache-Control header is generated.
> >
> > You can use the <cacheControl> option even if you have set
> > never304="true"
> > -->
> > <!--
> > <httpCaching never304="true" >
> > <cacheControl>max-age=30, public</cacheControl>
> > </httpCaching>
> > -->
> > <!-- To enable Solr to respond with automatically generated HTTP
> > Caching headers, and to response to Cache Validation requests
> > correctly, set the value of never304="false"
> >
> > This will cause Solr to generate Last-Modified and ETag
> > headers based on the properties of the Index.
> >
> > The following options can also be specified to affect the
> > values of these headers...
> >
> > lastModFrom - the default value is "openTime" which means the
> > Last-Modified value (and validation against If-Modified-Since
> > requests) will all be relative to when the current Searcher
> > was opened. You can change it to lastModFrom="dirLastMod" if
> > you want the value to exactly correspond to when the physical
> > index was last modified.
> >
> > etagSeed="..." is an option you can change to force the ETag
> > header (and validation against If-None-Match requests) to be
> > different even if the index has not changed (ie: when making
> > significant changes to your config file)
> >
> > (lastModifiedFrom and etagSeed are both ignored if you use
> > the never304="true" option)
> > -->
> > <!--
> > <httpCaching lastModifiedFrom="openTime"
> > etagSeed="Solr">
> > <cacheControl>max-age=30, public</cacheControl>
> > </httpCaching>
> > -->
> > </requestDispatcher>
> >
> > <!-- Request Handlers
> >
> > http://wiki.apache.org/solr/SolrRequestHandler
> >
> > incoming queries will be dispatched to the correct handler
> > based on the path or the qt (query type) param.
> >
> > Names starting with a '/' are accessed with the a path equal to
> > the registered name. Names without a leading '/' are accessed
> > with: http://host/app/[core/]select?qt=name
> >
> > If a /select request is processed with out a qt param
> > specified, the requestHandler that declares default="true" will
> > be used.
> >
> > If a Request Handler is declared with startup="lazy", then it will
> > not be initialized until the first request that uses it.
> >
> > -->
> > <!-- SearchHandler
> >
> > http://wiki.apache.org/solr/SearchHandler
> >
> > For processing Search Queries, the primary Request Handler
> > provided with Solr is "SearchHandler" It delegates to a sequent
> > of SearchComponents (see below) and supports distributed
> > queries across multiple shards
> > -->
> > <requestHandler name="search" class="solr.SearchHandler"
> default="true">
> > <!-- default values for query parameters can be specified, these
> > will be overridden by parameters in the request
> > -->
> > <lst name="defaults">
> > <str name="echoParams">explicit</str>
> > <int name="rows">10</int>
> > </lst>
> > <!-- In addition to defaults, "appends" params can be specified
> > to identify values which should be appended to the list of
> > multi-val params from the query (or the existing "defaults").
> > -->
> > <!-- In this example, the param "fq=instock:true" would be appended
> to
> > any query time fq params the user may specify, as a mechanism
> for
> > partitioning the index, independent of any user selected
> filtering
> > that may also be desired (perhaps as a result of faceted
> > searching).
> >
> > NOTE: there is *absolutely* nothing a client can do to prevent
> > these
> > "appends" values from being used, so don't use this mechanism
> > unless you are sure you always want it.
> > -->
> > <!--
> > <lst name="appends">
> > <str name="fq">inStock:true</str>
> > </lst>
> > -->
> > <!-- "invariants" are a way of letting the Solr maintainer lock down
> > the options available to Solr clients. Any params values
> > specified here are used regardless of what values may be
> specified
> > in either the query, the "defaults", or the "appends" params.
> >
> > In this example, the facet.field and facet.query params would
> > be fixed, limiting the facets clients can use. Faceting is
> > not turned on by default - but if the client does specify
> > facet=true in the request, these are the only facets they
> > will be able to see counts for; regardless of what other
> > facet.field or facet.query params they may specify.
> >
> > NOTE: there is *absolutely* nothing a client can do to prevent
> > these
> > "invariants" values from being used, so don't use this mechanism
> > unless you are sure you always want it.
> > -->
> > <!--
> > <lst name="invariants">
> > <str name="facet.field">cat</str>
> > <str name="facet.field">manu_exact</str>
> > <str name="facet.query">price:[* TO 500]</str>
> > <str name="facet.query">price:[500 TO *]</str>
> > </lst>
> > -->
> > <!-- If the default list of SearchComponents is not desired, that
> > list can either be overridden completely, or components can be
> > prepended or appended to the default list. (see below)
> > -->
> > <!--
> > <arr name="components">
> > <str>nameOfCustomComponent1</str>
> > <str>nameOfCustomComponent2</str>
> > </arr>
> > -->
> > </requestHandler>
> >
> > <!-- A Robust Example
> >
> > This example SearchHandler declaration shows off usage of the
> > SearchHandler with many defaults declared
> >
> > Note that multiple instances of the same Request Handler
> > (SearchHandler) can be registered multiple times with different
> > names (and different init parameters)
> > -->
> > <requestHandler name="/browse" class="solr.SearchHandler">
> > <lst name="defaults">
> > <str name="echoParams">explicit</str>
> >
> > <!-- VelocityResponseWriter settings -->
> > <!---
> > <str name="wt">velocity</str>
> > -->
> >
> > <str name="v.template">browse</str>
> > <str name="v.layout">layout</str>
> > <str name="title">Solritas</str>
> >
> > <str name="defType">edismax</str>
> > <str name="q.alt">*:*</str>
> > <str name="rows">10</str>
> > <str name="fl">*,score</str>
> > <str name="mlt.qf">
> > text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
> > </str>
> > <str name="mlt.fl">text,features,name,sku,id,manu,cat</str>
> > <int name="mlt.count">3</int>
> >
> > <str name="qf">
> > text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
> > </str>
> >
> > <str name="facet">on</str>
> > <str name="facet.field">cat</str>
> > <str name="facet.field">manu_exact</str>
> > <str name="facet.query">ipod</str>
> > <str name="facet.query">GB</str>
> > <str name="facet.mincount">1</str>
> > <str name="facet.pivot">cat,inStock</str>
> > <str name="facet.range.other">after</str>
> > <str name="facet.range">price</str>
> > <int name="f.price.facet.range.start">0</int>
> > <int name="f.price.facet.range.end">600</int>
> > <int name="f.price.facet.range.gap">50</int>
> > <str name="facet.range">popularity</str>
> > <int name="f.popularity.facet.range.start">0</int>
> > <int name="f.popularity.facet.range.end">10</int>
> > <int name="f.popularity.facet.range.gap">3</int>
> > <str name="facet.range">manufacturedate_dt</str>
> > <str
> > name="f.manufacturedate_dt.facet.range.start">NOW/YEAR-10YEARS</str>
> > <str name="f.manufacturedate_dt.facet.range.end">NOW</str>
> > <str name="f.manufacturedate_dt.facet.range.gap">+1YEAR</str>
> > <str name="f.manufacturedate_dt.facet.range.other">before</str>
> > <str name="f.manufacturedate_dt.facet.range.other">after</str>
> >
> >
> > <!-- Highlighting defaults -->
> > <str name="hl">on</str>
> > <str name="hl.fl">text features name</str>
> > <str name="f.name.hl.fragsize">0</str>
> > <str name="f.name.hl.alternateField">name</str>
> > </lst>
> > <arr name="last-components">
> > <str>spellcheck</str>
> > </arr>
> > <!--
> > <str name="url-scheme">httpx</str>
> > -->
> > </requestHandler>
> >
> > <!-- XML Update Request Handler.
> >
> > http://wiki.apache.org/solr/UpdateXmlMessages
> >
> > The canonical Request Handler for Modifying the Index through
> > commands specified using XML.
> >
> > Note: Since solr1.1 requestHandlers requires a valid content
> > type header if posted in the body. For example, curl now
> > requires: -H 'Content-type:text/xml; charset=utf-8'
> > -->
> > <requestHandler name="/update"
> > class="solr.XmlUpdateRequestHandler">
> > <!-- See below for information on defining
> > updateRequestProcessorChains that can be used by name
> > on each Update Request
> > -->
> > <!--
> > <lst name="defaults">
> > <str name="update.chain">dedupe</str>
> > </lst>
> > -->
> > </requestHandler>
> > <!-- Binary Update Request Handler
> > http://wiki.apache.org/solr/javabin
> > -->
> > <requestHandler name="/update/javabin"
> > class="solr.BinaryUpdateRequestHandler" />
> >
> > <!-- CSV Update Request Handler
> > http://wiki.apache.org/solr/UpdateCSV
> > -->
> > <requestHandler name="/update/csv"
> > class="solr.CSVRequestHandler"
> > startup="lazy" />
> >
> > <!-- JSON Update Request Handler
> > http://wiki.apache.org/solr/UpdateJSON
> > -->
> > <requestHandler name="/update/json"
> > class="solr.JsonUpdateRequestHandler"
> > startup="lazy" />
> >
> > <!-- Solr Cell Update Request Handler
> >
> > http://wiki.apache.org/solr/ExtractingRequestHandler
> >
> > -->
> > <requestHandler name="/update/extract"
> > startup="lazy"
> > class="solr.extraction.ExtractingRequestHandler" >
> > <lst name="defaults">
> > <!-- All the main content goes into "text"... if you need to return
> > the extracted text or do highlighting, use a stored field. -->
> > <str name="fmap.content">text</str>
> > <str name="lowernames">true</str>
> > <str name="uprefix">ignored_</str>
> >
> > <!-- capture link hrefs but ignore div attributes -->
> > <str name="captureAttr">true</str>
> > <str name="fmap.a">links</str>
> > <str name="fmap.div">ignored_</str>
> > </lst>
> > </requestHandler>
> >
> > <!-- XSLT Update Request Handler
> > Transforms incoming XML with stylesheet identified by tr=
> > -->
> > <requestHandler name="/update/xslt"
> > startup="lazy"
> > class="solr.XsltUpdateRequestHandler"/>
> >
> > <!-- Field Analysis Request Handler
> >
> > RequestHandler that provides much the same functionality as
> > analysis.jsp. Provides the ability to specify multiple field
> > types and field names in the same request and outputs
> > index-time and query-time analysis for each of them.
> >
> > Request parameters are:
> > analysis.fieldname - field name whose analyzers are to be used
> >
> > analysis.fieldtype - field type whose analyzers are to be used
> > analysis.fieldvalue - text for index-time analysis
> > q (or analysis.q) - text for query time analysis
> > analysis.showmatch (true|false) - When set to true and when
> > query analysis is performed, the produced tokens of the
> > field value analysis will be marked as "matched" for every
> > token that is produces by the query analysis
> > -->
> > <requestHandler name="/analysis/field"
> > startup="lazy"
> > class="solr.FieldAnalysisRequestHandler" />
> >
> >
> > <!-- Document Analysis Handler
> >
> > http://wiki.apache.org/solr/AnalysisRequestHandler
> >
> > An analysis handler that provides a breakdown of the analysis
> > process of provided docuemnts. This handler expects a (single)
> > content stream with the following format:
> >
> > <docs>
> > <doc>
> > <field name="id">1</field>
> > <field name="name">The Name</field>
> > <field name="text">The Text Value</field>
> > </doc>
> > <doc>...</doc>
> > <doc>...</doc>
> > ...
> > </docs>
> >
> > Note: Each document must contain a field which serves as the
> > unique key. This key is used in the returned response to associate
> > an analysis breakdown to the analyzed document.
> >
> > Like the FieldAnalysisRequestHandler, this handler also supports
> > query analysis by sending either an "analysis.query" or "q"
> > request parameter that holds the query text to be analyzed. It
> > also supports the "analysis.showmatch" parameter which when set to
> > true, all field tokens that match the query tokens will be marked
> > as a "match".
> > -->
> > <requestHandler name="/analysis/document"
> > class="solr.DocumentAnalysisRequestHandler"
> > startup="lazy" />
> >
> > <!-- Admin Handlers
> >
> > Admin Handlers - This will register all the standard admin
> > RequestHandlers.
> > -->
> > <requestHandler name="/admin/"
> > class="solr.admin.AdminHandlers" />
> > <!-- This single handler is equivalent to the following... -->
> > <!--
> > <requestHandler name="/admin/luke"
> > class="solr.admin.LukeRequestHandler" />
> > <requestHandler name="/admin/system"
> > class="solr.admin.SystemInfoHandler" />
> > <requestHandler name="/admin/plugins"
> > class="solr.admin.PluginInfoHandler" />
> > <requestHandler name="/admin/threads"
> > class="solr.admin.ThreadDumpHandler" />
> > <requestHandler name="/admin/properties"
> > class="solr.admin.PropertiesRequestHandler" />
> > <requestHandler name="/admin/file"
> > class="solr.admin.ShowFileRequestHandler" >
> > -->
> > <!-- If you wish to hide files under ${solr.home}/conf, explicitly
> > register the ShowFileRequestHandler using:
> > -->
> > <!--
> > <requestHandler name="/admin/file"
> > class="solr.admin.ShowFileRequestHandler" >
> > <lst name="invariants">
> > <str name="hidden">synonyms.txt</str>
> > <str name="hidden">anotherfile.txt</str>
> > </lst>
> > </requestHandler>
> > -->
> >
> > <!-- ping/healthcheck -->
> > <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
> > <lst name="invariants">
> > <str name="qt">search</str>
> > <str name="q">solrpingquery</str>
> > </lst>
> > <lst name="defaults">
> > <str name="echoParams">all</str>
> > </lst>
> > </requestHandler>
> >
> > <!-- Echo the request contents back to the client -->
> > <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
> > <lst name="defaults">
> > <str name="echoParams">explicit</str>
> > <str name="echoHandler">true</str>
> > </lst>
> > </requestHandler>
> >
> > <!-- Solr Replication
> >
> > The SolrReplicationHandler supports replicating indexes from a
> > "master" used for indexing and "salves" used for queries.
> >
> > http://wiki.apache.org/solr/SolrReplication
> >
> > In the example below, remove the <lst name="master"> section if
> > this is just a slave and remove the <lst name="slave"> section
> > if this is just a master.
> > -->
> > <!--
> > <requestHandler name="/replication" class="solr.ReplicationHandler"
> >
> > <lst name="master">
> > <str name="replicateAfter">commit</str>
> > <str name="replicateAfter">startup</str>
> > <str name="confFiles">schema.xml,stopwords.txt</str>
> > </lst>
> > <lst name="slave">
> > <str name="masterUrl">http://localhost:8983/solr/replication
> > </str>
> > <str name="pollInterval">00:00:60</str>
> > </lst>
> > </requestHandler>
> > -->
> >
> > <!-- Search Components
> >
> > Search components are registered to SolrCore and used by
> > instances of SearchHandler (which can access them by name)
> >
> > By default, the following components are available:
> >
> > <searchComponent name="query" class="solr.QueryComponent" />
> > <searchComponent name="facet" class="solr.FacetComponent" />
> > <searchComponent name="mlt"
> > class="solr.MoreLikeThisComponent" />
> > <searchComponent name="highlight" class="solr.HighlightComponent"
> />
> > <searchComponent name="stats" class="solr.StatsComponent" />
> > <searchComponent name="debug" class="solr.DebugComponent" />
> >
> > Default configuration in a requestHandler would look like:
> >
> > <arr name="components">
> > <str>query</str>
> > <str>facet</str>
> > <str>mlt</str>
> > <str>highlight</str>
> > <str>stats</str>
> > <str>debug</str>
> > </arr>
> >
> > If you register a searchComponent to one of the standard names,
> > that will be used instead of the default.
> >
> > To insert components before or after the 'standard' components,
> use:
> >
> > <arr name="first-components">
> > <str>myFirstComponentName</str>
> > </arr>
> >
> > <arr name="last-components">
> > <str>myLastComponentName</str>
> > </arr>
> >
> > NOTE: The component registered with the name "debug" will
> > always be executed after the "last-components"
> >
> > -->
> >
> > <!-- Spell Check
> >
> > The spell check component can return a list of alternative
> spelling
> > suggestions.
> >
> > http://wiki.apache.org/solr/SpellCheckComponent
> > -->
> > <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
> >
> > <str name="queryAnalyzerFieldType">textSpell</str>
> >
> > <!-- Multiple "Spell Checkers" can be declared and used by this
> > component
> > -->
> >
> > <!-- a spellchecker built from a field of the main index, and
> > written to disk
> > -->
> > <lst name="spellchecker">
> > <str name="name">default</str>
> > <str name="field">name</str>
> > <str name="spellcheckIndexDir">spellchecker</str>
> > <!-- uncomment this to require terms to occur in 1% of the
> documents
> > in order to be included in the dictionary
> > <float name="thresholdTokenFrequency">.01</float>
> > -->
> > </lst>
> >
> > <!-- a spellchecker that uses a different distance measure -->
> > <!--
> > <lst name="spellchecker">
> > <str name="name">jarowinkler</str>
> > <str name="field">spell</str>
> > <str name="distanceMeasure">
> > org.apache.lucene.search.spell.JaroWinklerDistance
> > </str>
> > <str name="spellcheckIndexDir">spellcheckerJaro</str>
> > </lst>
> > -->
> >
> > <!-- a spellchecker that use an alternate comparator
> >
> > comparatorClass be one of:
> > 1. score (default)
> > 2. freq (Frequency first, then score)
> > 3. A fully qualified class name
> > -->
> > <!--
> > <lst name="spellchecker">
> > <str name="name">freq</str>
> > <str name="field">lowerfilt</str>
> > <str name="spellcheckIndexDir">spellcheckerFreq</str>
> > <str name="comparatorClass">freq</str>
> > <str name="buildOnCommit">true</str>
> > -->
> >
> > <!-- A spellchecker that reads the list of words from a file -->
> > <!--
> > <lst name="spellchecker">
> > <str name="classname">solr.FileBasedSpellChecker</str>
> > <str name="name">file</str>
> > <str name="sourceLocation">spellings.txt</str>
> > <str name="characterEncoding">UTF-8</str>
> > <str name="spellcheckIndexDir">spellcheckerFile</str>
> > </lst>
> > -->
> > </searchComponent>
> >
> > <!-- A request handler for demonstrating the spellcheck component.
> >
> > NOTE: This is purely as an example. The whole purpose of the
> > SpellCheckComponent is to hook it into the request handler that
> > handles your normal user queries so that a separate request is
> > not needed to get suggestions.
> >
> > IN OTHER WORDS, THERE IS REALLY GOOD CHANCE THE SETUP BELOW IS
> > NOT WHAT YOU WANT FOR YOUR PRODUCTION SYSTEM!
> >
> > See http://wiki.apache.org/solr/SpellCheckComponent for details
> > on the request parameters.
> > -->
> > <requestHandler name="/spell" class="solr.SearchHandler"
> startup="lazy">
> > <lst name="defaults">
> > <str name="spellcheck.onlyMorePopular">false</str>
> > <str name="spellcheck.extendedResults">false</str>
> > <str name="spellcheck.count">1</str>
> > </lst>
> > <arr name="last-components">
> > <str>spellcheck</str>
> > </arr>
> > </requestHandler>
> >
> > <!-- Term Vector Component
> >
> > http://wiki.apache.org/solr/TermVectorComponent
> > -->
> > <searchComponent name="tvComponent" class="solr.TermVectorComponent"/>
> >
> > <!-- A request handler for demonstrating the term vector component
> >
> > This is purely as an example.
> >
> > In reality you will likely want to add the component to your
> > already specified request handlers.
> > -->
> > <requestHandler name="tvrh" class="solr.SearchHandler" startup="lazy">
> > <lst name="defaults">
> > <bool name="tv">true</bool>
> > </lst>
> > <arr name="last-components">
> > <str>tvComponent</str>
> > </arr>
> > </requestHandler>
> >
> > <!-- Clustering Component
> >
> > http://wiki.apache.org/solr/ClusteringComponent
> >
> > This relies on third party jars which are notincluded in the
> > release. To use this component (and the "/clustering" handler)
> > Those jars will need to be downloaded, and you'll need to set
> > the solr.cluster.enabled system property when running solr...
> >
> > java -Dsolr.clustering.enabled=true -jar start.jar
> > -->
> > <searchComponent name="clustering"
> > enable="${solr.clustering.enabled:false}"
> > class="solr.clustering.ClusteringComponent" >
> > <!-- Declare an engine -->
> > <lst name="engine">
> > <!-- The name, only one can be named "default" -->
> > <str name="name">default</str>
> >
> > <!-- Class name of Carrot2 clustering algorithm.
> >
> > Currently available algorithms are:
> >
> > * org.carrot2.clustering.lingo.LingoClusteringAlgorithm
> > * org.carrot2.clustering.stc.STCClusteringAlgorithm
> > *
> > org.carrot2.clustering.kmeans.BisectingKMeansClusteringAlgorithm
> >
> > See http://project.carrot2.org/algorithms.html for the
> > algorithm's characteristics.
> > -->
> > <str
> >
> name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm</str>
> >
> > <!-- Overriding values for Carrot2 default algorithm attributes.
> >
> > For a description of all available attributes, see:
> > http://download.carrot2.org/stable/manual/#chapter.components
> .
> > Use attribute key as name attribute of str elements
> > below. These can be further overridden for individual
> > requests by specifying attribute key as request parameter
> > name and attribute value as parameter value.
> > -->
> > <str
> name="LingoClusteringAlgorithm.desiredClusterCountBase">20</str>
> >
> > <!-- Location of Carrot2 lexical resources.
> >
> > A directory from which to load Carrot2-specific stop words
> > and stop labels. Absolute or relative to Solr config
> directory.
> > If a specific resource (e.g. stopwords.en) is present in the
> > specified dir, it will completely override the corresponding
> > default one that ships with Carrot2.
> >
> > For an overview of Carrot2 lexical resources, see:
> >
> > http://download.carrot2.org/head/manual/#chapter.lexical-resources
> > -->
> > <str name="carrot.lexicalResourcesDir">clustering/carrot2</str>
> >
> > <!-- The language to assume for the documents.
> >
> > For a list of allowed values, see:
> >
> >
> http://download.carrot2.org/stable/manual/#section.attribute.lingo.MultilingualClustering.defaultLanguage
> > -->
> > <str name="MultilingualClustering.defaultLanguage">ENGLISH</str>
> > </lst>
> > <lst name="engine">
> > <str name="name">stc</str>
> > <str
> >
> name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm</str>
> > </lst>
> > </searchComponent>
> >
> > <!-- A request handler for demonstrating the clustering component
> >
> > This is purely as an example.
> >
> > In reality you will likely want to add the component to your
> > already specified request handlers.
> > -->
> > <requestHandler name="/clustering"
> > startup="lazy"
> > enable="${solr.clustering.enabled:false}"
> > class="solr.SearchHandler">
> > <lst name="defaults">
> > <bool name="clustering">true</bool>
> > <str name="clustering.engine">default</str>
> > <bool name="clustering.results">true</bool>
> > <!-- The title field -->
> > <str name="carrot.title">name</str>
> > <str name="carrot.url">id</str>
> > <!-- The field to cluster on -->
> > <str name="carrot.snippet">features</str>
> > <!-- produce summaries -->
> > <bool name="carrot.produceSummary">true</bool>
> > <!-- the maximum number of labels per cluster -->
> > <!--<int name="carrot.numDescriptions">5</int>-->
> > <!-- produce sub clusters -->
> > <bool name="carrot.outputSubClusters">false</bool>
> >
> > <str name="defType">edismax</str>
> > <str name="qf">
> > text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
> > </str>
> > <str name="q.alt">*:*</str>
> > <str name="rows">10</str>
> > <str name="fl">*,score</str>
> > </lst>
> > <arr name="last-components">
> > <str>clustering</str>
> > </arr>
> > </requestHandler>
> >
> > <!-- Terms Component
> >
> > http://wiki.apache.org/solr/TermsComponent
> >
> > A component to return terms and document frequency of those
> > terms
> > -->
> > <searchComponent name="terms" class="solr.TermsComponent"/>
> >
> > <!-- A request handler for demonstrating the terms component -->
> > <requestHandler name="/terms" class="solr.SearchHandler"
> startup="lazy">
> > <lst name="defaults">
> > <bool name="terms">true</bool>
> > </lst>
> > <arr name="components">
> > <str>terms</str>
> > </arr>
> > </requestHandler>
> >
> >
> > <!-- Query Elevation Component
> >
> > http://wiki.apache.org/solr/QueryElevationComponent
> >
> > a search component that enables you to configure the top
> > results for a given query regardless of the normal lucene
> > scoring.
> > -->
> > <!--
> > <searchComponent name="elevator" class="solr.QueryElevationComponent" >
> > <str name="queryFieldType">string</str>
> > <str name="config-file">elevate.xml</str>
> > </searchComponent>
> > -->
> >
> > <!-- A request handler for demonstrating the elevator component -->
> > <requestHandler name="/elevate" class="solr.SearchHandler"
> > startup="lazy">
> > <lst name="defaults">
> > <str name="echoParams">explicit</str>
> > </lst>
> > <arr name="last-components">
> > <str>elevator</str>
> > </arr>
> > </requestHandler>
> >
> > <!-- Highlighting Component
> >
> > http://wiki.apache.org/solr/HighlightingParameters
> > -->
> > <searchComponent class="solr.HighlightComponent" name="highlight">
> > <highlighting>
> > <!-- Configure the standard fragmenter -->
> > <!-- This could most likely be commented out in the "default" case
> > -->
> > <fragmenter name="gap"
> > default="true"
> > class="solr.highlight.GapFragmenter">
> > <lst name="defaults">
> > <int name="hl.fragsize">100</int>
> > </lst>
> > </fragmenter>
> >
> > <!-- A regular-expression-based fragmenter
> > (for sentence extraction)
> > -->
> > <fragmenter name="regex"
> > class="solr.highlight.RegexFragmenter">
> > <lst name="defaults">
> > <!-- slightly smaller fragsizes work better because of slop -->
> > <int name="hl.fragsize">70</int>
> > <!-- allow 50% slop on fragment sizes -->
> > <float name="hl.regex.slop">0.5</float>
> > <!-- a basic sentence pattern -->
> > <str name="hl.regex.pattern">[-\w
> > ,/\n\"']{20,200}</str>
> > </lst>
> > </fragmenter>
> >
> > <!-- Configure the standard formatter -->
> > <formatter name="html"
> > default="true"
> > class="solr.highlight.HtmlFormatter">
> > <lst name="defaults">
> > <str name="hl.simple.pre"><![CDATA[<em>]]></str>
> > <str name="hl.simple.post"><![CDATA[</em>]]></str>
> > </lst>
> > </formatter>
> >
> > <!-- Configure the standard encoder -->
> > <encoder name="html"
> > class="solr.highlight.HtmlEncoder" />
> >
> > <!-- Configure the standard fragListBuilder -->
> > <fragListBuilder name="simple"
> > default="true"
> > class="solr.highlight.SimpleFragListBuilder"/>
> >
> > <!-- Configure the single fragListBuilder -->
> > <fragListBuilder name="single"
> > class="solr.highlight.SingleFragListBuilder"/>
> >
> > <!-- default tag FragmentsBuilder -->
> > <fragmentsBuilder name="default"
> > default="true"
> >
> class="solr.highlight.ScoreOrderFragmentsBuilder">
> > <!--
> > <lst name="defaults">
> > <str name="hl.multiValuedSeparatorChar">/</str>
> > </lst>
> > -->
> > </fragmentsBuilder>
> >
> > <!-- multi-colored tag FragmentsBuilder -->
> > <fragmentsBuilder name="colored"
> >
> class="solr.highlight.ScoreOrderFragmentsBuilder">
> > <lst name="defaults">
> > <str name="hl.tag.pre"><![CDATA[
> > <b style="background:yellow">,<b
> > style="background:lawgreen">,
> > <b style="background:aquamarine">,<b
> > style="background:magenta">,
> > <b style="background:palegreen">,<b
> > style="background:coral">,
> > <b style="background:wheat">,<b style="background:khaki">,
> > <b style="background:lime">,<b
> > style="background:deepskyblue">]]></str>
> > <str name="hl.tag.post"><![CDATA[</b>]]></str>
> > </lst>
> > </fragmentsBuilder>
> >
> > <boundaryScanner name="default"
> > default="true"
> > class="solr.highlight.SimpleBoundaryScanner">
> > <lst name="defaults">
> > <str name="hl.bs.maxScan">10</str>
> > <str name="hl.bs.chars">.,!? 	 </str>
> > </lst>
> > </boundaryScanner>
> >
> > <boundaryScanner name="breakIterator"
> >
> class="solr.highlight.BreakIteratorBoundaryScanner">
> > <lst name="defaults">
> > <!-- type should be one of CHARACTER, WORD(default), LINE and
> > SENTENCE -->
> > <str name="hl.bs.type">WORD</str>
> > <!-- language and country are used when constructing Locale
> > object. -->
> > <!-- And the Locale object will be used when getting instance
> of
> > BreakIterator -->
> > <str name="hl.bs.language">en</str>
> > <str name="hl.bs.country">US</str>
> > </lst>
> > </boundaryScanner>
> > </highlighting>
> > </searchComponent>
> >
> > <!-- Update Processors
> >
> > Chains of Update Processor Factories for dealing with Update
> > Requests can be declared, and then used by name in Update
> > Request Processors
> >
> > http://wiki.apache.org/solr/UpdateRequestProcessor
> >
> > -->
> > <!-- Deduplication
> >
> > An example dedup update processor that creates the "id" field
> > on the fly based on the hash code of some other fields. This
> > example has overwriteDupes set to false since we are using the
> > id field as the signatureField and Solr will maintain
> > uniqueness based on that anyway.
> >
> > -->
> > <!--
> > <updateRequestProcessorChain name="dedupe">
> > <processor class="solr.processor.SignatureUpdateProcessorFactory">
> > <bool name="enabled">true</bool>
> > <str name="signatureField">id</str>
> > <bool name="overwriteDupes">false</bool>
> > <str name="fields">name,features,cat</str>
> > <str name="signatureClass">solr.processor.Lookup3Signature</str>
> > </processor>
> > <processor class="solr.LogUpdateProcessorFactory" />
> > <processor class="solr.RunUpdateProcessorFactory" />
> > </updateRequestProcessorChain>
> > -->
> >
> > <!--
> > This example update chain identifies the language of the incoming
> > documents using the langid contrib. The detected language is
> > written to field language_s. No field name mapping is done.
> > The fields used for detection are text, title, subject and
> > description,
> > making this example suitable for detecting languages form
> full-text
> > rich documents injected via ExtractingRequestHandler.
> > See more about langId at
> > http://wiki.apache.org/solr/LanguageDetection
> > -->
> > <!--
> > <updateRequestProcessorChain name="langid">
> > <processor
> >
> class="org.apache.solr.update.processor.TikaLanguageIdentifierUpdateProcessorFactory">
> > <str name="langid.fl">text,title,subject,description</str>
> > <str name="langid.langField">language_s</str>
> > <str name="langid.fallback">en</str>
> > </processor>
> > <processor class="solr.LogUpdateProcessorFactory" />
> > <processor class="solr.RunUpdateProcessorFactory" />
> > </updateRequestProcessorChain>
> > -->
> >
> > <!-- Response Writers
> >
> > http://wiki.apache.org/solr/QueryResponseWriter
> >
> > Request responses will be written using the writer specified by
> > the 'wt' request parameter matching the name of a registered
> > writer.
> >
> > The "default" writer is the default and will be used if 'wt' is
> > not specified in the request.
> > -->
> > <!-- The following response writers are implicitly configured unless
> > overridden...
> > -->
> > <!--
> > <queryResponseWriter name="xml"
> > default="true"
> > class="solr.XMLResponseWriter" />
> > <queryResponseWriter name="json" class="solr.JSONResponseWriter"/>
> > <queryResponseWriter name="python"
> class="solr.PythonResponseWriter"/>
> > <queryResponseWriter name="ruby" class="solr.RubyResponseWriter"/>
> > <queryResponseWriter name="php" class="solr.PHPResponseWriter"/>
> > <queryResponseWriter name="phps"
> > class="solr.PHPSerializedResponseWriter"/>
> > <queryResponseWriter name="csv" class="solr.CSVResponseWriter"/>
> > -->
> >
> > <queryResponseWriter name="json" class="solr.JSONResponseWriter">
> > <!-- For the purposes of the tutorial, JSON responses are written as
> > plain text so that they are easy to read in *any* browser.
> > If you expect a MIME type of "application/json" just remove this
> > override.
> > -->
> > <str name="content-type">text/plain; charset=UTF-8</str>
> > </queryResponseWriter>
> >
> > <!--
> > Custom response writers can be declared as needed...
> > -->
> > <!-- The solr.velocity.enabled flag is used by Solr's test cases so
> > that this response writer is not
> > loaded (causing an error if contrib/velocity has not been built
> > fully) -->
> > <!--
> > <queryResponseWriter name="velocity"
> > class="solr.VelocityResponseWriter"
> enable="${solr.velocity.enabled:true}"/>
> > -->
> >
> >
> > <!-- XSLT response writer transforms the XML output by any xslt file
> > found
> > in Solr's conf/xslt directory. Changes to xslt files are checked
> > for
> > every xsltCacheLifetimeSeconds.
> > -->
> > <queryResponseWriter name="xslt" class="solr.XSLTResponseWriter">
> > <int name="xsltCacheLifetimeSeconds">5</int>
> > </queryResponseWriter>
> >
> > <!-- Query Parsers
> >
> > http://wiki.apache.org/solr/SolrQuerySyntax
> >
> > Multiple QParserPlugins can be registered by name, and then
> > used in either the "defType" param for the QueryComponent (used
> > by SearchHandler) or in LocalParams
> > -->
> > <!-- example of registering a query parser -->
> > <!--
> > <queryParser name="myparser" class="com.mycompany.MyQParserPlugin"/>
> > -->
> >
> > <!-- Function Parsers
> >
> > http://wiki.apache.org/solr/FunctionQuery
> >
> > Multiple ValueSourceParsers can be registered by name, and then
> > used as function names when using the "func" QParser.
> > -->
> > <!-- example of registering a custom function parser -->
> > <!--
> > <valueSourceParser name="myfunc"
> > class="com.mycompany.MyValueSourceParser" />
> > -->
> >
> > <!-- Legacy config for the admin interface -->
> > <admin>
> > <defaultQuery>*:*</defaultQuery>
> >
> > <!-- configure a healthcheck file for servers behind a
> > loadbalancer
> > -->
> > <!--
> > <healthcheck type="file">server-enabled</healthcheck>
> > -->
> > </admin>
> >
> > <requestHandler name="/replication" class="solr.ReplicationHandler" >
> > <lst name="master">
> > <!--Replicate on 'startup' and 'commit'. 'optimize' is also a
> > valid value for replicateAfter. -->
> > <str name="replicateAfter">startup</str>
> > <str name="replicateAfter">commit</str>
> >
> > <!--Create a backup after 'optimize'. Other values can be
> > 'commit', 'startup'. It is possible to have multiple entries of this
> config
> > string. Note that this is just for backup, replication does not require
> > this. -->
> > <!-- <str name="backupAfter">optimize</str> -->
> >
> > <!--If configuration files need to be replicated give the names
> > here, separated by comma -->
> > <str name="confFiles">schema.xml,stopwords.txt,elevate.xml</str>
> > <!--The default value of reservation is 10 secs.See the
> > documentation below . Normally , you should not need to specify this -->
> > <str name="commitReserveDuration">00:01:00</str>
> > </lst>
> > </requestHandler>
> >
> > </config>
> >
> >
> >
> >
> >
> > On Mon, Feb 13, 2012 at 12:36 PM, Li Li <fa...@gmail.com> wrote:
> >
> >> can you post your config file?
> >> I found there are 2 places to config ramBufferSizeMB in latest svn of
> >> 3.6's
> >> example solrconfig.xml. trying to modify them both?
> >>
> >> <indexDefaults>
> >>
> >> <useCompoundFile>false</useCompoundFile>
> >>
> >> <mergeFactor>10</mergeFactor>
> >> <!-- Sets the amount of RAM that may be used by Lucene indexing
> >> for buffering added documents and deletions before they are
> >> flushed to the Directory. -->
> >> <ramBufferSizeMB>32</ramBufferSizeMB>
> >> <!-- If both ramBufferSizeMB and maxBufferedDocs is set, then
> >> Lucene will flush based on whichever limit is hit first.
> >> -->
> >> <!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
> >>
> >> <maxFieldLength>10000</maxFieldLength>
> >> <writeLockTimeout>1000</writeLockTimeout>
> >>
> >> .............
> >> <!-- <termIndexInterval>256</termIndexInterval> -->
> >> </indexDefaults>
> >>
> >> <!-- Main Index
> >>
> >> Values here override the values in the <indexDefaults> section
> >> for the main on disk index.
> >> -->
> >> <mainIndex>
> >>
> >> <useCompoundFile>false</useCompoundFile>
> >> <ramBufferSizeMB>32</ramBufferSizeMB>
> >> <mergeFactor>10</mergeFactor>
> >> ........
> >> </mainIndex>
> >>
> >> On Tue, Feb 14, 2012 at 1:10 AM, Huy Le <hu...@springpartners.com>
> wrote:
> >>
> >> > Hi,
> >> >
> >> > I am using solr 3.5. As I understood it, NRT is a solr 4 feature, but
> >> solr
> >> > 4 is not released yet.
> >> >
> >> > I understand commit after adding each document is expensive, but the
> >> > application requires that documents be available after adding to the
> >> index.
> >> >
> >> > What I don't understand is why new segment files are created so often.
> >> > Are the commit calls triggering new segment files being created? I
> >> don't
> >> > see this behavior in another environment of the same version of solr.
> >> >
> >> > Huy
> >> >
> >> > On Mon, Feb 13, 2012 at 11:55 AM, Li Li <fa...@gmail.com> wrote:
> >> >
> >> > > Commit is called
> >> > > after adding each document
> >> > >
> >> > >
> >> > > you should add enough documents and then calling a commit. commit
> is
> >> a
> >> > > cost operation.
> >> > > if you want to get latest feeded documents, you could use NRT
> >> > >
> >> > > On Tue, Feb 14, 2012 at 12:47 AM, Huy Le <hu...@springpartners.com>
> >> > wrote:
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > I am using solr 3.5. I seeing solr keeps creating new segment
> files
> >> > > (<1MB
> >> > > > files) so often that it triggers segment merge about every one
> >> minute.
> >> > I
> >> > > > search the news archive, but could not find any info on this
> issue.
> >> I
> >> > am
> >> > > > indexing about 10 docs of less 2KB each every second. Commit is
> >> called
> >> > > > after adding each document. Relevant config params are:
> >> > > >
> >> > > > <mergeFactor>10</mergeFactor>
> >> > > > <ramBufferSizeMB>1024</ramBufferSizeMB>
> >> > > > <maxMergeDocs>2147483647</maxMergeDocs>
> >> > > >
> >> > > > What might be triggering this frequent new segment files creation?
> >> > > Thanks!
> >> > > >
> >> > > > Huy
> >> > > >
> >> > > > --
> >> > > > Huy Le
> >> > > > Spring Partners, Inc.
> >> > > > http://springpadit.com
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Huy Le
> >> > Spring Partners, Inc.
> >> > http://springpadit.com
> >> >
> >>
> >
> >
> >
> > --
> > Huy Le
> > Spring Partners, Inc.
> > http://springpadit.com
> >
>
>
>
> --
> Huy Le
> Spring Partners, Inc.
> http://springpadit.com
>
Re: New segment file created too often
Posted by Huy Le <hu...@springpartners.com>.
Li and Erick,
Thanks for your help!
Huy
On Tue, Feb 14, 2012 at 9:28 AM, Erick Erickson <er...@gmail.com>wrote:
> bq: I don't understand why new segment is created per
> commit call
>
> Because that's how it's designed from the start. Solr guarantees
> that segments are write-once, that's how searchers get a
> snapshot of the index, they open up all the segments when
> they are opened. But to do this, it requires that the segments
> do NOT change. So, when you start indexing data to the master,
> a new segment is opened, and until you issue a commit, that
> data is not visible for the searchers. Once you commit
> (or once one of the autocommit parameters triggers a commit),
> currently open segment files are closed and the searchers are
> notified that the index has changed. Then they open all
> closed segments and you get to search the new data.
>
> Through the 3x code line, this behavior is just one of the basic
> assumptions built into the way Solr works, commits are the
> trigger for index changes to be made visible, and closing
> segments is the mechanism for enforcing snapshots of the
> index.
>
> Solr 4.0 and the NRT and SolrCloud work will change this
> somewhat with "soft commits", but that's another story.
>
> Best
> Erick
>
>
>
> On Mon, Feb 13, 2012 at 2:24 PM, Huy Le <hu...@springpartners.com> wrote:
> > Hi,
> >
> > This turned out to be a two-issue question. Issue one is that segment
> > files get merged too frequent. Issue two is that new segment file is
> > created on commit call.
> >
> > Looks like issue one is the behavior of tieredMergePolicy. This
> > environment is newly setup, and the segment files are small. And
> > mergeFactor is set at 10. Because the segment files are small, they are
> > all of a similar file size, this trigger merge call by tier merge policy
> > when there are 10 segments. So this is not an issue; it works as
> designed.
> >
> > Issue two, however, I don't understand why new segment is created per
> > commit call? Does this have anything to do with replication setup I have
> > (<str name="replicateAfter">commit</str>)?
> >
> > Huy
> >
> >
> > On Mon, Feb 13, 2012 at 1:18 PM, Huy Le <hu...@springpartners.com>
> wrote:
> >
> >> I do have the indexDefault and mainIndex sections configured with the
> >> desired values. As trying to get to the root cause of this issue, I
> have
> >> the application disable commit calls, and segment files no longer that
> >> created/merged so often. I then manually call commit (curl '
> >> http://localhost:8983/solr/sbr01/update?commit=true'), segment files
> get
> >> merged immediately. This suggest somethings to do with commits.
> >>
> >> Here is content of the config file:
> >>
> >>
> >> <?xml version="1.0" encoding="UTF-8" ?>
> >> <!--
> >> Licensed to the Apache Software Foundation (ASF) under one or more
> >> contributor license agreements. See the NOTICE file distributed with
> >> this work for additional information regarding copyright ownership.
> >> The ASF licenses this file to You under the Apache License, Version 2.0
> >> (the "License"); you may not use this file except in compliance with
> >> the License. You may obtain a copy of the License at
> >>
> >> http://www.apache.org/licenses/LICENSE-2.0
> >>
> >> Unless required by applicable law or agreed to in writing, software
> >> distributed under the License is distributed on an "AS IS" BASIS,
> >> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> >> See the License for the specific language governing permissions and
> >> limitations under the License.
> >> -->
> >>
> >> <!--
> >> For more details about configurations options that may appear in
> >> this file, see http://wiki.apache.org/solr/SolrConfigXml.
> >> -->
> >> <config>
> >> <!-- In all configuration below, a prefix of "solr." for class names
> >> is an alias that causes solr to search appropriate packages,
> >> including org.apache.solr.(search|update|request|core|analysis)
> >>
> >> You may also specify a fully qualified Java classname if you
> >> have your own custom plugins.
> >> -->
> >>
> >> <!-- Set this to 'false' if you want solr to continue working after
> >> it has encountered an severe configuration error. In a
> >> production environment, you may want solr to keep working even
> >> if one handler is mis-configured.
> >>
> >> You may also set this to false using by setting the system
> >> property:
> >>
> >> -Dsolr.abortOnConfigurationError=false
> >> -->
> >>
> >>
> <abortOnConfigurationError>${solr.abortOnConfigurationError:true}</abortOnConfigurationError>
> >>
> >> <!-- Controls what version of Lucene various components of Solr
> >> adhere to. Generally, you want to use the latest version to
> >> get all bug fixes and improvements. It is highly recommended
> >> that you fully re-index after changing this setting as it can
> >> affect both how text is indexed and queried.
> >> -->
> >> <luceneMatchVersion>LUCENE_35</luceneMatchVersion>
> >>
> >> <!-- lib directives can be used to instruct Solr to load an Jars
> >> identified and use them to resolve any "plugins" specified in
> >> your solrconfig.xml or schema.xml (ie: Analyzers, Request
> >> Handlers, etc...).
> >>
> >> All directories and paths are resolved relative to the
> >> instanceDir.
> >>
> >> If a "./lib" directory exists in your instanceDir, all files
> >> found in it are included as if you had used the following
> >> syntax...
> >>
> >> <lib dir="./lib" />
> >> -->
> >>
> >> <!-- A dir option by itself adds any files found in the directory to
> >> the classpath, this is useful for including all jars in a
> >> directory.
> >> -->
> >> <lib dir="../../contrib/extraction/lib" />
> >> <lib dir="../../contrib/clustering/lib/" />
> >> <lib dir="../../contrib/velocity/lib" />
> >>
> >> <!-- When a regex is specified in addition to a directory, only the
> >> files in that directory which completely match the regex
> >> (anchored on both ends) will be included.
> >> -->
> >> <lib dir="../../dist/" regex="apache-solr-cell-\d.*\.jar" />
> >> <lib dir="../../dist/" regex="apache-solr-clustering-\d.*\.jar" />
> >> <lib dir="../../dist/"
> regex="apache-solr-dataimporthandler-\d.*\.jar" />
> >> <lib dir="../../dist/" regex="apache-solr-langid-\d.*\.jar" />
> >> <lib dir="../../dist/" regex="apache-solr-velocity-\d.*\.jar" />
> >>
> >> <!-- If a dir option (with or without a regex) is used and nothing
> >> is found that matches, it will be ignored
> >> -->
> >> <lib dir="../../contrib/clustering/lib/" />
> >> <lib dir="/total/crap/dir/ignored" />
> >>
> >> <!-- an exact path can be used to specify a specific file. This
> >> will cause a serious error to be logged if it can't be loaded.
> >> -->
> >> <!--
> >> <lib path="../a-jar-that-does-not-exist.jar" />
> >> -->
> >>
> >> <!-- Data Directory
> >>
> >> Used to specify an alternate directory to hold all index data
> >> other than the default ./data under the Solr home. If
> >> replication is in use, this should match the replication
> >> configuration.
> >> -->
> >> <dataDir>${solr.data.dir:/u01/solr/sbr01/data}</dataDir>
> >>
> >>
> >> <!-- The DirectoryFactory to use for indexes.
> >>
> >> solr.StandardDirectoryFactory, the default, is filesystem
> >> based and tries to pick the best implementation for the current
> >> JVM and platform. One can force a particular implementation
> >> via solr.MMapDirectoryFactory, solr.NIOFSDirectoryFactory, or
> >> solr.SimpleFSDirectoryFactory.
> >>
> >> solr.RAMDirectoryFactory is memory based, not
> >> persistent, and doesn't work with replication.
> >> -->
> >> <directoryFactory name="DirectoryFactory"
> >>
> >> class="${solr.directoryFactory:solr.StandardDirectoryFactory}"/>
> >>
> >>
> >> <!-- Index Defaults
> >>
> >> Values here affect all index writers and act as a default
> >> unless overridden.
> >>
> >> WARNING: See also the <mainIndex> section below for parameters
> >> that overfor Solr's main Lucene index.
> >> -->
> >> <indexDefaults>
> >>
> >> <useCompoundFile>true</useCompoundFile>
> >>
> >>
> >> <mergeFactor>10</mergeFactor>
> >> <!-- Sets the amount of RAM that may be used by Lucene indexing
> >> for buffering added documents and deletions before they are
> >> flushed to the Directory. -->
> >> <ramBufferSizeMB>1024</ramBufferSizeMB>
> >>
> >> <!-- If both ramBufferSizeMB and maxBufferedDocs is set, then
> >> Lucene will flush based on whichever limit is hit first.
> >> -->
> >> <!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
> >> <!-- springpad configure -->
> >> <maxMergeDocs>2147483647</maxMergeDocs>
> >>
> >> <!-- end springpad configure -->
> >>
> >>
> >> <maxFieldLength>10000</maxFieldLength>
> >> <writeLockTimeout>1000</writeLockTimeout>
> >>
> >> <!-- Expert: Merge Policy
> >>
> >> The Merge Policy in Lucene controls how merging is handled by
> >> Lucene. The default in Solr 3.3 is TieredMergePolicy.
> >>
> >> The default in 2.3 was the LogByteSizeMergePolicy,
> >> previous versions used LogDocMergePolicy.
> >>
> >> LogByteSizeMergePolicy chooses segments to merge based on
> >> their size. The Lucene 2.2 default, LogDocMergePolicy chose
> >> when to merge based on number of documents
> >>
> >> Other implementations of MergePolicy must have a no-argument
> >> constructor
> >> -->
> >> <!--
> >> <mergePolicy class="org.apache.lucene.index.TieredMergePolicy"/>
> >> -->
> >>
> >> <!-- Expert: Merge Scheduler
> >>
> >> The Merge Scheduler in Lucene controls how merges are
> >> performed. The ConcurrentMergeScheduler (Lucene 2.3 default)
> >> can perform merges in the background using separate threads.
> >> The SerialMergeScheduler (Lucene 2.2 default) does not.
> >> -->
> >> <!--
> >> <mergeScheduler
> >> class="org.apache.lucene.index.ConcurrentMergeScheduler"/>
> >> -->
> >>
> >> <!-- LockFactory
> >>
> >> This option specifies which Lucene LockFactory implementation
> >> to use.
> >>
> >> single = SingleInstanceLockFactory - suggested for a
> >> read-only index or when there is no possibility of
> >> another process trying to modify the index.
> >> native = NativeFSLockFactory - uses OS native file locking.
> >> Do not use when multiple solr webapps in the same
> >> JVM are attempting to share a single index.
> >> simple = SimpleFSLockFactory - uses a plain file for locking
> >>
> >> (For backwards compatibility with Solr 1.2, 'simple' is the
> >> default if not specified.)
> >>
> >> More details on the nuances of each LockFactory...
> >> http://wiki.apache.org/lucene-java/AvailableLockFactories
> >> -->
> >> <lockType>native</lockType>
> >>
> >> <!-- Expert: Controls how often Lucene loads terms into memory
> >> Default is 128 and is likely good for most everyone.
> >> -->
> >>
> >> <!-- <termIndexInterval>256</termIndexInterval> -->
> >> </indexDefaults>
> >>
> >> <!-- Main Index
> >>
> >> Values here override the values in the <indexDefaults> section
> >> for the main on disk index.
> >> -->
> >> <mainIndex>
> >>
> >> <maxMergeDocs>2147483647</maxMergeDocs>
> >>
> >> <useCompoundFile>true</useCompoundFile>
> >> <ramBufferSizeMB>1024</ramBufferSizeMB>
> >> <mergeFactor>30</mergeFactor>
> >>
> >> <!-- Unlock On Startup
> >>
> >> If true, unlock any held write or commit locks on startup.
> >> This defeats the locking mechanism that allows multiple
> >> processes to safely access a lucene index, and should be used
> >> with care.
> >>
> >> This is not needed if lock type is 'none' or 'single'
> >> -->
> >> <unlockOnStartup>false</unlockOnStartup>
> >>
> >> <!-- If true, IndexReaders will be reopened (often more efficient)
> >> instead of closed and then opened.
> >> -->
> >> <reopenReaders>true</reopenReaders>
> >>
> >> <!-- Commit Deletion Policy
> >>
> >> Custom deletion policies can specified here. The class must
> >> implement org.apache.lucene.index.IndexDeletionPolicy.
> >>
> >>
> >>
> http://lucene.apache.org/java/2_9_1/api/all/org/apache/lucene/index/IndexDeletionPolicy.html
> >>
> >> The standard Solr IndexDeletionPolicy implementation supports
> >> deleting index commit points on number of commits, age of
> >> commit point and optimized status.
> >>
> >> The latest commit point should always be preserved regardless
> >> of the criteria.
> >> -->
> >> <deletionPolicy class="solr.SolrDeletionPolicy">
> >> <!-- The number of commit points to be kept -->
> >> <str name="maxCommitsToKeep">1</str>
> >> <!-- The number of optimized commit points to be kept -->
> >> <str name="maxOptimizedCommitsToKeep">0</str>
> >> <!--
> >> Delete all commit points once they have reached the given age.
> >> Supports DateMathParser syntax e.g.
> >> -->
> >> <!--
> >> <str name="maxCommitAge">30MINUTES</str>
> >> <str name="maxCommitAge">1DAY</str>
> >> -->
> >> </deletionPolicy>
> >>
> >> <!-- Lucene Infostream
> >>
> >> To aid in advanced debugging, Lucene provides an "InfoStream"
> >> of detailed information when indexing.
> >>
> >> Setting The value to true will instruct the underlying Lucene
> >> IndexWriter to write its debugging info the specified file
> >> -->
> >> <infoStream file="INFOSTREAM.txt">false</infoStream>
> >>
> >> </mainIndex>
> >>
> >> <!-- JMX
> >>
> >> This example enables JMX if and only if an existing MBeanServer
> >> is found, use this if you want to configure JMX through JVM
> >> parameters. Remove this to disable exposing Solr configuration
> >> and statistics to JMX.
> >>
> >> For more details see http://wiki.apache.org/solr/SolrJmx
> >> -->
> >> <jmx />
> >> <!-- If you want to connect to a particular server, specify the
> >> agentId
> >> -->
> >> <!-- <jmx agentId="myAgent" /> -->
> >> <!-- If you want to start a new MBeanServer, specify the serviceUrl
> -->
> >> <!-- <jmx
> >> serviceUrl="service:jmx:rmi:///jndi/rmi://localhost:9999/solr"/>
> >> -->
> >>
> >> <!-- The default high-performance update handler -->
> >> <updateHandler class="solr.DirectUpdateHandler2">
> >>
> >> <!-- AutoCommit
> >>
> >> Perform a <commit/> automatically under certain conditions.
> >> Instead of enabling autoCommit, consider using "commitWithin"
> >> when adding documents.
> >>
> >> http://wiki.apache.org/solr/UpdateXmlMessages
> >>
> >> maxDocs - Maximum number of documents to add since the last
> >> commit before automatically triggering a new commit.
> >>
> >> maxTime - Maximum amount of time that is allowed to pass
> >> since a document was added before automaticly
> >> triggering a new commit.
> >> -->
> >> <autoCommit>
> >> <maxDocs>10000</maxDocs>
> >> <maxTime>300000</maxTime>
> >> </autoCommit>
> >>
> >> <!-- Update Related Event Listeners
> >>
> >> Various IndexWriter related events can trigger Listeners to
> >> take actions.
> >>
> >> postCommit - fired after every commit or optimize command
> >> postOptimize - fired after every optimize command
> >> -->
> >> <!-- The RunExecutableListener executes an external command from a
> >> hook such as postCommit or postOptimize.
> >>
> >> exe - the name of the executable to run
> >> dir - dir to use as the current working directory.
> (default=".")
> >> wait - the calling thread waits until the executable returns.
> >> (default="true")
> >> args - the arguments to pass to the program. (default is none)
> >> env - environment variables to set. (default is none)
> >> -->
> >> <!-- This example shows how RunExecutableListener could be used
> >> with the script based replication...
> >> http://wiki.apache.org/solr/CollectionDistribution
> >> -->
> >> <!--
> >> <listener event="postCommit" class="solr.RunExecutableListener">
> >> <str name="exe">solr/bin/snapshooter</str>
> >> <str name="dir">.</str>
> >> <bool name="wait">true</bool>
> >> <arr name="args"> <str>arg1</str> <str>arg2</str> </arr>
> >> <arr name="env"> <str>MYVAR=val1</str> </arr>
> >> </listener>
> >> -->
> >> </updateHandler>
> >>
> >> <!-- IndexReaderFactory
> >>
> >> Use the following format to specify a custom IndexReaderFactory,
> >> which allows for alternate IndexReader implementations.
> >>
> >> ** Experimental Feature **
> >>
> >> Please note - Using a custom IndexReaderFactory may prevent
> >> certain other features from working. The API to
> >> IndexReaderFactory may change without warning or may even be
> >> removed from future releases if the problems cannot be
> >> resolved.
> >>
> >>
> >> ** Features that may not work with custom IndexReaderFactory **
> >>
> >> The ReplicationHandler assumes a disk-resident index. Using a
> >> custom IndexReader implementation may cause incompatibility
> >> with ReplicationHandler and may cause replication to not work
> >> correctly. See SOLR-1366 for details.
> >>
> >> -->
> >> <!--
> >> <indexReaderFactory name="IndexReaderFactory" class="package.class">
> >> <str name="someArg">Some Value</str>
> >> </indexReaderFactory >
> >> -->
> >> <!-- By explicitly declaring the Factory, the termIndexDivisor can
> >> be specified.
> >> -->
> >> <!--
> >> <indexReaderFactory name="IndexReaderFactory"
> >> class="solr.StandardIndexReaderFactory">
> >> <int name="setTermIndexDivisor">12</int>
> >> </indexReaderFactory >
> >> -->
> >>
> >>
> >> <query>
> >> <!-- Max Boolean Clauses
> >>
> >> Maximum number of clauses in each BooleanQuery, an exception
> >> is thrown if exceeded.
> >>
> >> ** WARNING **
> >>
> >> This option actually modifies a global Lucene property that
> >> will affect all SolrCores. If multiple solrconfig.xml files
> >> disagree on this property, the value at any given moment will
> >> be based on the last SolrCore to be initialized.
> >>
> >> -->
> >> <maxBooleanClauses>1024</maxBooleanClauses>
> >>
> >>
> >> <!-- Solr Internal Query Caches
> >>
> >> There are two implementations of cache available for Solr,
> >> LRUCache, based on a synchronized LinkedHashMap, and
> >> FastLRUCache, based on a ConcurrentHashMap.
> >>
> >> FastLRUCache has faster gets and slower puts in single
> >> threaded operation and thus is generally faster than LRUCache
> >> when the hit ratio of the cache is high (> 75%), and may be
> >> faster under other scenarios on multi-cpu systems.
> >> -->
> >>
> >> <!-- Filter Cache
> >>
> >> Cache used by SolrIndexSearcher for filters (DocSets),
> >> unordered sets of *all* documents that match a query. When a
> >> new searcher is opened, its caches may be prepopulated or
> >> "autowarmed" using data from caches in the old searcher.
> >> autowarmCount is the number of items to prepopulate. For
> >> LRUCache, the autowarmed items will be the most recently
> >> accessed items.
> >>
> >> Parameters:
> >> class - the SolrCache implementation LRUCache or
> >> (LRUCache or FastLRUCache)
> >> size - the maximum number of entries in the cache
> >> initialSize - the initial capacity (number of entries) of
> >> the cache. (see java.util.HashMap)
> >> autowarmCount - the number of entries to prepopulate from
> >> and old cache.
> >> -->
> >> <filterCache class="solr.FastLRUCache"
> >> size="20000"
> >> initialSize="1000"
> >> autowarmCount="0"/>
> >>
> >> <!-- Query Result Cache
> >>
> >> Caches results of searches - ordered lists of document ids
> >> (DocList) based on a query, a sort, and the range of documents
> >> requested.
> >> -->
> >> <queryResultCache class="solr.LRUCache"
> >> size="10000"
> >> initialSize="1000"
> >> autowarmCount="0"/>
> >>
> >> <!-- Document Cache
> >>
> >> Caches Lucene Document objects (the stored fields for each
> >> document). Since Lucene internal document ids are transient,
> >> this cache will not be autowarmed.
> >> -->
> >> <documentCache class="solr.LRUCache"
> >> size="15000"
> >> initialSize="1000"
> >> autowarmCount="0"/>
> >>
> >> <!-- Field Value Cache
> >>
> >> Cache used to hold field values that are quickly accessible
> >> by document id. The fieldValueCache is created by default
> >> even if not configured here.
> >> -->
> >> <!--
> >> <fieldValueCache class="solr.FastLRUCache"
> >> size="512"
> >> autowarmCount="128"
> >> showItems="32" />
> >> -->
> >>
> >> <!-- Custom Cache
> >>
> >> Example of a generic cache. These caches may be accessed by
> >> name through SolrIndexSearcher.getCache(),cacheLookup(), and
> >> cacheInsert(). The purpose is to enable easy caching of
> >> user/application level data. The regenerator argument should
> >> be specified as an implementation of solr.CacheRegenerator
> >> if autowarming is desired.
> >> -->
> >> <!--
> >> <cache name="myUserCache"
> >> class="solr.LRUCache"
> >> size="4096"
> >> initialSize="1024"
> >> autowarmCount="1024"
> >> regenerator="com.mycompany.MyRegenerator"
> >> />
> >> -->
> >>
> >>
> >> <!-- Lazy Field Loading
> >>
> >> If true, stored fields that are not requested will be loaded
> >> lazily. This can result in a significant speed improvement
> >> if the usual case is to not load all stored fields,
> >> especially if the skipped fields are large compressed text
> >> fields.
> >> -->
> >> <enableLazyFieldLoading>true</enableLazyFieldLoading>
> >>
> >> <!-- Use Filter For Sorted Query
> >>
> >> A possible optimization that attempts to use a filter to
> >> satisfy a search. If the requested sort does not include
> >> score, then the filterCache will be checked for a filter
> >> matching the query. If found, the filter will be used as the
> >> source of document ids, and then the sort will be applied to
> >> that.
> >>
> >> For most situations, this will not be useful unless you
> >> frequently get the same search repeatedly with different sort
> >> options, and none of them ever use "score"
> >> -->
> >> <!--
> >> <useFilterForSortedQuery>true</useFilterForSortedQuery>
> >> -->
> >>
> >> <!-- Result Window Size
> >>
> >> An optimization for use with the queryResultCache. When a
> search
> >> is requested, a superset of the requested number of document ids
> >> are collected. For example, if a search for a particular query
> >> requests matching documents 10 through 19, and queryWindowSize
> is
> >> 50,
> >> then documents 0 through 49 will be collected and cached. Any
> >> further
> >> requests in that range can be satisfied via the cache.
> >> -->
> >> <queryResultWindowSize>50</queryResultWindowSize>
> >>
> >> <!-- Maximum number of documents to cache for any entry in the
> >> queryResultCache.
> >> -->
> >> <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
> >>
> >> <!-- Query Related Event Listeners
> >>
> >> Various IndexSearcher related events can trigger Listeners to
> >> take actions.
> >>
> >> newSearcher - fired whenever a new searcher is being prepared
> >> and there is a current searcher handling requests (aka
> >> registered). It can be used to prime certain caches to
> >> prevent long request times for certain requests.
> >>
> >> firstSearcher - fired whenever a new searcher is being
> >> prepared but there is no current registered searcher to handle
> >> requests or to gain autowarming data from.
> >>
> >>
> >> -->
> >> <!-- QuerySenderListener takes an array of NamedList and executes a
> >> local query request for each NamedList in sequence.
> >> -->
> >> <listener event="newSearcher" class="solr.QuerySenderListener">
> >> <arr name="queries">
> >> <!--
> >> <lst><str name="q">solr</str><str name="sort">price
> >> asc</str></lst>
> >> <lst><str name="q">rocks</str><str name="sort">weight
> >> asc</str></lst>
> >> -->
> >> </arr>
> >> </listener>
> >> <listener event="firstSearcher" class="solr.QuerySenderListener">
> >> <arr name="queries">
> >> <lst>
> >> <str name="q">static firstSearcher warming in
> >> solrconfig.xml</str>
> >> </lst>
> >> </arr>
> >> </listener>
> >>
> >> <!-- Use Cold Searcher
> >>
> >> If a search request comes in and there is no current
> >> registered searcher, then immediately register the still
> >> warming searcher and use it. If "false" then all requests
> >> will block until the first searcher is done warming.
> >> -->
> >> <useColdSearcher>true</useColdSearcher>
> >>
> >> <!-- Max Warming Searchers
> >>
> >> Maximum number of searchers that may be warming in the
> >> background concurrently. An error is returned if this limit
> >> is exceeded.
> >>
> >> Recommend values of 1-2 for read-only slaves, higher for
> >> masters w/o cache warming.
> >> -->
> >> <maxWarmingSearchers>100</maxWarmingSearchers>
> >>
> >> </query>
> >>
> >>
> >> <!-- Request Dispatcher
> >>
> >> This section contains instructions for how the SolrDispatchFilter
> >> should behave when processing requests for this SolrCore.
> >>
> >> handleSelect affects the behavior of requests such as
> /select?qt=XXX
> >>
> >> handleSelect="true" will cause the SolrDispatchFilter to process
> >> the request and will result in consistent error handling and
> >> formatting for all types of requests.
> >>
> >> handleSelect="false" will cause the SolrDispatchFilter to
> >> ignore "/select" requests and fallback to using the legacy
> >> SolrServlet and it's Solr 1.1 style error formatting
> >> -->
> >> <requestDispatcher handleSelect="true" >
> >> <!-- Request Parsing
> >>
> >> These settings indicate how Solr Requests may be parsed, and
> >> what restrictions may be placed on the ContentStreams from
> >> those requests
> >>
> >> enableRemoteStreaming - enables use of the stream.file
> >> and stream.url parameters for specifying remote streams.
> >>
> >> multipartUploadLimitInKB - specifies the max size of
> >> Multipart File Uploads that Solr will allow in a Request.
> >>
> >> *** WARNING ***
> >> The settings below authorize Solr to fetch remote files, You
> >> should make sure your system has some authentication before
> >> using enableRemoteStreaming="true"
> >>
> >> -->
> >> <requestParsers enableRemoteStreaming="true"
> >> multipartUploadLimitInKB="2048000" />
> >>
> >> <!-- HTTP Caching
> >>
> >> Set HTTP caching related parameters (for proxy caches and
> >> clients).
> >>
> >> The options below instruct Solr not to output any HTTP Caching
> >> related headers
> >> -->
> >> <httpCaching never304="true" />
> >> <!-- If you include a <cacheControl> directive, it will be used to
> >> generate a Cache-Control header (as well as an Expires header
> >> if the value contains "max-age=")
> >>
> >> By default, no Cache-Control header is generated.
> >>
> >> You can use the <cacheControl> option even if you have set
> >> never304="true"
> >> -->
> >> <!--
> >> <httpCaching never304="true" >
> >> <cacheControl>max-age=30, public</cacheControl>
> >> </httpCaching>
> >> -->
> >> <!-- To enable Solr to respond with automatically generated HTTP
> >> Caching headers, and to response to Cache Validation requests
> >> correctly, set the value of never304="false"
> >>
> >> This will cause Solr to generate Last-Modified and ETag
> >> headers based on the properties of the Index.
> >>
> >> The following options can also be specified to affect the
> >> values of these headers...
> >>
> >> lastModFrom - the default value is "openTime" which means the
> >> Last-Modified value (and validation against If-Modified-Since
> >> requests) will all be relative to when the current Searcher
> >> was opened. You can change it to lastModFrom="dirLastMod" if
> >> you want the value to exactly correspond to when the physical
> >> index was last modified.
> >>
> >> etagSeed="..." is an option you can change to force the ETag
> >> header (and validation against If-None-Match requests) to be
> >> different even if the index has not changed (ie: when making
> >> significant changes to your config file)
> >>
> >> (lastModifiedFrom and etagSeed are both ignored if you use
> >> the never304="true" option)
> >> -->
> >> <!--
> >> <httpCaching lastModifiedFrom="openTime"
> >> etagSeed="Solr">
> >> <cacheControl>max-age=30, public</cacheControl>
> >> </httpCaching>
> >> -->
> >> </requestDispatcher>
> >>
> >> <!-- Request Handlers
> >>
> >> http://wiki.apache.org/solr/SolrRequestHandler
> >>
> >> incoming queries will be dispatched to the correct handler
> >> based on the path or the qt (query type) param.
> >>
> >> Names starting with a '/' are accessed with the a path equal to
> >> the registered name. Names without a leading '/' are accessed
> >> with: http://host/app/[core/]select?qt=name
> >>
> >> If a /select request is processed with out a qt param
> >> specified, the requestHandler that declares default="true" will
> >> be used.
> >>
> >> If a Request Handler is declared with startup="lazy", then it
> will
> >> not be initialized until the first request that uses it.
> >>
> >> -->
> >> <!-- SearchHandler
> >>
> >> http://wiki.apache.org/solr/SearchHandler
> >>
> >> For processing Search Queries, the primary Request Handler
> >> provided with Solr is "SearchHandler" It delegates to a sequent
> >> of SearchComponents (see below) and supports distributed
> >> queries across multiple shards
> >> -->
> >> <requestHandler name="search" class="solr.SearchHandler"
> default="true">
> >> <!-- default values for query parameters can be specified, these
> >> will be overridden by parameters in the request
> >> -->
> >> <lst name="defaults">
> >> <str name="echoParams">explicit</str>
> >> <int name="rows">10</int>
> >> </lst>
> >> <!-- In addition to defaults, "appends" params can be specified
> >> to identify values which should be appended to the list of
> >> multi-val params from the query (or the existing "defaults").
> >> -->
> >> <!-- In this example, the param "fq=instock:true" would be appended
> to
> >> any query time fq params the user may specify, as a mechanism
> for
> >> partitioning the index, independent of any user selected
> filtering
> >> that may also be desired (perhaps as a result of faceted
> >> searching).
> >>
> >> NOTE: there is *absolutely* nothing a client can do to prevent
> >> these
> >> "appends" values from being used, so don't use this mechanism
> >> unless you are sure you always want it.
> >> -->
> >> <!--
> >> <lst name="appends">
> >> <str name="fq">inStock:true</str>
> >> </lst>
> >> -->
> >> <!-- "invariants" are a way of letting the Solr maintainer lock down
> >> the options available to Solr clients. Any params values
> >> specified here are used regardless of what values may be
> specified
> >> in either the query, the "defaults", or the "appends" params.
> >>
> >> In this example, the facet.field and facet.query params would
> >> be fixed, limiting the facets clients can use. Faceting is
> >> not turned on by default - but if the client does specify
> >> facet=true in the request, these are the only facets they
> >> will be able to see counts for; regardless of what other
> >> facet.field or facet.query params they may specify.
> >>
> >> NOTE: there is *absolutely* nothing a client can do to prevent
> >> these
> >> "invariants" values from being used, so don't use this
> mechanism
> >> unless you are sure you always want it.
> >> -->
> >> <!--
> >> <lst name="invariants">
> >> <str name="facet.field">cat</str>
> >> <str name="facet.field">manu_exact</str>
> >> <str name="facet.query">price:[* TO 500]</str>
> >> <str name="facet.query">price:[500 TO *]</str>
> >> </lst>
> >> -->
> >> <!-- If the default list of SearchComponents is not desired, that
> >> list can either be overridden completely, or components can be
> >> prepended or appended to the default list. (see below)
> >> -->
> >> <!--
> >> <arr name="components">
> >> <str>nameOfCustomComponent1</str>
> >> <str>nameOfCustomComponent2</str>
> >> </arr>
> >> -->
> >> </requestHandler>
> >>
> >> <!-- A Robust Example
> >>
> >> This example SearchHandler declaration shows off usage of the
> >> SearchHandler with many defaults declared
> >>
> >> Note that multiple instances of the same Request Handler
> >> (SearchHandler) can be registered multiple times with different
> >> names (and different init parameters)
> >> -->
> >> <requestHandler name="/browse" class="solr.SearchHandler">
> >> <lst name="defaults">
> >> <str name="echoParams">explicit</str>
> >>
> >> <!-- VelocityResponseWriter settings -->
> >> <!---
> >> <str name="wt">velocity</str>
> >> -->
> >>
> >> <str name="v.template">browse</str>
> >> <str name="v.layout">layout</str>
> >> <str name="title">Solritas</str>
> >>
> >> <str name="defType">edismax</str>
> >> <str name="q.alt">*:*</str>
> >> <str name="rows">10</str>
> >> <str name="fl">*,score</str>
> >> <str name="mlt.qf">
> >> text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
> >> </str>
> >> <str name="mlt.fl">text,features,name,sku,id,manu,cat</str>
> >> <int name="mlt.count">3</int>
> >>
> >> <str name="qf">
> >> text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1
> cat^1.4
> >> </str>
> >>
> >> <str name="facet">on</str>
> >> <str name="facet.field">cat</str>
> >> <str name="facet.field">manu_exact</str>
> >> <str name="facet.query">ipod</str>
> >> <str name="facet.query">GB</str>
> >> <str name="facet.mincount">1</str>
> >> <str name="facet.pivot">cat,inStock</str>
> >> <str name="facet.range.other">after</str>
> >> <str name="facet.range">price</str>
> >> <int name="f.price.facet.range.start">0</int>
> >> <int name="f.price.facet.range.end">600</int>
> >> <int name="f.price.facet.range.gap">50</int>
> >> <str name="facet.range">popularity</str>
> >> <int name="f.popularity.facet.range.start">0</int>
> >> <int name="f.popularity.facet.range.end">10</int>
> >> <int name="f.popularity.facet.range.gap">3</int>
> >> <str name="facet.range">manufacturedate_dt</str>
> >> <str
> >> name="f.manufacturedate_dt.facet.range.start">NOW/YEAR-10YEARS</str>
> >> <str name="f.manufacturedate_dt.facet.range.end">NOW</str>
> >> <str name="f.manufacturedate_dt.facet.range.gap">+1YEAR</str>
> >> <str name="f.manufacturedate_dt.facet.range.other">before</str>
> >> <str name="f.manufacturedate_dt.facet.range.other">after</str>
> >>
> >>
> >> <!-- Highlighting defaults -->
> >> <str name="hl">on</str>
> >> <str name="hl.fl">text features name</str>
> >> <str name="f.name.hl.fragsize">0</str>
> >> <str name="f.name.hl.alternateField">name</str>
> >> </lst>
> >> <arr name="last-components">
> >> <str>spellcheck</str>
> >> </arr>
> >> <!--
> >> <str name="url-scheme">httpx</str>
> >> -->
> >> </requestHandler>
> >>
> >> <!-- XML Update Request Handler.
> >>
> >> http://wiki.apache.org/solr/UpdateXmlMessages
> >>
> >> The canonical Request Handler for Modifying the Index through
> >> commands specified using XML.
> >>
> >> Note: Since solr1.1 requestHandlers requires a valid content
> >> type header if posted in the body. For example, curl now
> >> requires: -H 'Content-type:text/xml; charset=utf-8'
> >> -->
> >> <requestHandler name="/update"
> >> class="solr.XmlUpdateRequestHandler">
> >> <!-- See below for information on defining
> >> updateRequestProcessorChains that can be used by name
> >> on each Update Request
> >> -->
> >> <!--
> >> <lst name="defaults">
> >> <str name="update.chain">dedupe</str>
> >> </lst>
> >> -->
> >> </requestHandler>
> >> <!-- Binary Update Request Handler
> >> http://wiki.apache.org/solr/javabin
> >> -->
> >> <requestHandler name="/update/javabin"
> >> class="solr.BinaryUpdateRequestHandler" />
> >>
> >> <!-- CSV Update Request Handler
> >> http://wiki.apache.org/solr/UpdateCSV
> >> -->
> >> <requestHandler name="/update/csv"
> >> class="solr.CSVRequestHandler"
> >> startup="lazy" />
> >>
> >> <!-- JSON Update Request Handler
> >> http://wiki.apache.org/solr/UpdateJSON
> >> -->
> >> <requestHandler name="/update/json"
> >> class="solr.JsonUpdateRequestHandler"
> >> startup="lazy" />
> >>
> >> <!-- Solr Cell Update Request Handler
> >>
> >> http://wiki.apache.org/solr/ExtractingRequestHandler
> >>
> >> -->
> >> <requestHandler name="/update/extract"
> >> startup="lazy"
> >> class="solr.extraction.ExtractingRequestHandler" >
> >> <lst name="defaults">
> >> <!-- All the main content goes into "text"... if you need to
> return
> >> the extracted text or do highlighting, use a stored field.
> -->
> >> <str name="fmap.content">text</str>
> >> <str name="lowernames">true</str>
> >> <str name="uprefix">ignored_</str>
> >>
> >> <!-- capture link hrefs but ignore div attributes -->
> >> <str name="captureAttr">true</str>
> >> <str name="fmap.a">links</str>
> >> <str name="fmap.div">ignored_</str>
> >> </lst>
> >> </requestHandler>
> >>
> >> <!-- XSLT Update Request Handler
> >> Transforms incoming XML with stylesheet identified by tr=
> >> -->
> >> <requestHandler name="/update/xslt"
> >> startup="lazy"
> >> class="solr.XsltUpdateRequestHandler"/>
> >>
> >> <!-- Field Analysis Request Handler
> >>
> >> RequestHandler that provides much the same functionality as
> >> analysis.jsp. Provides the ability to specify multiple field
> >> types and field names in the same request and outputs
> >> index-time and query-time analysis for each of them.
> >>
> >> Request parameters are:
> >> analysis.fieldname - field name whose analyzers are to be used
> >>
> >> analysis.fieldtype - field type whose analyzers are to be used
> >> analysis.fieldvalue - text for index-time analysis
> >> q (or analysis.q) - text for query time analysis
> >> analysis.showmatch (true|false) - When set to true and when
> >> query analysis is performed, the produced tokens of the
> >> field value analysis will be marked as "matched" for every
> >> token that is produces by the query analysis
> >> -->
> >> <requestHandler name="/analysis/field"
> >> startup="lazy"
> >> class="solr.FieldAnalysisRequestHandler" />
> >>
> >>
> >> <!-- Document Analysis Handler
> >>
> >> http://wiki.apache.org/solr/AnalysisRequestHandler
> >>
> >> An analysis handler that provides a breakdown of the analysis
> >> process of provided docuemnts. This handler expects a (single)
> >> content stream with the following format:
> >>
> >> <docs>
> >> <doc>
> >> <field name="id">1</field>
> >> <field name="name">The Name</field>
> >> <field name="text">The Text Value</field>
> >> </doc>
> >> <doc>...</doc>
> >> <doc>...</doc>
> >> ...
> >> </docs>
> >>
> >> Note: Each document must contain a field which serves as the
> >> unique key. This key is used in the returned response to associate
> >> an analysis breakdown to the analyzed document.
> >>
> >> Like the FieldAnalysisRequestHandler, this handler also supports
> >> query analysis by sending either an "analysis.query" or "q"
> >> request parameter that holds the query text to be analyzed. It
> >> also supports the "analysis.showmatch" parameter which when set to
> >> true, all field tokens that match the query tokens will be marked
> >> as a "match".
> >> -->
> >> <requestHandler name="/analysis/document"
> >> class="solr.DocumentAnalysisRequestHandler"
> >> startup="lazy" />
> >>
> >> <!-- Admin Handlers
> >>
> >> Admin Handlers - This will register all the standard admin
> >> RequestHandlers.
> >> -->
> >> <requestHandler name="/admin/"
> >> class="solr.admin.AdminHandlers" />
> >> <!-- This single handler is equivalent to the following... -->
> >> <!--
> >> <requestHandler name="/admin/luke"
> >> class="solr.admin.LukeRequestHandler" />
> >> <requestHandler name="/admin/system"
> >> class="solr.admin.SystemInfoHandler" />
> >> <requestHandler name="/admin/plugins"
> >> class="solr.admin.PluginInfoHandler" />
> >> <requestHandler name="/admin/threads"
> >> class="solr.admin.ThreadDumpHandler" />
> >> <requestHandler name="/admin/properties"
> >> class="solr.admin.PropertiesRequestHandler" />
> >> <requestHandler name="/admin/file"
> >> class="solr.admin.ShowFileRequestHandler" >
> >> -->
> >> <!-- If you wish to hide files under ${solr.home}/conf, explicitly
> >> register the ShowFileRequestHandler using:
> >> -->
> >> <!--
> >> <requestHandler name="/admin/file"
> >> class="solr.admin.ShowFileRequestHandler" >
> >> <lst name="invariants">
> >> <str name="hidden">synonyms.txt</str>
> >> <str name="hidden">anotherfile.txt</str>
> >> </lst>
> >> </requestHandler>
> >> -->
> >>
> >> <!-- ping/healthcheck -->
> >> <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
> >> <lst name="invariants">
> >> <str name="qt">search</str>
> >> <str name="q">solrpingquery</str>
> >> </lst>
> >> <lst name="defaults">
> >> <str name="echoParams">all</str>
> >> </lst>
> >> </requestHandler>
> >>
> >> <!-- Echo the request contents back to the client -->
> >> <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
> >> <lst name="defaults">
> >> <str name="echoParams">explicit</str>
> >> <str name="echoHandler">true</str>
> >> </lst>
> >> </requestHandler>
> >>
> >> <!-- Solr Replication
> >>
> >> The SolrReplicationHandler supports replicating indexes from a
> >> "master" used for indexing and "salves" used for queries.
> >>
> >> http://wiki.apache.org/solr/SolrReplication
> >>
> >> In the example below, remove the <lst name="master"> section if
> >> this is just a slave and remove the <lst name="slave"> section
> >> if this is just a master.
> >> -->
> >> <!--
> >> <requestHandler name="/replication"
> class="solr.ReplicationHandler" >
> >> <lst name="master">
> >> <str name="replicateAfter">commit</str>
> >> <str name="replicateAfter">startup</str>
> >> <str name="confFiles">schema.xml,stopwords.txt</str>
> >> </lst>
> >> <lst name="slave">
> >> <str name="masterUrl">http://localhost:8983/solr/replication
> >> </str>
> >> <str name="pollInterval">00:00:60</str>
> >> </lst>
> >> </requestHandler>
> >> -->
> >>
> >> <!-- Search Components
> >>
> >> Search components are registered to SolrCore and used by
> >> instances of SearchHandler (which can access them by name)
> >>
> >> By default, the following components are available:
> >>
> >> <searchComponent name="query" class="solr.QueryComponent" />
> >> <searchComponent name="facet" class="solr.FacetComponent" />
> >> <searchComponent name="mlt"
> >> class="solr.MoreLikeThisComponent" />
> >> <searchComponent name="highlight"
> class="solr.HighlightComponent" />
> >> <searchComponent name="stats" class="solr.StatsComponent" />
> >> <searchComponent name="debug" class="solr.DebugComponent" />
> >>
> >> Default configuration in a requestHandler would look like:
> >>
> >> <arr name="components">
> >> <str>query</str>
> >> <str>facet</str>
> >> <str>mlt</str>
> >> <str>highlight</str>
> >> <str>stats</str>
> >> <str>debug</str>
> >> </arr>
> >>
> >> If you register a searchComponent to one of the standard names,
> >> that will be used instead of the default.
> >>
> >> To insert components before or after the 'standard' components,
> use:
> >>
> >> <arr name="first-components">
> >> <str>myFirstComponentName</str>
> >> </arr>
> >>
> >> <arr name="last-components">
> >> <str>myLastComponentName</str>
> >> </arr>
> >>
> >> NOTE: The component registered with the name "debug" will
> >> always be executed after the "last-components"
> >>
> >> -->
> >>
> >> <!-- Spell Check
> >>
> >> The spell check component can return a list of alternative
> spelling
> >> suggestions.
> >>
> >> http://wiki.apache.org/solr/SpellCheckComponent
> >> -->
> >> <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
> >>
> >> <str name="queryAnalyzerFieldType">textSpell</str>
> >>
> >> <!-- Multiple "Spell Checkers" can be declared and used by this
> >> component
> >> -->
> >>
> >> <!-- a spellchecker built from a field of the main index, and
> >> written to disk
> >> -->
> >> <lst name="spellchecker">
> >> <str name="name">default</str>
> >> <str name="field">name</str>
> >> <str name="spellcheckIndexDir">spellchecker</str>
> >> <!-- uncomment this to require terms to occur in 1% of the
> documents
> >> in order to be included in the dictionary
> >> <float name="thresholdTokenFrequency">.01</float>
> >> -->
> >> </lst>
> >>
> >> <!-- a spellchecker that uses a different distance measure -->
> >> <!--
> >> <lst name="spellchecker">
> >> <str name="name">jarowinkler</str>
> >> <str name="field">spell</str>
> >> <str name="distanceMeasure">
> >> org.apache.lucene.search.spell.JaroWinklerDistance
> >> </str>
> >> <str name="spellcheckIndexDir">spellcheckerJaro</str>
> >> </lst>
> >> -->
> >>
> >> <!-- a spellchecker that use an alternate comparator
> >>
> >> comparatorClass be one of:
> >> 1. score (default)
> >> 2. freq (Frequency first, then score)
> >> 3. A fully qualified class name
> >> -->
> >> <!--
> >> <lst name="spellchecker">
> >> <str name="name">freq</str>
> >> <str name="field">lowerfilt</str>
> >> <str name="spellcheckIndexDir">spellcheckerFreq</str>
> >> <str name="comparatorClass">freq</str>
> >> <str name="buildOnCommit">true</str>
> >> -->
> >>
> >> <!-- A spellchecker that reads the list of words from a file -->
> >> <!--
> >> <lst name="spellchecker">
> >> <str name="classname">solr.FileBasedSpellChecker</str>
> >> <str name="name">file</str>
> >> <str name="sourceLocation">spellings.txt</str>
> >> <str name="characterEncoding">UTF-8</str>
> >> <str name="spellcheckIndexDir">spellcheckerFile</str>
> >> </lst>
> >> -->
> >> </searchComponent>
> >>
> >> <!-- A request handler for demonstrating the spellcheck component.
> >>
> >> NOTE: This is purely as an example. The whole purpose of the
> >> SpellCheckComponent is to hook it into the request handler that
> >> handles your normal user queries so that a separate request is
> >> not needed to get suggestions.
> >>
> >> IN OTHER WORDS, THERE IS REALLY GOOD CHANCE THE SETUP BELOW IS
> >> NOT WHAT YOU WANT FOR YOUR PRODUCTION SYSTEM!
> >>
> >> See http://wiki.apache.org/solr/SpellCheckComponent for details
> >> on the request parameters.
> >> -->
> >> <requestHandler name="/spell" class="solr.SearchHandler"
> startup="lazy">
> >> <lst name="defaults">
> >> <str name="spellcheck.onlyMorePopular">false</str>
> >> <str name="spellcheck.extendedResults">false</str>
> >> <str name="spellcheck.count">1</str>
> >> </lst>
> >> <arr name="last-components">
> >> <str>spellcheck</str>
> >> </arr>
> >> </requestHandler>
> >>
> >> <!-- Term Vector Component
> >>
> >> http://wiki.apache.org/solr/TermVectorComponent
> >> -->
> >> <searchComponent name="tvComponent" class="solr.TermVectorComponent"/>
> >>
> >> <!-- A request handler for demonstrating the term vector component
> >>
> >> This is purely as an example.
> >>
> >> In reality you will likely want to add the component to your
> >> already specified request handlers.
> >> -->
> >> <requestHandler name="tvrh" class="solr.SearchHandler" startup="lazy">
> >> <lst name="defaults">
> >> <bool name="tv">true</bool>
> >> </lst>
> >> <arr name="last-components">
> >> <str>tvComponent</str>
> >> </arr>
> >> </requestHandler>
> >>
> >> <!-- Clustering Component
> >>
> >> http://wiki.apache.org/solr/ClusteringComponent
> >>
> >> This relies on third party jars which are notincluded in the
> >> release. To use this component (and the "/clustering" handler)
> >> Those jars will need to be downloaded, and you'll need to set
> >> the solr.cluster.enabled system property when running solr...
> >>
> >> java -Dsolr.clustering.enabled=true -jar start.jar
> >> -->
> >> <searchComponent name="clustering"
> >> enable="${solr.clustering.enabled:false}"
> >> class="solr.clustering.ClusteringComponent" >
> >> <!-- Declare an engine -->
> >> <lst name="engine">
> >> <!-- The name, only one can be named "default" -->
> >> <str name="name">default</str>
> >>
> >> <!-- Class name of Carrot2 clustering algorithm.
> >>
> >> Currently available algorithms are:
> >>
> >> * org.carrot2.clustering.lingo.LingoClusteringAlgorithm
> >> * org.carrot2.clustering.stc.STCClusteringAlgorithm
> >> *
> >> org.carrot2.clustering.kmeans.BisectingKMeansClusteringAlgorithm
> >>
> >> See http://project.carrot2.org/algorithms.html for the
> >> algorithm's characteristics.
> >> -->
> >> <str
> >>
> name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm</str>
> >>
> >> <!-- Overriding values for Carrot2 default algorithm attributes.
> >>
> >> For a description of all available attributes, see:
> >>
> http://download.carrot2.org/stable/manual/#chapter.components.
> >> Use attribute key as name attribute of str elements
> >> below. These can be further overridden for individual
> >> requests by specifying attribute key as request parameter
> >> name and attribute value as parameter value.
> >> -->
> >> <str
> name="LingoClusteringAlgorithm.desiredClusterCountBase">20</str>
> >>
> >> <!-- Location of Carrot2 lexical resources.
> >>
> >> A directory from which to load Carrot2-specific stop words
> >> and stop labels. Absolute or relative to Solr config
> directory.
> >> If a specific resource (e.g. stopwords.en) is present in the
> >> specified dir, it will completely override the corresponding
> >> default one that ships with Carrot2.
> >>
> >> For an overview of Carrot2 lexical resources, see:
> >>
> >> http://download.carrot2.org/head/manual/#chapter.lexical-resources
> >> -->
> >> <str name="carrot.lexicalResourcesDir">clustering/carrot2</str>
> >>
> >> <!-- The language to assume for the documents.
> >>
> >> For a list of allowed values, see:
> >>
> >>
> http://download.carrot2.org/stable/manual/#section.attribute.lingo.MultilingualClustering.defaultLanguage
> >> -->
> >> <str name="MultilingualClustering.defaultLanguage">ENGLISH</str>
> >> </lst>
> >> <lst name="engine">
> >> <str name="name">stc</str>
> >> <str
> >>
> name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm</str>
> >> </lst>
> >> </searchComponent>
> >>
> >> <!-- A request handler for demonstrating the clustering component
> >>
> >> This is purely as an example.
> >>
> >> In reality you will likely want to add the component to your
> >> already specified request handlers.
> >> -->
> >> <requestHandler name="/clustering"
> >> startup="lazy"
> >> enable="${solr.clustering.enabled:false}"
> >> class="solr.SearchHandler">
> >> <lst name="defaults">
> >> <bool name="clustering">true</bool>
> >> <str name="clustering.engine">default</str>
> >> <bool name="clustering.results">true</bool>
> >> <!-- The title field -->
> >> <str name="carrot.title">name</str>
> >> <str name="carrot.url">id</str>
> >> <!-- The field to cluster on -->
> >> <str name="carrot.snippet">features</str>
> >> <!-- produce summaries -->
> >> <bool name="carrot.produceSummary">true</bool>
> >> <!-- the maximum number of labels per cluster -->
> >> <!--<int name="carrot.numDescriptions">5</int>-->
> >> <!-- produce sub clusters -->
> >> <bool name="carrot.outputSubClusters">false</bool>
> >>
> >> <str name="defType">edismax</str>
> >> <str name="qf">
> >> text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1
> cat^1.4
> >> </str>
> >> <str name="q.alt">*:*</str>
> >> <str name="rows">10</str>
> >> <str name="fl">*,score</str>
> >> </lst>
> >> <arr name="last-components">
> >> <str>clustering</str>
> >> </arr>
> >> </requestHandler>
> >>
> >> <!-- Terms Component
> >>
> >> http://wiki.apache.org/solr/TermsComponent
> >>
> >> A component to return terms and document frequency of those
> >> terms
> >> -->
> >> <searchComponent name="terms" class="solr.TermsComponent"/>
> >>
> >> <!-- A request handler for demonstrating the terms component -->
> >> <requestHandler name="/terms" class="solr.SearchHandler"
> startup="lazy">
> >> <lst name="defaults">
> >> <bool name="terms">true</bool>
> >> </lst>
> >> <arr name="components">
> >> <str>terms</str>
> >> </arr>
> >> </requestHandler>
> >>
> >>
> >> <!-- Query Elevation Component
> >>
> >> http://wiki.apache.org/solr/QueryElevationComponent
> >>
> >> a search component that enables you to configure the top
> >> results for a given query regardless of the normal lucene
> >> scoring.
> >> -->
> >> <!--
> >> <searchComponent name="elevator" class="solr.QueryElevationComponent"
> >
> >> <str name="queryFieldType">string</str>
> >> <str name="config-file">elevate.xml</str>
> >> </searchComponent>
> >> -->
> >>
> >> <!-- A request handler for demonstrating the elevator component -->
> >> <requestHandler name="/elevate" class="solr.SearchHandler"
> >> startup="lazy">
> >> <lst name="defaults">
> >> <str name="echoParams">explicit</str>
> >> </lst>
> >> <arr name="last-components">
> >> <str>elevator</str>
> >> </arr>
> >> </requestHandler>
> >>
> >> <!-- Highlighting Component
> >>
> >> http://wiki.apache.org/solr/HighlightingParameters
> >> -->
> >> <searchComponent class="solr.HighlightComponent" name="highlight">
> >> <highlighting>
> >> <!-- Configure the standard fragmenter -->
> >> <!-- This could most likely be commented out in the "default" case
> >> -->
> >> <fragmenter name="gap"
> >> default="true"
> >> class="solr.highlight.GapFragmenter">
> >> <lst name="defaults">
> >> <int name="hl.fragsize">100</int>
> >> </lst>
> >> </fragmenter>
> >>
> >> <!-- A regular-expression-based fragmenter
> >> (for sentence extraction)
> >> -->
> >> <fragmenter name="regex"
> >> class="solr.highlight.RegexFragmenter">
> >> <lst name="defaults">
> >> <!-- slightly smaller fragsizes work better because of slop
> -->
> >> <int name="hl.fragsize">70</int>
> >> <!-- allow 50% slop on fragment sizes -->
> >> <float name="hl.regex.slop">0.5</float>
> >> <!-- a basic sentence pattern -->
> >> <str name="hl.regex.pattern">[-\w
> >> ,/\n\"']{20,200}</str>
> >> </lst>
> >> </fragmenter>
> >>
> >> <!-- Configure the standard formatter -->
> >> <formatter name="html"
> >> default="true"
> >> class="solr.highlight.HtmlFormatter">
> >> <lst name="defaults">
> >> <str name="hl.simple.pre"><![CDATA[<em>]]></str>
> >> <str name="hl.simple.post"><![CDATA[</em>]]></str>
> >> </lst>
> >> </formatter>
> >>
> >> <!-- Configure the standard encoder -->
> >> <encoder name="html"
> >> class="solr.highlight.HtmlEncoder" />
> >>
> >> <!-- Configure the standard fragListBuilder -->
> >> <fragListBuilder name="simple"
> >> default="true"
> >> class="solr.highlight.SimpleFragListBuilder"/>
> >>
> >> <!-- Configure the single fragListBuilder -->
> >> <fragListBuilder name="single"
> >> class="solr.highlight.SingleFragListBuilder"/>
> >>
> >> <!-- default tag FragmentsBuilder -->
> >> <fragmentsBuilder name="default"
> >> default="true"
> >>
> class="solr.highlight.ScoreOrderFragmentsBuilder">
> >> <!--
> >> <lst name="defaults">
> >> <str name="hl.multiValuedSeparatorChar">/</str>
> >> </lst>
> >> -->
> >> </fragmentsBuilder>
> >>
> >> <!-- multi-colored tag FragmentsBuilder -->
> >> <fragmentsBuilder name="colored"
> >>
> class="solr.highlight.ScoreOrderFragmentsBuilder">
> >> <lst name="defaults">
> >> <str name="hl.tag.pre"><![CDATA[
> >> <b style="background:yellow">,<b
> >> style="background:lawgreen">,
> >> <b style="background:aquamarine">,<b
> >> style="background:magenta">,
> >> <b style="background:palegreen">,<b
> >> style="background:coral">,
> >> <b style="background:wheat">,<b
> style="background:khaki">,
> >> <b style="background:lime">,<b
> >> style="background:deepskyblue">]]></str>
> >> <str name="hl.tag.post"><![CDATA[</b>]]></str>
> >> </lst>
> >> </fragmentsBuilder>
> >>
> >> <boundaryScanner name="default"
> >> default="true"
> >> class="solr.highlight.SimpleBoundaryScanner">
> >> <lst name="defaults">
> >> <str name="hl.bs.maxScan">10</str>
> >> <str name="hl.bs.chars">.,!? 	 </str>
> >> </lst>
> >> </boundaryScanner>
> >>
> >> <boundaryScanner name="breakIterator"
> >>
> class="solr.highlight.BreakIteratorBoundaryScanner">
> >> <lst name="defaults">
> >> <!-- type should be one of CHARACTER, WORD(default), LINE and
> >> SENTENCE -->
> >> <str name="hl.bs.type">WORD</str>
> >> <!-- language and country are used when constructing Locale
> >> object. -->
> >> <!-- And the Locale object will be used when getting instance
> of
> >> BreakIterator -->
> >> <str name="hl.bs.language">en</str>
> >> <str name="hl.bs.country">US</str>
> >> </lst>
> >> </boundaryScanner>
> >> </highlighting>
> >> </searchComponent>
> >>
> >> <!-- Update Processors
> >>
> >> Chains of Update Processor Factories for dealing with Update
> >> Requests can be declared, and then used by name in Update
> >> Request Processors
> >>
> >> http://wiki.apache.org/solr/UpdateRequestProcessor
> >>
> >> -->
> >> <!-- Deduplication
> >>
> >> An example dedup update processor that creates the "id" field
> >> on the fly based on the hash code of some other fields. This
> >> example has overwriteDupes set to false since we are using the
> >> id field as the signatureField and Solr will maintain
> >> uniqueness based on that anyway.
> >>
> >> -->
> >> <!--
> >> <updateRequestProcessorChain name="dedupe">
> >> <processor
> class="solr.processor.SignatureUpdateProcessorFactory">
> >> <bool name="enabled">true</bool>
> >> <str name="signatureField">id</str>
> >> <bool name="overwriteDupes">false</bool>
> >> <str name="fields">name,features,cat</str>
> >> <str
> name="signatureClass">solr.processor.Lookup3Signature</str>
> >> </processor>
> >> <processor class="solr.LogUpdateProcessorFactory" />
> >> <processor class="solr.RunUpdateProcessorFactory" />
> >> </updateRequestProcessorChain>
> >> -->
> >>
> >> <!--
> >> This example update chain identifies the language of the incoming
> >> documents using the langid contrib. The detected language is
> >> written to field language_s. No field name mapping is done.
> >> The fields used for detection are text, title, subject and
> >> description,
> >> making this example suitable for detecting languages form
> full-text
> >> rich documents injected via ExtractingRequestHandler.
> >> See more about langId at
> >> http://wiki.apache.org/solr/LanguageDetection
> >> -->
> >> <!--
> >> <updateRequestProcessorChain name="langid">
> >> <processor
> >>
> class="org.apache.solr.update.processor.TikaLanguageIdentifierUpdateProcessorFactory">
> >> <str name="langid.fl">text,title,subject,description</str>
> >> <str name="langid.langField">language_s</str>
> >> <str name="langid.fallback">en</str>
> >> </processor>
> >> <processor class="solr.LogUpdateProcessorFactory" />
> >> <processor class="solr.RunUpdateProcessorFactory" />
> >> </updateRequestProcessorChain>
> >> -->
> >>
> >> <!-- Response Writers
> >>
> >> http://wiki.apache.org/solr/QueryResponseWriter
> >>
> >> Request responses will be written using the writer specified by
> >> the 'wt' request parameter matching the name of a registered
> >> writer.
> >>
> >> The "default" writer is the default and will be used if 'wt' is
> >> not specified in the request.
> >> -->
> >> <!-- The following response writers are implicitly configured unless
> >> overridden...
> >> -->
> >> <!--
> >> <queryResponseWriter name="xml"
> >> default="true"
> >> class="solr.XMLResponseWriter" />
> >> <queryResponseWriter name="json" class="solr.JSONResponseWriter"/>
> >> <queryResponseWriter name="python"
> class="solr.PythonResponseWriter"/>
> >> <queryResponseWriter name="ruby" class="solr.RubyResponseWriter"/>
> >> <queryResponseWriter name="php" class="solr.PHPResponseWriter"/>
> >> <queryResponseWriter name="phps"
> >> class="solr.PHPSerializedResponseWriter"/>
> >> <queryResponseWriter name="csv" class="solr.CSVResponseWriter"/>
> >> -->
> >>
> >> <queryResponseWriter name="json" class="solr.JSONResponseWriter">
> >> <!-- For the purposes of the tutorial, JSON responses are written
> as
> >> plain text so that they are easy to read in *any* browser.
> >> If you expect a MIME type of "application/json" just remove this
> >> override.
> >> -->
> >> <str name="content-type">text/plain; charset=UTF-8</str>
> >> </queryResponseWriter>
> >>
> >> <!--
> >> Custom response writers can be declared as needed...
> >> -->
> >> <!-- The solr.velocity.enabled flag is used by Solr's test cases so
> >> that this response writer is not
> >> loaded (causing an error if contrib/velocity has not been built
> >> fully) -->
> >> <!--
> >> <queryResponseWriter name="velocity"
> >> class="solr.VelocityResponseWriter"
> enable="${solr.velocity.enabled:true}"/>
> >> -->
> >>
> >>
> >> <!-- XSLT response writer transforms the XML output by any xslt file
> >> found
> >> in Solr's conf/xslt directory. Changes to xslt files are checked
> >> for
> >> every xsltCacheLifetimeSeconds.
> >> -->
> >> <queryResponseWriter name="xslt" class="solr.XSLTResponseWriter">
> >> <int name="xsltCacheLifetimeSeconds">5</int>
> >> </queryResponseWriter>
> >>
> >> <!-- Query Parsers
> >>
> >> http://wiki.apache.org/solr/SolrQuerySyntax
> >>
> >> Multiple QParserPlugins can be registered by name, and then
> >> used in either the "defType" param for the QueryComponent (used
> >> by SearchHandler) or in LocalParams
> >> -->
> >> <!-- example of registering a query parser -->
> >> <!--
> >> <queryParser name="myparser"
> class="com.mycompany.MyQParserPlugin"/>
> >> -->
> >>
> >> <!-- Function Parsers
> >>
> >> http://wiki.apache.org/solr/FunctionQuery
> >>
> >> Multiple ValueSourceParsers can be registered by name, and then
> >> used as function names when using the "func" QParser.
> >> -->
> >> <!-- example of registering a custom function parser -->
> >> <!--
> >> <valueSourceParser name="myfunc"
> >> class="com.mycompany.MyValueSourceParser" />
> >> -->
> >>
> >> <!-- Legacy config for the admin interface -->
> >> <admin>
> >> <defaultQuery>*:*</defaultQuery>
> >>
> >> <!-- configure a healthcheck file for servers behind a
> >> loadbalancer
> >> -->
> >> <!--
> >> <healthcheck type="file">server-enabled</healthcheck>
> >> -->
> >> </admin>
> >>
> >> <requestHandler name="/replication" class="solr.ReplicationHandler" >
> >> <lst name="master">
> >> <!--Replicate on 'startup' and 'commit'. 'optimize' is also a
> >> valid value for replicateAfter. -->
> >> <str name="replicateAfter">startup</str>
> >> <str name="replicateAfter">commit</str>
> >>
> >> <!--Create a backup after 'optimize'. Other values can be
> >> 'commit', 'startup'. It is possible to have multiple entries of this
> config
> >> string. Note that this is just for backup, replication does not require
> >> this. -->
> >> <!-- <str name="backupAfter">optimize</str> -->
> >>
> >> <!--If configuration files need to be replicated give the names
> >> here, separated by comma -->
> >> <str name="confFiles">schema.xml,stopwords.txt,elevate.xml</str>
> >> <!--The default value of reservation is 10 secs.See the
> >> documentation below . Normally , you should not need to specify this -->
> >> <str name="commitReserveDuration">00:01:00</str>
> >> </lst>
> >> </requestHandler>
> >>
> >> </config>
> >>
> >>
> >>
> >>
> >>
> >> On Mon, Feb 13, 2012 at 12:36 PM, Li Li <fa...@gmail.com> wrote:
> >>
> >>> can you post your config file?
> >>> I found there are 2 places to config ramBufferSizeMB in latest svn of
> >>> 3.6's
> >>> example solrconfig.xml. trying to modify them both?
> >>>
> >>> <indexDefaults>
> >>>
> >>> <useCompoundFile>false</useCompoundFile>
> >>>
> >>> <mergeFactor>10</mergeFactor>
> >>> <!-- Sets the amount of RAM that may be used by Lucene indexing
> >>> for buffering added documents and deletions before they are
> >>> flushed to the Directory. -->
> >>> <ramBufferSizeMB>32</ramBufferSizeMB>
> >>> <!-- If both ramBufferSizeMB and maxBufferedDocs is set, then
> >>> Lucene will flush based on whichever limit is hit first.
> >>> -->
> >>> <!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
> >>>
> >>> <maxFieldLength>10000</maxFieldLength>
> >>> <writeLockTimeout>1000</writeLockTimeout>
> >>>
> >>> .............
> >>> <!-- <termIndexInterval>256</termIndexInterval> -->
> >>> </indexDefaults>
> >>>
> >>> <!-- Main Index
> >>>
> >>> Values here override the values in the <indexDefaults> section
> >>> for the main on disk index.
> >>> -->
> >>> <mainIndex>
> >>>
> >>> <useCompoundFile>false</useCompoundFile>
> >>> <ramBufferSizeMB>32</ramBufferSizeMB>
> >>> <mergeFactor>10</mergeFactor>
> >>> ........
> >>> </mainIndex>
> >>>
> >>> On Tue, Feb 14, 2012 at 1:10 AM, Huy Le <hu...@springpartners.com>
> wrote:
> >>>
> >>> > Hi,
> >>> >
> >>> > I am using solr 3.5. As I understood it, NRT is a solr 4 feature,
> but
> >>> solr
> >>> > 4 is not released yet.
> >>> >
> >>> > I understand commit after adding each document is expensive, but the
> >>> > application requires that documents be available after adding to the
> >>> index.
> >>> >
> >>> > What I don't understand is why new segment files are created so
> often.
> >>> > Are the commit calls triggering new segment files being created? I
> >>> don't
> >>> > see this behavior in another environment of the same version of solr.
> >>> >
> >>> > Huy
> >>> >
> >>> > On Mon, Feb 13, 2012 at 11:55 AM, Li Li <fa...@gmail.com> wrote:
> >>> >
> >>> > > Commit is called
> >>> > > after adding each document
> >>> > >
> >>> > >
> >>> > > you should add enough documents and then calling a commit. commit
> is
> >>> a
> >>> > > cost operation.
> >>> > > if you want to get latest feeded documents, you could use NRT
> >>> > >
> >>> > > On Tue, Feb 14, 2012 at 12:47 AM, Huy Le <huyle@springpartners.com
> >
> >>> > wrote:
> >>> > >
> >>> > > > Hi,
> >>> > > >
> >>> > > > I am using solr 3.5. I seeing solr keeps creating new segment
> files
> >>> > > (<1MB
> >>> > > > files) so often that it triggers segment merge about every one
> >>> minute.
> >>> > I
> >>> > > > search the news archive, but could not find any info on this
> issue.
> >>> I
> >>> > am
> >>> > > > indexing about 10 docs of less 2KB each every second. Commit is
> >>> called
> >>> > > > after adding each document. Relevant config params are:
> >>> > > >
> >>> > > > <mergeFactor>10</mergeFactor>
> >>> > > > <ramBufferSizeMB>1024</ramBufferSizeMB>
> >>> > > > <maxMergeDocs>2147483647</maxMergeDocs>
> >>> > > >
> >>> > > > What might be triggering this frequent new segment files
> creation?
> >>> > > Thanks!
> >>> > > >
> >>> > > > Huy
> >>> > > >
> >>> > > > --
> >>> > > > Huy Le
> >>> > > > Spring Partners, Inc.
> >>> > > > http://springpadit.com
> >>> > > >
> >>> > >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Huy Le
> >>> > Spring Partners, Inc.
> >>> > http://springpadit.com
> >>> >
> >>>
> >>
> >>
> >>
> >> --
> >> Huy Le
> >> Spring Partners, Inc.
> >> http://springpadit.com
> >>
> >
> >
> >
> > --
> > Huy Le
> > Spring Partners, Inc.
> > http://springpadit.com
>
--
Huy Le
Spring Partners, Inc.
http://springpadit.com
Re: New segment file created too often
Posted by Erick Erickson <er...@gmail.com>.
bq: I don't understand why new segment is created per
commit call
Because that's how it's designed from the start. Solr guarantees
that segments are write-once, that's how searchers get a
snapshot of the index, they open up all the segments when
they are opened. But to do this, it requires that the segments
do NOT change. So, when you start indexing data to the master,
a new segment is opened, and until you issue a commit, that
data is not visible for the searchers. Once you commit
(or once one of the autocommit parameters triggers a commit),
currently open segment files are closed and the searchers are
notified that the index has changed. Then they open all
closed segments and you get to search the new data.
Through the 3x code line, this behavior is just one of the basic
assumptions built into the way Solr works, commits are the
trigger for index changes to be made visible, and closing
segments is the mechanism for enforcing snapshots of the
index.
Solr 4.0 and the NRT and SolrCloud work will change this
somewhat with "soft commits", but that's another story.
Best
Erick
On Mon, Feb 13, 2012 at 2:24 PM, Huy Le <hu...@springpartners.com> wrote:
> Hi,
>
> This turned out to be a two-issue question. Â Issue one is that segment
> files get merged too frequent. Â Issue two is that new segment file is
> created on commit call.
>
> Looks like issue one is the behavior of tieredMergePolicy. Â This
> environment is newly setup, and the segment files are small. Â And
> mergeFactor is set at 10. Â Because the segment files are small, they are
> all of a similar file size, this trigger merge call by tier merge policy
> when there are 10 segments. Â So this is not an issue; it works as designed.
>
> Issue two, however, I don't understand why new segment is created per
> commit call? Â Does this have anything to do with replication setup I have
> (<str name="replicateAfter">commit</str>)?
>
> Huy
>
>
> On Mon, Feb 13, 2012 at 1:18 PM, Huy Le <hu...@springpartners.com> wrote:
>
>> I do have the indexDefault and mainIndex sections configured with the
>> desired values. Â As trying to get to the root cause of this issue, I have
>> the application disable commit calls, and segment files no longer that
>> created/merged so often. I then manually call commit (curl '
>> http://localhost:8983/solr/sbr01/update?commit=true'), segment files get
>> merged immediately. Â This suggest somethings to do with commits.
>>
>> Here is content of the config file:
>>
>>
>> <?xml version="1.0" encoding="UTF-8" ?>
>> <!--
>> Â Licensed to the Apache Software Foundation (ASF) under one or more
>> Â contributor license agreements. Â See the NOTICE file distributed with
>> Â this work for additional information regarding copyright ownership.
>> Â The ASF licenses this file to You under the Apache License, Version 2.0
>> Â (the "License"); you may not use this file except in compliance with
>> Â the License. Â You may obtain a copy of the License at
>>
>> Â Â Â http://www.apache.org/licenses/LICENSE-2.0
>>
>> Â Unless required by applicable law or agreed to in writing, software
>> Â distributed under the License is distributed on an "AS IS" BASIS,
>> Â WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> Â See the License for the specific language governing permissions and
>> Â limitations under the License.
>> -->
>>
>> <!--
>> Â Â Â For more details about configurations options that may appear in
>> Â Â Â this file, see http://wiki.apache.org/solr/SolrConfigXml.
>> -->
>> <config>
>> Â <!-- In all configuration below, a prefix of "solr." for class names
>> Â Â Â Â is an alias that causes solr to search appropriate packages,
>> Â Â Â Â including org.apache.solr.(search|update|request|core|analysis)
>>
>> Â Â Â Â You may also specify a fully qualified Java classname if you
>> Â Â Â Â have your own custom plugins.
>> Â Â -->
>>
>> Â <!-- Set this to 'false' if you want solr to continue working after
>> Â Â Â Â it has encountered an severe configuration error. Â In a
>> Â Â Â Â production environment, you may want solr to keep working even
>> Â Â Â Â if one handler is mis-configured.
>>
>> Â Â Â Â You may also set this to false using by setting the system
>> Â Â Â Â property:
>>
>> Â Â Â Â Â -Dsolr.abortOnConfigurationError=false
>> Â Â -->
>>
>> <abortOnConfigurationError>${solr.abortOnConfigurationError:true}</abortOnConfigurationError>
>>
>> Â <!-- Controls what version of Lucene various components of Solr
>> Â Â Â Â adhere to. Â Generally, you want to use the latest version to
>> Â Â Â Â get all bug fixes and improvements. It is highly recommended
>> Â Â Â Â that you fully re-index after changing this setting as it can
>> Â Â Â Â affect both how text is indexed and queried.
>> Â Â -->
>> Â <luceneMatchVersion>LUCENE_35</luceneMatchVersion>
>>
>> Â <!-- lib directives can be used to instruct Solr to load an Jars
>> Â Â Â Â identified and use them to resolve any "plugins" specified in
>> Â Â Â Â your solrconfig.xml or schema.xml (ie: Analyzers, Request
>> Â Â Â Â Handlers, etc...).
>>
>> Â Â Â Â All directories and paths are resolved relative to the
>> Â Â Â Â instanceDir.
>>
>> Â Â Â Â If a "./lib" directory exists in your instanceDir, all files
>> Â Â Â Â found in it are included as if you had used the following
>> Â Â Â Â syntax...
>>
>> Â Â Â Â Â Â Â <lib dir="./lib" />
>> Â Â -->
>>
>> Â <!-- A dir option by itself adds any files found in the directory to
>> Â Â Â Â the classpath, this is useful for including all jars in a
>> Â Â Â Â directory.
>> Â Â -->
>> Â <lib dir="../../contrib/extraction/lib" />
>> Â <lib dir="../../contrib/clustering/lib/" />
>> Â <lib dir="../../contrib/velocity/lib" />
>>
>> Â <!-- When a regex is specified in addition to a directory, only the
>> Â Â Â Â files in that directory which completely match the regex
>> Â Â Â Â (anchored on both ends) will be included.
>> Â Â -->
>> Â <lib dir="../../dist/" regex="apache-solr-cell-\d.*\.jar" />
>> Â <lib dir="../../dist/" regex="apache-solr-clustering-\d.*\.jar" />
>> Â <lib dir="../../dist/" regex="apache-solr-dataimporthandler-\d.*\.jar" />
>> Â <lib dir="../../dist/" regex="apache-solr-langid-\d.*\.jar" />
>> Â <lib dir="../../dist/" regex="apache-solr-velocity-\d.*\.jar" />
>>
>> Â <!-- If a dir option (with or without a regex) is used and nothing
>> Â Â Â Â is found that matches, it will be ignored
>> Â Â -->
>> Â <lib dir="../../contrib/clustering/lib/" />
>> Â <lib dir="/total/crap/dir/ignored" />
>>
>> Â <!-- an exact path can be used to specify a specific file. Â This
>> Â Â Â Â will cause a serious error to be logged if it can't be loaded.
>> Â Â -->
>> Â <!--
>> Â <lib path="../a-jar-that-does-not-exist.jar" />
>> Â -->
>>
>> Â <!-- Data Directory
>>
>> Â Â Â Â Used to specify an alternate directory to hold all index data
>> Â Â Â Â other than the default ./data under the Solr home. Â If
>> Â Â Â Â replication is in use, this should match the replication
>> Â Â Â Â configuration.
>> Â Â -->
>> Â <dataDir>${solr.data.dir:/u01/solr/sbr01/data}</dataDir>
>>
>>
>> Â <!-- The DirectoryFactory to use for indexes.
>>
>> Â Â Â Â solr.StandardDirectoryFactory, the default, is filesystem
>> Â Â Â Â based and tries to pick the best implementation for the current
>> Â Â Â Â JVM and platform. Â One can force a particular implementation
>> Â Â Â Â via solr.MMapDirectoryFactory, solr.NIOFSDirectoryFactory, or
>> Â Â Â Â solr.SimpleFSDirectoryFactory.
>>
>> Â Â Â Â solr.RAMDirectoryFactory is memory based, not
>> Â Â Â Â persistent, and doesn't work with replication.
>> Â Â -->
>> Â <directoryFactory name="DirectoryFactory"
>>
>> class="${solr.directoryFactory:solr.StandardDirectoryFactory}"/>
>>
>>
>> Â <!-- Index Defaults
>>
>> Â Â Â Â Values here affect all index writers and act as a default
>> Â Â Â Â unless overridden.
>>
>> Â Â Â Â WARNING: See also the <mainIndex> section below for parameters
>> Â Â Â Â that overfor Solr's main Lucene index.
>> Â Â -->
>> Â <indexDefaults>
>>
>> Â Â <useCompoundFile>true</useCompoundFile>
>>
>>
>> Â Â <mergeFactor>10</mergeFactor>
>> Â Â <!-- Sets the amount of RAM that may be used by Lucene indexing
>> Â Â Â Â Â for buffering added documents and deletions before they are
>> Â Â Â Â Â flushed to the Directory. Â -->
>> Â Â <ramBufferSizeMB>1024</ramBufferSizeMB>
>>
>> Â Â <!-- If both ramBufferSizeMB and maxBufferedDocs is set, then
>> Â Â Â Â Â Lucene will flush based on whichever limit is hit first.
>> Â Â Â -->
>> Â Â <!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
>> Â Â <!-- springpad configure -->
>> Â Â <maxMergeDocs>2147483647</maxMergeDocs>
>>
>> Â Â <!-- end springpad configure -->
>>
>>
>> Â Â <maxFieldLength>10000</maxFieldLength>
>> Â Â <writeLockTimeout>1000</writeLockTimeout>
>>
>> Â Â <!-- Expert: Merge Policy
>>
>> Â Â Â Â Â The Merge Policy in Lucene controls how merging is handled by
>> Â Â Â Â Â Lucene. Â The default in Solr 3.3 is TieredMergePolicy.
>>
>> Â Â Â Â Â The default in 2.3 was the LogByteSizeMergePolicy,
>> Â Â Â Â Â previous versions used LogDocMergePolicy.
>>
>> Â Â Â Â Â LogByteSizeMergePolicy chooses segments to merge based on
>> Â Â Â Â Â their size. Â The Lucene 2.2 default, LogDocMergePolicy chose
>> Â Â Â Â Â when to merge based on number of documents
>>
>> Â Â Â Â Â Other implementations of MergePolicy must have a no-argument
>> Â Â Â Â Â constructor
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <mergePolicy class="org.apache.lucene.index.TieredMergePolicy"/>
>> Â Â Â Â -->
>>
>> Â Â <!-- Expert: Merge Scheduler
>>
>> Â Â Â Â Â The Merge Scheduler in Lucene controls how merges are
>> Â Â Â Â Â performed. Â The ConcurrentMergeScheduler (Lucene 2.3 default)
>> Â Â Â Â Â can perform merges in the background using separate threads.
>> Â Â Â Â Â The SerialMergeScheduler (Lucene 2.2 default) does not.
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <mergeScheduler
>> class="org.apache.lucene.index.ConcurrentMergeScheduler"/>
>> Â Â Â Â -->
>>
>> Â Â <!-- LockFactory
>>
>> Â Â Â Â Â This option specifies which Lucene LockFactory implementation
>> Â Â Â Â Â to use.
>>
>> Â Â Â Â Â single = SingleInstanceLockFactory - suggested for a
>> Â Â Â Â Â Â Â Â Â read-only index or when there is no possibility of
>> Â Â Â Â Â Â Â Â Â another process trying to modify the index.
>> Â Â Â Â Â native = NativeFSLockFactory - uses OS native file locking.
>> Â Â Â Â Â Â Â Â Â Do not use when multiple solr webapps in the same
>> Â Â Â Â Â Â Â Â Â JVM are attempting to share a single index.
>>      simple = SimpleFSLockFactory  - uses a plain file for locking
>>
>> Â Â Â Â Â (For backwards compatibility with Solr 1.2, 'simple' is the
>> Â Â Â Â Â default if not specified.)
>>
>> Â Â Â Â Â More details on the nuances of each LockFactory...
>> Â Â Â Â Â http://wiki.apache.org/lucene-java/AvailableLockFactories
>> Â Â -->
>> Â Â <lockType>native</lockType>
>>
>> Â Â <!-- Expert: Controls how often Lucene loads terms into memory
>> Â Â Â Â Â Default is 128 and is likely good for most everyone.
>> Â Â Â -->
>>
>> Â Â <!-- <termIndexInterval>256</termIndexInterval> -->
>> Â </indexDefaults>
>>
>> Â <!-- Main Index
>>
>> Â Â Â Â Values here override the values in the <indexDefaults> section
>> Â Â Â Â for the main on disk index.
>> Â Â -->
>> Â <mainIndex>
>>
>> Â Â <maxMergeDocs>2147483647</maxMergeDocs>
>>
>> Â Â <useCompoundFile>true</useCompoundFile>
>> Â Â <ramBufferSizeMB>1024</ramBufferSizeMB>
>> Â Â <mergeFactor>30</mergeFactor>
>>
>> Â Â <!-- Unlock On Startup
>>
>> Â Â Â Â Â If true, unlock any held write or commit locks on startup.
>> Â Â Â Â Â This defeats the locking mechanism that allows multiple
>> Â Â Â Â Â processes to safely access a lucene index, and should be used
>> Â Â Â Â Â with care.
>>
>> Â Â Â Â Â This is not needed if lock type is 'none' or 'single'
>> Â Â Â -->
>> Â Â <unlockOnStartup>false</unlockOnStartup>
>>
>> Â Â <!-- If true, IndexReaders will be reopened (often more efficient)
>> Â Â Â Â Â instead of closed and then opened.
>> Â Â Â -->
>> Â Â <reopenReaders>true</reopenReaders>
>>
>> Â Â <!-- Commit Deletion Policy
>>
>> Â Â Â Â Â Custom deletion policies can specified here. The class must
>> Â Â Â Â Â implement org.apache.lucene.index.IndexDeletionPolicy.
>>
>>
>> http://lucene.apache.org/java/2_9_1/api/all/org/apache/lucene/index/IndexDeletionPolicy.html
>>
>> Â Â Â Â Â The standard Solr IndexDeletionPolicy implementation supports
>> Â Â Â Â Â deleting index commit points on number of commits, age of
>> Â Â Â Â Â commit point and optimized status.
>>
>> Â Â Â Â Â The latest commit point should always be preserved regardless
>> Â Â Â Â Â of the criteria.
>> Â Â -->
>> Â Â <deletionPolicy class="solr.SolrDeletionPolicy">
>> Â Â Â <!-- The number of commit points to be kept -->
>> Â Â Â <str name="maxCommitsToKeep">1</str>
>> Â Â Â <!-- The number of optimized commit points to be kept -->
>> Â Â Â <str name="maxOptimizedCommitsToKeep">0</str>
>> Â Â Â <!--
>> Â Â Â Â Â Delete all commit points once they have reached the given age.
>> Â Â Â Â Â Supports DateMathParser syntax e.g.
>> Â Â Â Â -->
>> Â Â Â <!--
>> Â Â Â Â Â <str name="maxCommitAge">30MINUTES</str>
>> Â Â Â Â Â <str name="maxCommitAge">1DAY</str>
>> Â Â Â -->
>> Â Â </deletionPolicy>
>>
>> Â Â <!-- Lucene Infostream
>>
>> Â Â Â Â Â To aid in advanced debugging, Lucene provides an "InfoStream"
>> Â Â Â Â Â of detailed information when indexing.
>>
>> Â Â Â Â Â Setting The value to true will instruct the underlying Lucene
>> Â Â Â Â Â IndexWriter to write its debugging info the specified file
>> Â Â Â -->
>> Â Â Â <infoStream file="INFOSTREAM.txt">false</infoStream>
>>
>> Â </mainIndex>
>>
>> Â <!-- JMX
>>
>> Â Â Â Â This example enables JMX if and only if an existing MBeanServer
>> Â Â Â Â is found, use this if you want to configure JMX through JVM
>> Â Â Â Â parameters. Remove this to disable exposing Solr configuration
>> Â Â Â Â and statistics to JMX.
>>
>> Â Â Â Â For more details see http://wiki.apache.org/solr/SolrJmx
>> Â Â -->
>> Â <jmx />
>> Â <!-- If you want to connect to a particular server, specify the
>> Â Â Â Â agentId
>> Â Â -->
>> Â <!-- <jmx agentId="myAgent" /> -->
>> Â <!-- If you want to start a new MBeanServer, specify the serviceUrl -->
>> Â <!-- <jmx
>> serviceUrl="service:jmx:rmi:///jndi/rmi://localhost:9999/solr"/>
>> Â Â -->
>>
>> Â <!-- The default high-performance update handler -->
>> Â <updateHandler class="solr.DirectUpdateHandler2">
>>
>> Â Â <!-- AutoCommit
>>
>> Â Â Â Â Â Perform a <commit/> automatically under certain conditions.
>> Â Â Â Â Â Instead of enabling autoCommit, consider using "commitWithin"
>> Â Â Â Â Â when adding documents.
>>
>> Â Â Â Â Â http://wiki.apache.org/solr/UpdateXmlMessages
>>
>> Â Â Â Â Â maxDocs - Maximum number of documents to add since the last
>> Â Â Â Â Â Â Â Â Â Â commit before automatically triggering a new commit.
>>
>> Â Â Â Â Â maxTime - Maximum amount of time that is allowed to pass
>> Â Â Â Â Â Â Â Â Â Â since a document was added before automaticly
>> Â Â Â Â Â Â Â Â Â Â triggering a new commit.
>> Â Â Â -->
>> Â Â Â Â <autoCommit>
>> Â Â Â Â Â <maxDocs>10000</maxDocs>
>> Â Â Â Â Â <maxTime>300000</maxTime>
>> Â Â Â Â </autoCommit>
>>
>> Â Â <!-- Update Related Event Listeners
>>
>> Â Â Â Â Â Various IndexWriter related events can trigger Listeners to
>> Â Â Â Â Â take actions.
>>
>> Â Â Â Â Â postCommit - fired after every commit or optimize command
>> Â Â Â Â Â postOptimize - fired after every optimize command
>> Â Â Â -->
>> Â Â <!-- The RunExecutableListener executes an external command from a
>> Â Â Â Â Â hook such as postCommit or postOptimize.
>>
>> Â Â Â Â Â exe - the name of the executable to run
>> Â Â Â Â Â dir - dir to use as the current working directory. (default=".")
>> Â Â Â Â Â wait - the calling thread waits until the executable returns.
>> Â Â Â Â Â Â Â Â (default="true")
>> Â Â Â Â Â args - the arguments to pass to the program. Â (default is none)
>> Â Â Â Â Â env - environment variables to set. Â (default is none)
>> Â Â Â -->
>> Â Â <!-- This example shows how RunExecutableListener could be used
>> Â Â Â Â Â with the script based replication...
>> Â Â Â Â Â http://wiki.apache.org/solr/CollectionDistribution
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <listener event="postCommit" class="solr.RunExecutableListener">
>> Â Â Â Â Â <str name="exe">solr/bin/snapshooter</str>
>> Â Â Â Â Â <str name="dir">.</str>
>> Â Â Â Â Â <bool name="wait">true</bool>
>> Â Â Â Â Â <arr name="args"> <str>arg1</str> <str>arg2</str> </arr>
>> Â Â Â Â Â <arr name="env"> <str>MYVAR=val1</str> </arr>
>> Â Â Â Â </listener>
>> Â Â Â -->
>> Â </updateHandler>
>>
>> Â <!-- IndexReaderFactory
>>
>> Â Â Â Â Use the following format to specify a custom IndexReaderFactory,
>> Â Â Â Â which allows for alternate IndexReader implementations.
>>
>> Â Â Â Â ** Experimental Feature **
>>
>> Â Â Â Â Please note - Using a custom IndexReaderFactory may prevent
>> Â Â Â Â certain other features from working. The API to
>> Â Â Â Â IndexReaderFactory may change without warning or may even be
>> Â Â Â Â removed from future releases if the problems cannot be
>> Â Â Â Â resolved.
>>
>>
>> Â Â Â Â ** Features that may not work with custom IndexReaderFactory **
>>
>> Â Â Â Â The ReplicationHandler assumes a disk-resident index. Using a
>> Â Â Â Â custom IndexReader implementation may cause incompatibility
>> Â Â Â Â with ReplicationHandler and may cause replication to not work
>> Â Â Â Â correctly. See SOLR-1366 for details.
>>
>> Â Â -->
>> Â <!--
>> Â <indexReaderFactory name="IndexReaderFactory" class="package.class">
>> Â Â <str name="someArg">Some Value</str>
>> Â </indexReaderFactory >
>> Â -->
>> Â <!-- By explicitly declaring the Factory, the termIndexDivisor can
>> Â Â Â Â be specified.
>> Â Â -->
>> Â <!--
>> Â Â Â <indexReaderFactory name="IndexReaderFactory"
>> Â Â Â Â Â Â Â Â Â Â Â Â Â class="solr.StandardIndexReaderFactory">
>> Â Â Â Â <int name="setTermIndexDivisor">12</int>
>> Â Â Â </indexReaderFactory >
>> Â Â -->
>>
>>
>> Â <query>
>> Â Â <!-- Max Boolean Clauses
>>
>> Â Â Â Â Â Maximum number of clauses in each BooleanQuery, Â an exception
>> Â Â Â Â Â is thrown if exceeded.
>>
>> Â Â Â Â Â ** WARNING **
>>
>> Â Â Â Â Â This option actually modifies a global Lucene property that
>> Â Â Â Â Â will affect all SolrCores. Â If multiple solrconfig.xml files
>> Â Â Â Â Â disagree on this property, the value at any given moment will
>> Â Â Â Â Â be based on the last SolrCore to be initialized.
>>
>> Â Â Â -->
>> Â Â <maxBooleanClauses>1024</maxBooleanClauses>
>>
>>
>> Â Â <!-- Solr Internal Query Caches
>>
>> Â Â Â Â Â There are two implementations of cache available for Solr,
>> Â Â Â Â Â LRUCache, based on a synchronized LinkedHashMap, and
>> Â Â Â Â Â FastLRUCache, based on a ConcurrentHashMap.
>>
>> Â Â Â Â Â FastLRUCache has faster gets and slower puts in single
>> Â Â Â Â Â threaded operation and thus is generally faster than LRUCache
>> Â Â Â Â Â when the hit ratio of the cache is high (> 75%), and may be
>> Â Â Â Â Â faster under other scenarios on multi-cpu systems.
>> Â Â -->
>>
>> Â Â <!-- Filter Cache
>>
>> Â Â Â Â Â Cache used by SolrIndexSearcher for filters (DocSets),
>> Â Â Â Â Â unordered sets of *all* documents that match a query. Â When a
>> Â Â Â Â Â new searcher is opened, its caches may be prepopulated or
>> Â Â Â Â Â "autowarmed" using data from caches in the old searcher.
>> Â Â Â Â Â autowarmCount is the number of items to prepopulate. Â For
>> Â Â Â Â Â LRUCache, the autowarmed items will be the most recently
>> Â Â Â Â Â accessed items.
>>
>> Â Â Â Â Â Parameters:
>> Â Â Â Â Â Â class - the SolrCache implementation LRUCache or
>> Â Â Â Â Â Â Â Â (LRUCache or FastLRUCache)
>> Â Â Â Â Â Â size - the maximum number of entries in the cache
>> Â Â Â Â Â Â initialSize - the initial capacity (number of entries) of
>> Â Â Â Â Â Â Â Â the cache. Â (see java.util.HashMap)
>> Â Â Â Â Â Â autowarmCount - the number of entries to prepopulate from
>> Â Â Â Â Â Â Â Â and old cache.
>> Â Â Â -->
>> Â Â <filterCache class="solr.FastLRUCache"
>> Â Â Â Â Â Â Â Â Â size="20000"
>> Â Â Â Â Â Â Â Â Â initialSize="1000"
>> Â Â Â Â Â Â Â Â Â autowarmCount="0"/>
>>
>> Â Â <!-- Query Result Cache
>>
>> Â Â Â Â Â Caches results of searches - ordered lists of document ids
>> Â Â Â Â Â (DocList) based on a query, a sort, and the range of documents
>> requested.
>> Â Â Â -->
>> Â Â <queryResultCache class="solr.LRUCache"
>> Â Â Â Â Â Â Â Â Â Â Â size="10000"
>> Â Â Â Â Â Â Â Â Â Â Â initialSize="1000"
>> Â Â Â Â Â Â Â Â Â Â Â autowarmCount="0"/>
>>
>> Â Â <!-- Document Cache
>>
>> Â Â Â Â Â Caches Lucene Document objects (the stored fields for each
>> Â Â Â Â Â document). Â Since Lucene internal document ids are transient,
>> Â Â Â Â Â this cache will not be autowarmed.
>> Â Â Â -->
>> Â Â <documentCache class="solr.LRUCache"
>> Â Â Â Â Â Â Â Â Â Â size="15000"
>> Â Â Â Â Â Â Â Â Â Â initialSize="1000"
>> Â Â Â Â Â Â Â Â Â Â autowarmCount="0"/>
>>
>> Â Â <!-- Field Value Cache
>>
>> Â Â Â Â Â Cache used to hold field values that are quickly accessible
>> Â Â Â Â Â by document id. Â The fieldValueCache is created by default
>> Â Â Â Â Â even if not configured here.
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <fieldValueCache class="solr.FastLRUCache"
>> Â Â Â Â Â Â Â Â Â Â Â Â size="512"
>> Â Â Â Â Â Â Â Â Â Â Â Â autowarmCount="128"
>> Â Â Â Â Â Â Â Â Â Â Â Â showItems="32" />
>> Â Â Â -->
>>
>> Â Â <!-- Custom Cache
>>
>> Â Â Â Â Â Example of a generic cache. Â These caches may be accessed by
>> Â Â Â Â Â name through SolrIndexSearcher.getCache(),cacheLookup(), and
>> Â Â Â Â Â cacheInsert(). Â The purpose is to enable easy caching of
>> Â Â Â Â Â user/application level data. Â The regenerator argument should
>> Â Â Â Â Â be specified as an implementation of solr.CacheRegenerator
>> Â Â Â Â Â if autowarming is desired.
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <cache name="myUserCache"
>> Â Â Â Â Â Â Â class="solr.LRUCache"
>> Â Â Â Â Â Â Â size="4096"
>> Â Â Â Â Â Â Â initialSize="1024"
>> Â Â Â Â Â Â Â autowarmCount="1024"
>> Â Â Â Â Â Â Â regenerator="com.mycompany.MyRegenerator"
>> Â Â Â Â Â Â Â />
>> Â Â Â -->
>>
>>
>> Â Â <!-- Lazy Field Loading
>>
>> Â Â Â Â Â If true, stored fields that are not requested will be loaded
>> Â Â Â Â Â lazily. Â This can result in a significant speed improvement
>> Â Â Â Â Â if the usual case is to not load all stored fields,
>> Â Â Â Â Â especially if the skipped fields are large compressed text
>> Â Â Â Â Â fields.
>> Â Â -->
>> Â Â <enableLazyFieldLoading>true</enableLazyFieldLoading>
>>
>> Â Â <!-- Use Filter For Sorted Query
>>
>> Â Â Â Â A possible optimization that attempts to use a filter to
>> Â Â Â Â satisfy a search. Â If the requested sort does not include
>> Â Â Â Â score, then the filterCache will be checked for a filter
>> Â Â Â Â matching the query. If found, the filter will be used as the
>> Â Â Â Â source of document ids, and then the sort will be applied to
>> Â Â Â Â that.
>>
>> Â Â Â Â For most situations, this will not be useful unless you
>> Â Â Â Â frequently get the same search repeatedly with different sort
>> Â Â Â Â options, and none of them ever use "score"
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â <useFilterForSortedQuery>true</useFilterForSortedQuery>
>> Â Â Â -->
>>
>> Â Â <!-- Result Window Size
>>
>> Â Â Â Â An optimization for use with the queryResultCache. Â When a search
>> Â Â Â Â is requested, a superset of the requested number of document ids
>> Â Â Â Â are collected. Â For example, if a search for a particular query
>> Â Â Â Â requests matching documents 10 through 19, and queryWindowSize is
>> 50,
>> Â Â Â Â then documents 0 through 49 will be collected and cached. Â Any
>> further
>> Â Â Â Â requests in that range can be satisfied via the cache.
>> Â Â Â -->
>> Â Â <queryResultWindowSize>50</queryResultWindowSize>
>>
>> Â Â <!-- Maximum number of documents to cache for any entry in the
>> Â Â Â Â queryResultCache.
>> Â Â Â -->
>> Â Â <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
>>
>> Â Â <!-- Query Related Event Listeners
>>
>> Â Â Â Â Various IndexSearcher related events can trigger Listeners to
>> Â Â Â Â take actions.
>>
>> Â Â Â Â newSearcher - fired whenever a new searcher is being prepared
>> Â Â Â Â and there is a current searcher handling requests (aka
>> Â Â Â Â registered). Â It can be used to prime certain caches to
>> Â Â Â Â prevent long request times for certain requests.
>>
>> Â Â Â Â firstSearcher - fired whenever a new searcher is being
>> Â Â Â Â prepared but there is no current registered searcher to handle
>> Â Â Â Â requests or to gain autowarming data from.
>>
>>
>> Â Â Â -->
>> Â Â <!-- QuerySenderListener takes an array of NamedList and executes a
>> Â Â Â Â Â local query request for each NamedList in sequence.
>> Â Â Â -->
>> Â Â <listener event="newSearcher" class="solr.QuerySenderListener">
>> Â Â Â <arr name="queries">
>> Â Â Â Â <!--
>> Â Â Â Â Â Â <lst><str name="q">solr</str><str name="sort">price
>> asc</str></lst>
>> Â Â Â Â Â Â <lst><str name="q">rocks</str><str name="sort">weight
>> asc</str></lst>
>> Â Â Â Â Â -->
>> Â Â Â </arr>
>> Â Â </listener>
>> Â Â <listener event="firstSearcher" class="solr.QuerySenderListener">
>> Â Â Â <arr name="queries">
>> Â Â Â Â <lst>
>> Â Â Â Â Â <str name="q">static firstSearcher warming in
>> solrconfig.xml</str>
>> Â Â Â Â </lst>
>> Â Â Â </arr>
>> Â Â </listener>
>>
>> Â Â <!-- Use Cold Searcher
>>
>> Â Â Â Â Â If a search request comes in and there is no current
>> Â Â Â Â Â registered searcher, then immediately register the still
>> Â Â Â Â Â warming searcher and use it. Â If "false" then all requests
>> Â Â Â Â Â will block until the first searcher is done warming.
>> Â Â Â -->
>> Â Â <useColdSearcher>true</useColdSearcher>
>>
>> Â Â <!-- Max Warming Searchers
>>
>> Â Â Â Â Â Maximum number of searchers that may be warming in the
>> Â Â Â Â Â background concurrently. Â An error is returned if this limit
>> Â Â Â Â Â is exceeded.
>>
>> Â Â Â Â Â Recommend values of 1-2 for read-only slaves, higher for
>> Â Â Â Â Â masters w/o cache warming.
>> Â Â Â -->
>> Â Â <maxWarmingSearchers>100</maxWarmingSearchers>
>>
>> Â </query>
>>
>>
>> Â <!-- Request Dispatcher
>>
>> Â Â Â Â This section contains instructions for how the SolrDispatchFilter
>> Â Â Â Â should behave when processing requests for this SolrCore.
>>
>> Â Â Â Â handleSelect affects the behavior of requests such as /select?qt=XXX
>>
>> Â Â Â Â handleSelect="true" will cause the SolrDispatchFilter to process
>> Â Â Â Â the request and will result in consistent error handling and
>> Â Â Â Â formatting for all types of requests.
>>
>> Â Â Â Â handleSelect="false" will cause the SolrDispatchFilter to
>> Â Â Â Â ignore "/select" requests and fallback to using the legacy
>> Â Â Â Â SolrServlet and it's Solr 1.1 style error formatting
>> Â Â -->
>> Â <requestDispatcher handleSelect="true" >
>> Â Â <!-- Request Parsing
>>
>> Â Â Â Â Â These settings indicate how Solr Requests may be parsed, and
>> Â Â Â Â Â what restrictions may be placed on the ContentStreams from
>> Â Â Â Â Â those requests
>>
>> Â Â Â Â Â enableRemoteStreaming - enables use of the stream.file
>> Â Â Â Â Â and stream.url parameters for specifying remote streams.
>>
>> Â Â Â Â Â multipartUploadLimitInKB - specifies the max size of
>> Â Â Â Â Â Multipart File Uploads that Solr will allow in a Request.
>>
>> Â Â Â Â Â *** WARNING ***
>> Â Â Â Â Â The settings below authorize Solr to fetch remote files, You
>> Â Â Â Â Â should make sure your system has some authentication before
>> Â Â Â Â Â using enableRemoteStreaming="true"
>>
>> Â Â Â -->
>> Â Â <requestParsers enableRemoteStreaming="true"
>> Â Â Â Â Â Â Â Â Â Â multipartUploadLimitInKB="2048000" />
>>
>> Â Â <!-- HTTP Caching
>>
>> Â Â Â Â Â Set HTTP caching related parameters (for proxy caches and
>> clients).
>>
>> Â Â Â Â Â The options below instruct Solr not to output any HTTP Caching
>> Â Â Â Â Â related headers
>> Â Â Â -->
>> Â Â <httpCaching never304="true" />
>> Â Â <!-- If you include a <cacheControl> directive, it will be used to
>> Â Â Â Â Â generate a Cache-Control header (as well as an Expires header
>> Â Â Â Â Â if the value contains "max-age=")
>>
>> Â Â Â Â Â By default, no Cache-Control header is generated.
>>
>> Â Â Â Â Â You can use the <cacheControl> option even if you have set
>> Â Â Â Â Â never304="true"
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <httpCaching never304="true" >
>> Â Â Â Â Â <cacheControl>max-age=30, public</cacheControl>
>> Â Â Â Â </httpCaching>
>> Â Â Â -->
>> Â Â <!-- To enable Solr to respond with automatically generated HTTP
>> Â Â Â Â Â Caching headers, and to response to Cache Validation requests
>> Â Â Â Â Â correctly, set the value of never304="false"
>>
>> Â Â Â Â Â This will cause Solr to generate Last-Modified and ETag
>> Â Â Â Â Â headers based on the properties of the Index.
>>
>> Â Â Â Â Â The following options can also be specified to affect the
>> Â Â Â Â Â values of these headers...
>>
>> Â Â Â Â Â lastModFrom - the default value is "openTime" which means the
>> Â Â Â Â Â Last-Modified value (and validation against If-Modified-Since
>> Â Â Â Â Â requests) will all be relative to when the current Searcher
>> Â Â Â Â Â was opened. Â You can change it to lastModFrom="dirLastMod" if
>> Â Â Â Â Â you want the value to exactly correspond to when the physical
>> Â Â Â Â Â index was last modified.
>>
>> Â Â Â Â Â etagSeed="..." is an option you can change to force the ETag
>> Â Â Â Â Â header (and validation against If-None-Match requests) to be
>> Â Â Â Â Â different even if the index has not changed (ie: when making
>> Â Â Â Â Â significant changes to your config file)
>>
>> Â Â Â Â Â (lastModifiedFrom and etagSeed are both ignored if you use
>> Â Â Â Â Â the never304="true" option)
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <httpCaching lastModifiedFrom="openTime"
>> Â Â Â Â Â Â Â Â Â Â etagSeed="Solr">
>> Â Â Â Â Â <cacheControl>max-age=30, public</cacheControl>
>> Â Â Â Â </httpCaching>
>> Â Â Â -->
>> Â </requestDispatcher>
>>
>> Â <!-- Request Handlers
>>
>> Â Â Â Â http://wiki.apache.org/solr/SolrRequestHandler
>>
>> Â Â Â Â incoming queries will be dispatched to the correct handler
>> Â Â Â Â based on the path or the qt (query type) param.
>>
>> Â Â Â Â Names starting with a '/' are accessed with the a path equal to
>> Â Â Â Â the registered name. Â Names without a leading '/' are accessed
>> Â Â Â Â with: http://host/app/[core/]select?qt=name
>>
>> Â Â Â Â If a /select request is processed with out a qt param
>> Â Â Â Â specified, the requestHandler that declares default="true" will
>> Â Â Â Â be used.
>>
>> Â Â Â Â If a Request Handler is declared with startup="lazy", then it will
>> Â Â Â Â not be initialized until the first request that uses it.
>>
>> Â Â -->
>> Â <!-- SearchHandler
>>
>> Â Â Â Â http://wiki.apache.org/solr/SearchHandler
>>
>> Â Â Â Â For processing Search Queries, the primary Request Handler
>> Â Â Â Â provided with Solr is "SearchHandler" It delegates to a sequent
>> Â Â Â Â of SearchComponents (see below) and supports distributed
>> Â Â Â Â queries across multiple shards
>> Â Â -->
>> Â <requestHandler name="search" class="solr.SearchHandler" default="true">
>> Â Â <!-- default values for query parameters can be specified, these
>> Â Â Â Â Â will be overridden by parameters in the request
>> Â Â Â -->
>> Â Â Â <lst name="defaults">
>> Â Â Â Â <str name="echoParams">explicit</str>
>> Â Â Â Â <int name="rows">10</int>
>> Â Â Â </lst>
>> Â Â <!-- In addition to defaults, "appends" params can be specified
>> Â Â Â Â Â to identify values which should be appended to the list of
>> Â Â Â Â Â multi-val params from the query (or the existing "defaults").
>> Â Â Â -->
>> Â Â <!-- In this example, the param "fq=instock:true" would be appended to
>> Â Â Â Â Â any query time fq params the user may specify, as a mechanism for
>> Â Â Â Â Â partitioning the index, independent of any user selected filtering
>> Â Â Â Â Â that may also be desired (perhaps as a result of faceted
>> searching).
>>
>> Â Â Â Â Â NOTE: there is *absolutely* nothing a client can do to prevent
>> these
>> Â Â Â Â Â "appends" values from being used, so don't use this mechanism
>> Â Â Â Â Â unless you are sure you always want it.
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <lst name="appends">
>> Â Â Â Â Â <str name="fq">inStock:true</str>
>> Â Â Â Â </lst>
>> Â Â Â -->
>> Â Â <!-- "invariants" are a way of letting the Solr maintainer lock down
>> Â Â Â Â Â the options available to Solr clients. Â Any params values
>> Â Â Â Â Â specified here are used regardless of what values may be specified
>> Â Â Â Â Â in either the query, the "defaults", or the "appends" params.
>>
>> Â Â Â Â Â In this example, the facet.field and facet.query params would
>> Â Â Â Â Â be fixed, limiting the facets clients can use. Â Faceting is
>> Â Â Â Â Â not turned on by default - but if the client does specify
>> Â Â Â Â Â facet=true in the request, these are the only facets they
>> Â Â Â Â Â will be able to see counts for; regardless of what other
>> Â Â Â Â Â facet.field or facet.query params they may specify.
>>
>> Â Â Â Â Â NOTE: there is *absolutely* nothing a client can do to prevent
>> these
>> Â Â Â Â Â "invariants" values from being used, so don't use this mechanism
>> Â Â Â Â Â unless you are sure you always want it.
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <lst name="invariants">
>> Â Â Â Â Â <str name="facet.field">cat</str>
>> Â Â Â Â Â <str name="facet.field">manu_exact</str>
>> Â Â Â Â Â <str name="facet.query">price:[* TO 500]</str>
>> Â Â Â Â Â <str name="facet.query">price:[500 TO *]</str>
>> Â Â Â Â </lst>
>> Â Â Â -->
>> Â Â <!-- If the default list of SearchComponents is not desired, that
>> Â Â Â Â Â list can either be overridden completely, or components can be
>> Â Â Â Â Â prepended or appended to the default list. Â (see below)
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <arr name="components">
>> Â Â Â Â Â <str>nameOfCustomComponent1</str>
>> Â Â Â Â Â <str>nameOfCustomComponent2</str>
>> Â Â Â Â </arr>
>> Â Â Â -->
>> Â Â </requestHandler>
>>
>> Â <!-- A Robust Example
>>
>> Â Â Â Â This example SearchHandler declaration shows off usage of the
>> Â Â Â Â SearchHandler with many defaults declared
>>
>> Â Â Â Â Note that multiple instances of the same Request Handler
>> Â Â Â Â (SearchHandler) can be registered multiple times with different
>> Â Â Â Â names (and different init parameters)
>> Â Â -->
>> Â <requestHandler name="/browse" class="solr.SearchHandler">
>> Â Â Â <lst name="defaults">
>> Â Â Â Â <str name="echoParams">explicit</str>
>>
>> Â Â Â Â <!-- VelocityResponseWriter settings -->
>> Â Â Â Â <!---
>> Â Â Â Â <str name="wt">velocity</str>
>> Â Â -->
>>
>> Â Â Â Â <str name="v.template">browse</str>
>> Â Â Â Â <str name="v.layout">layout</str>
>> Â Â Â Â <str name="title">Solritas</str>
>>
>> Â Â Â Â <str name="defType">edismax</str>
>> Â Â Â Â <str name="q.alt">*:*</str>
>> Â Â Â Â <str name="rows">10</str>
>> Â Â Â Â <str name="fl">*,score</str>
>> Â Â Â Â <str name="mlt.qf">
>> Â Â Â Â Â text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
>> Â Â Â Â </str>
>> Â Â Â Â <str name="mlt.fl">text,features,name,sku,id,manu,cat</str>
>> Â Â Â Â <int name="mlt.count">3</int>
>>
>> Â Â Â Â <str name="qf">
>> Â Â Â Â Â text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
>> Â Â Â Â </str>
>>
>> Â Â Â Â <str name="facet">on</str>
>> Â Â Â Â <str name="facet.field">cat</str>
>> Â Â Â Â <str name="facet.field">manu_exact</str>
>> Â Â Â Â <str name="facet.query">ipod</str>
>> Â Â Â Â <str name="facet.query">GB</str>
>> Â Â Â Â <str name="facet.mincount">1</str>
>> Â Â Â Â <str name="facet.pivot">cat,inStock</str>
>> Â Â Â Â <str name="facet.range.other">after</str>
>> Â Â Â Â <str name="facet.range">price</str>
>> Â Â Â Â <int name="f.price.facet.range.start">0</int>
>> Â Â Â Â <int name="f.price.facet.range.end">600</int>
>> Â Â Â Â <int name="f.price.facet.range.gap">50</int>
>> Â Â Â Â <str name="facet.range">popularity</str>
>> Â Â Â Â <int name="f.popularity.facet.range.start">0</int>
>> Â Â Â Â <int name="f.popularity.facet.range.end">10</int>
>> Â Â Â Â <int name="f.popularity.facet.range.gap">3</int>
>> Â Â Â Â <str name="facet.range">manufacturedate_dt</str>
>> Â Â Â Â <str
>> name="f.manufacturedate_dt.facet.range.start">NOW/YEAR-10YEARS</str>
>> Â Â Â Â <str name="f.manufacturedate_dt.facet.range.end">NOW</str>
>> Â Â Â Â <str name="f.manufacturedate_dt.facet.range.gap">+1YEAR</str>
>> Â Â Â Â <str name="f.manufacturedate_dt.facet.range.other">before</str>
>> Â Â Â Â <str name="f.manufacturedate_dt.facet.range.other">after</str>
>>
>>
>> Â Â Â Â <!-- Highlighting defaults -->
>> Â Â Â Â <str name="hl">on</str>
>> Â Â Â Â <str name="hl.fl">text features name</str>
>> Â Â Â Â <str name="f.name.hl.fragsize">0</str>
>> Â Â Â Â <str name="f.name.hl.alternateField">name</str>
>> Â Â Â </lst>
>> Â Â Â <arr name="last-components">
>> Â Â Â Â <str>spellcheck</str>
>> Â Â Â </arr>
>> Â Â Â <!--
>> Â Â Â <str name="url-scheme">httpx</str>
>> Â Â Â -->
>> Â </requestHandler>
>>
>> Â <!-- XML Update Request Handler.
>>
>> Â Â Â Â http://wiki.apache.org/solr/UpdateXmlMessages
>>
>> Â Â Â Â The canonical Request Handler for Modifying the Index through
>> Â Â Â Â commands specified using XML.
>>
>> Â Â Â Â Note: Since solr1.1 requestHandlers requires a valid content
>> Â Â Â Â type header if posted in the body. For example, curl now
>> Â Â Â Â requires: -H 'Content-type:text/xml; charset=utf-8'
>> Â Â -->
>> Â <requestHandler name="/update"
>> Â Â Â Â Â Â Â Â Â class="solr.XmlUpdateRequestHandler">
>> Â Â <!-- See below for information on defining
>> Â Â Â Â Â updateRequestProcessorChains that can be used by name
>> Â Â Â Â Â on each Update Request
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <lst name="defaults">
>> Â Â Â Â Â <str name="update.chain">dedupe</str>
>> Â Â Â Â </lst>
>> Â Â Â Â -->
>> Â Â </requestHandler>
>> Â <!-- Binary Update Request Handler
>> Â Â Â Â http://wiki.apache.org/solr/javabin
>> Â Â -->
>> Â <requestHandler name="/update/javabin"
>> Â Â Â Â Â Â Â Â Â class="solr.BinaryUpdateRequestHandler" />
>>
>> Â <!-- CSV Update Request Handler
>> Â Â Â Â http://wiki.apache.org/solr/UpdateCSV
>> Â Â -->
>> Â <requestHandler name="/update/csv"
>> Â Â Â Â Â Â Â Â Â class="solr.CSVRequestHandler"
>> Â Â Â Â Â Â Â Â Â startup="lazy" />
>>
>> Â <!-- JSON Update Request Handler
>> Â Â Â Â http://wiki.apache.org/solr/UpdateJSON
>> Â Â -->
>> Â <requestHandler name="/update/json"
>> Â Â Â Â Â Â Â Â Â class="solr.JsonUpdateRequestHandler"
>> Â Â Â Â Â Â Â Â Â startup="lazy" />
>>
>> Â <!-- Solr Cell Update Request Handler
>>
>> Â Â Â Â http://wiki.apache.org/solr/ExtractingRequestHandler
>>
>> Â Â -->
>> Â <requestHandler name="/update/extract"
>> Â Â Â Â Â Â Â Â Â startup="lazy"
>> Â Â Â Â Â Â Â Â Â class="solr.extraction.ExtractingRequestHandler" >
>> Â Â <lst name="defaults">
>> Â Â Â <!-- All the main content goes into "text"... if you need to return
>> Â Â Â Â Â Â the extracted text or do highlighting, use a stored field. -->
>> Â Â Â <str name="fmap.content">text</str>
>> Â Â Â <str name="lowernames">true</str>
>> Â Â Â <str name="uprefix">ignored_</str>
>>
>> Â Â Â <!-- capture link hrefs but ignore div attributes -->
>> Â Â Â <str name="captureAttr">true</str>
>> Â Â Â <str name="fmap.a">links</str>
>> Â Â Â <str name="fmap.div">ignored_</str>
>> Â Â </lst>
>> Â </requestHandler>
>>
>> Â <!-- XSLT Update Request Handler
>> Â Â Â Â Transforms incoming XML with stylesheet identified by tr=
>> Â -->
>> Â <requestHandler name="/update/xslt"
>> Â Â Â Â Â Â Â Â Â Â startup="lazy"
>> Â Â Â Â Â Â Â Â Â Â class="solr.XsltUpdateRequestHandler"/>
>>
>> Â <!-- Field Analysis Request Handler
>>
>> Â Â Â Â RequestHandler that provides much the same functionality as
>> Â Â Â Â analysis.jsp. Provides the ability to specify multiple field
>> Â Â Â Â types and field names in the same request and outputs
>> Â Â Â Â index-time and query-time analysis for each of them.
>>
>> Â Â Â Â Request parameters are:
>> Â Â Â Â analysis.fieldname - field name whose analyzers are to be used
>>
>> Â Â Â Â analysis.fieldtype - field type whose analyzers are to be used
>> Â Â Â Â analysis.fieldvalue - text for index-time analysis
>> Â Â Â Â q (or analysis.q) - text for query time analysis
>> Â Â Â Â analysis.showmatch (true|false) - When set to true and when
>> Â Â Â Â Â Â query analysis is performed, the produced tokens of the
>> Â Â Â Â Â Â field value analysis will be marked as "matched" for every
>> Â Â Â Â Â Â token that is produces by the query analysis
>> Â Â -->
>> Â <requestHandler name="/analysis/field"
>> Â Â Â Â Â Â Â Â Â startup="lazy"
>> Â Â Â Â Â Â Â Â Â class="solr.FieldAnalysisRequestHandler" />
>>
>>
>> Â <!-- Document Analysis Handler
>>
>> Â Â Â Â http://wiki.apache.org/solr/AnalysisRequestHandler
>>
>> Â Â Â Â An analysis handler that provides a breakdown of the analysis
>> Â Â Â Â process of provided docuemnts. This handler expects a (single)
>> Â Â Â Â content stream with the following format:
>>
>> Â Â Â Â <docs>
>> Â Â Â Â Â <doc>
>> Â Â Â Â Â Â <field name="id">1</field>
>> Â Â Â Â Â Â <field name="name">The Name</field>
>> Â Â Â Â Â Â <field name="text">The Text Value</field>
>> Â Â Â Â Â </doc>
>> Â Â Â Â Â <doc>...</doc>
>> Â Â Â Â Â <doc>...</doc>
>> Â Â Â Â Â ...
>> Â Â Â Â </docs>
>>
>> Â Â Note: Each document must contain a field which serves as the
>> Â Â unique key. This key is used in the returned response to associate
>> Â Â an analysis breakdown to the analyzed document.
>>
>> Â Â Like the FieldAnalysisRequestHandler, this handler also supports
>> Â Â query analysis by sending either an "analysis.query" or "q"
>> Â Â request parameter that holds the query text to be analyzed. It
>> Â Â also supports the "analysis.showmatch" parameter which when set to
>> Â Â true, all field tokens that match the query tokens will be marked
>> Â Â as a "match".
>> Â -->
>> Â <requestHandler name="/analysis/document"
>> Â Â Â Â Â Â Â Â Â class="solr.DocumentAnalysisRequestHandler"
>> Â Â Â Â Â Â Â Â Â startup="lazy" />
>>
>> Â <!-- Admin Handlers
>>
>> Â Â Â Â Admin Handlers - This will register all the standard admin
>> Â Â Â Â RequestHandlers.
>> Â Â -->
>> Â <requestHandler name="/admin/"
>> Â Â Â Â Â Â Â Â Â class="solr.admin.AdminHandlers" />
>> Â <!-- This single handler is equivalent to the following... -->
>> Â <!--
>> Â Â Â <requestHandler name="/admin/luke"
>> class="solr.admin.LukeRequestHandler" />
>> Â Â Â <requestHandler name="/admin/system"
>> class="solr.admin.SystemInfoHandler" />
>> Â Â Â <requestHandler name="/admin/plugins"
>> class="solr.admin.PluginInfoHandler" />
>> Â Â Â <requestHandler name="/admin/threads"
>> class="solr.admin.ThreadDumpHandler" />
>> Â Â Â <requestHandler name="/admin/properties"
>> class="solr.admin.PropertiesRequestHandler" />
>> Â Â Â <requestHandler name="/admin/file"
>> class="solr.admin.ShowFileRequestHandler" >
>> Â Â -->
>> Â <!-- If you wish to hide files under ${solr.home}/conf, explicitly
>> Â Â Â Â register the ShowFileRequestHandler using:
>> Â Â -->
>> Â <!--
>> Â Â Â <requestHandler name="/admin/file"
>> Â Â Â Â Â Â Â Â Â Â Â class="solr.admin.ShowFileRequestHandler" >
>> Â Â Â Â <lst name="invariants">
>> Â Â Â Â Â <str name="hidden">synonyms.txt</str>
>> Â Â Â Â Â <str name="hidden">anotherfile.txt</str>
>> Â Â Â Â </lst>
>> Â Â Â </requestHandler>
>> Â Â -->
>>
>> Â <!-- ping/healthcheck -->
>> Â <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
>> Â Â <lst name="invariants">
>> Â Â Â <str name="qt">search</str>
>> Â Â Â <str name="q">solrpingquery</str>
>> Â Â </lst>
>> Â Â <lst name="defaults">
>> Â Â Â <str name="echoParams">all</str>
>> Â Â </lst>
>> Â </requestHandler>
>>
>> Â <!-- Echo the request contents back to the client -->
>> Â <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
>> Â Â <lst name="defaults">
>> Â Â Â <str name="echoParams">explicit</str>
>> Â Â Â <str name="echoHandler">true</str>
>> Â Â </lst>
>> Â </requestHandler>
>>
>> Â <!-- Solr Replication
>>
>> Â Â Â Â The SolrReplicationHandler supports replicating indexes from a
>> Â Â Â Â "master" used for indexing and "salves" used for queries.
>>
>> Â Â Â Â http://wiki.apache.org/solr/SolrReplication
>>
>> Â Â Â Â In the example below, remove the <lst name="master"> section if
>>     this is just a slave and remove  the <lst name="slave"> section
>> Â Â Â Â if this is just a master.
>> Â Â -->
>> Â <!--
>> Â Â Â <requestHandler name="/replication" class="solr.ReplicationHandler" >
>> Â Â Â Â <lst name="master">
>> Â Â Â Â Â <str name="replicateAfter">commit</str>
>> Â Â Â Â Â <str name="replicateAfter">startup</str>
>> Â Â Â Â Â <str name="confFiles">schema.xml,stopwords.txt</str>
>> Â Â Â Â </lst>
>> Â Â Â Â <lst name="slave">
>> Â Â Â Â Â <str name="masterUrl">http://localhost:8983/solr/replication
>> </str>
>> Â Â Â Â Â <str name="pollInterval">00:00:60</str>
>> Â Â Â Â </lst>
>> Â Â Â </requestHandler>
>> Â Â -->
>>
>> Â <!-- Search Components
>>
>> Â Â Â Â Search components are registered to SolrCore and used by
>> Â Â Â Â instances of SearchHandler (which can access them by name)
>>
>> Â Â Â Â By default, the following components are available:
>>
>> Â Â Â Â <searchComponent name="query" Â Â class="solr.QueryComponent" />
>> Â Â Â Â <searchComponent name="facet" Â Â class="solr.FacetComponent" />
>> Â Â Â Â <searchComponent name="mlt"
>> class="solr.MoreLikeThisComponent" />
>> Â Â Â Â <searchComponent name="highlight" class="solr.HighlightComponent" />
>> Â Â Â Â <searchComponent name="stats" Â Â class="solr.StatsComponent" />
>> Â Â Â Â <searchComponent name="debug" Â Â class="solr.DebugComponent" />
>>
>> Â Â Â Â Default configuration in a requestHandler would look like:
>>
>> Â Â Â Â <arr name="components">
>> Â Â Â Â Â <str>query</str>
>> Â Â Â Â Â <str>facet</str>
>> Â Â Â Â Â <str>mlt</str>
>> Â Â Â Â Â <str>highlight</str>
>> Â Â Â Â Â <str>stats</str>
>> Â Â Â Â Â <str>debug</str>
>> Â Â Â Â </arr>
>>
>> Â Â Â Â If you register a searchComponent to one of the standard names,
>> Â Â Â Â that will be used instead of the default.
>>
>> Â Â Â Â To insert components before or after the 'standard' components, use:
>>
>> Â Â Â Â <arr name="first-components">
>> Â Â Â Â Â <str>myFirstComponentName</str>
>> Â Â Â Â </arr>
>>
>> Â Â Â Â <arr name="last-components">
>> Â Â Â Â Â <str>myLastComponentName</str>
>> Â Â Â Â </arr>
>>
>> Â Â Â Â NOTE: The component registered with the name "debug" will
>> Â Â Â Â always be executed after the "last-components"
>>
>> Â Â Â -->
>>
>> Â Â <!-- Spell Check
>>
>> Â Â Â Â The spell check component can return a list of alternative spelling
>> Â Â Â Â suggestions.
>>
>> Â Â Â Â http://wiki.apache.org/solr/SpellCheckComponent
>> Â Â Â -->
>> Â <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
>>
>> Â Â <str name="queryAnalyzerFieldType">textSpell</str>
>>
>> Â Â <!-- Multiple "Spell Checkers" can be declared and used by this
>> Â Â Â Â Â component
>> Â Â Â -->
>>
>> Â Â <!-- a spellchecker built from a field of the main index, and
>> Â Â Â Â Â written to disk
>> Â Â Â -->
>> Â Â <lst name="spellchecker">
>> Â Â Â <str name="name">default</str>
>> Â Â Â <str name="field">name</str>
>> Â Â Â <str name="spellcheckIndexDir">spellchecker</str>
>> Â Â Â <!-- uncomment this to require terms to occur in 1% of the documents
>> in order to be included in the dictionary
>> Â Â Â Â Â <float name="thresholdTokenFrequency">.01</float>
>> Â Â Â -->
>> Â Â </lst>
>>
>> Â Â <!-- a spellchecker that uses a different distance measure -->
>> Â Â <!--
>> Â Â Â Â <lst name="spellchecker">
>> Â Â Â Â Â <str name="name">jarowinkler</str>
>> Â Â Â Â Â <str name="field">spell</str>
>> Â Â Â Â Â <str name="distanceMeasure">
>> Â Â Â Â Â Â org.apache.lucene.search.spell.JaroWinklerDistance
>> Â Â Â Â Â </str>
>> Â Â Â Â Â <str name="spellcheckIndexDir">spellcheckerJaro</str>
>> Â Â Â Â </lst>
>> Â Â Â -->
>>
>> Â Â <!-- a spellchecker that use an alternate comparator
>>
>> Â Â Â Â Â comparatorClass be one of:
>> Â Â Â Â Â 1. score (default)
>> Â Â Â Â Â 2. freq (Frequency first, then score)
>> Â Â Â Â Â 3. A fully qualified class name
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <lst name="spellchecker">
>> Â Â Â Â Â <str name="name">freq</str>
>> Â Â Â Â Â <str name="field">lowerfilt</str>
>> Â Â Â Â Â <str name="spellcheckIndexDir">spellcheckerFreq</str>
>> Â Â Â Â Â <str name="comparatorClass">freq</str>
>> Â Â Â Â Â <str name="buildOnCommit">true</str>
>> Â Â Â -->
>>
>> Â Â <!-- A spellchecker that reads the list of words from a file -->
>> Â Â <!--
>> Â Â Â Â <lst name="spellchecker">
>> Â Â Â Â Â <str name="classname">solr.FileBasedSpellChecker</str>
>> Â Â Â Â Â <str name="name">file</str>
>> Â Â Â Â Â <str name="sourceLocation">spellings.txt</str>
>> Â Â Â Â Â <str name="characterEncoding">UTF-8</str>
>> Â Â Â Â Â <str name="spellcheckIndexDir">spellcheckerFile</str>
>> Â Â Â Â </lst>
>> Â Â Â -->
>> Â </searchComponent>
>>
>> Â <!-- A request handler for demonstrating the spellcheck component.
>>
>> Â Â Â Â NOTE: This is purely as an example. Â The whole purpose of the
>> Â Â Â Â SpellCheckComponent is to hook it into the request handler that
>> Â Â Â Â handles your normal user queries so that a separate request is
>> Â Â Â Â not needed to get suggestions.
>>
>> Â Â Â Â IN OTHER WORDS, THERE IS REALLY GOOD CHANCE THE SETUP BELOW IS
>> Â Â Â Â NOT WHAT YOU WANT FOR YOUR PRODUCTION SYSTEM!
>>
>> Â Â Â Â See http://wiki.apache.org/solr/SpellCheckComponent for details
>> Â Â Â Â on the request parameters.
>> Â Â -->
>> Â <requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
>> Â Â <lst name="defaults">
>> Â Â Â <str name="spellcheck.onlyMorePopular">false</str>
>> Â Â Â <str name="spellcheck.extendedResults">false</str>
>> Â Â Â <str name="spellcheck.count">1</str>
>> Â Â </lst>
>> Â Â <arr name="last-components">
>> Â Â Â <str>spellcheck</str>
>> Â Â </arr>
>> Â </requestHandler>
>>
>> Â <!-- Term Vector Component
>>
>> Â Â Â Â http://wiki.apache.org/solr/TermVectorComponent
>> Â Â -->
>> Â <searchComponent name="tvComponent" class="solr.TermVectorComponent"/>
>>
>> Â <!-- A request handler for demonstrating the term vector component
>>
>> Â Â Â Â This is purely as an example.
>>
>> Â Â Â Â In reality you will likely want to add the component to your
>> Â Â Â Â already specified request handlers.
>> Â Â -->
>> Â <requestHandler name="tvrh" class="solr.SearchHandler" startup="lazy">
>> Â Â <lst name="defaults">
>> Â Â Â <bool name="tv">true</bool>
>> Â Â </lst>
>> Â Â <arr name="last-components">
>> Â Â Â <str>tvComponent</str>
>> Â Â </arr>
>> Â </requestHandler>
>>
>> Â <!-- Clustering Component
>>
>> Â Â Â Â http://wiki.apache.org/solr/ClusteringComponent
>>
>> Â Â Â Â This relies on third party jars which are notincluded in the
>> Â Â Â Â release. Â To use this component (and the "/clustering" handler)
>> Â Â Â Â Those jars will need to be downloaded, and you'll need to set
>> Â Â Â Â the solr.cluster.enabled system property when running solr...
>>
>> Â Â Â Â Â java -Dsolr.clustering.enabled=true -jar start.jar
>> Â Â -->
>> Â <searchComponent name="clustering"
>> Â Â Â Â Â Â Â Â Â Â enable="${solr.clustering.enabled:false}"
>> Â Â Â Â Â Â Â Â Â Â class="solr.clustering.ClusteringComponent" >
>> Â Â <!-- Declare an engine -->
>> Â Â <lst name="engine">
>> Â Â Â <!-- The name, only one can be named "default" -->
>> Â Â Â <str name="name">default</str>
>>
>> Â Â Â <!-- Class name of Carrot2 clustering algorithm.
>>
>> Â Â Â Â Â Â Currently available algorithms are:
>>
>> Â Â Â Â Â Â * org.carrot2.clustering.lingo.LingoClusteringAlgorithm
>> Â Â Â Â Â Â * org.carrot2.clustering.stc.STCClusteringAlgorithm
>> Â Â Â Â Â Â *
>> org.carrot2.clustering.kmeans.BisectingKMeansClusteringAlgorithm
>>
>> Â Â Â Â Â Â See http://project.carrot2.org/algorithms.html for the
>> Â Â Â Â Â Â algorithm's characteristics.
>> Â Â Â Â -->
>> Â Â Â <str
>> name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm</str>
>>
>> Â Â Â <!-- Overriding values for Carrot2 default algorithm attributes.
>>
>> Â Â Â Â Â Â For a description of all available attributes, see:
>> Â Â Â Â Â Â http://download.carrot2.org/stable/manual/#chapter.components.
>> Â Â Â Â Â Â Use attribute key as name attribute of str elements
>> Â Â Â Â Â Â below. These can be further overridden for individual
>> Â Â Â Â Â Â requests by specifying attribute key as request parameter
>> Â Â Â Â Â Â name and attribute value as parameter value.
>> Â Â Â Â -->
>> Â Â Â <str name="LingoClusteringAlgorithm.desiredClusterCountBase">20</str>
>>
>> Â Â Â <!-- Location of Carrot2 lexical resources.
>>
>> Â Â Â Â Â Â A directory from which to load Carrot2-specific stop words
>> Â Â Â Â Â Â and stop labels. Absolute or relative to Solr config directory.
>> Â Â Â Â Â Â If a specific resource (e.g. stopwords.en) is present in the
>> Â Â Â Â Â Â specified dir, it will completely override the corresponding
>> Â Â Â Â Â Â default one that ships with Carrot2.
>>
>> Â Â Â Â Â Â For an overview of Carrot2 lexical resources, see:
>>
>> http://download.carrot2.org/head/manual/#chapter.lexical-resources
>> Â Â Â Â -->
>> Â Â Â <str name="carrot.lexicalResourcesDir">clustering/carrot2</str>
>>
>> Â Â Â <!-- The language to assume for the documents.
>>
>> Â Â Â Â Â Â For a list of allowed values, see:
>>
>> http://download.carrot2.org/stable/manual/#section.attribute.lingo.MultilingualClustering.defaultLanguage
>> Â Â Â Â -->
>> Â Â Â <str name="MultilingualClustering.defaultLanguage">ENGLISH</str>
>> Â Â </lst>
>> Â Â <lst name="engine">
>> Â Â Â <str name="name">stc</str>
>> Â Â Â <str
>> name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm</str>
>> Â Â </lst>
>> Â </searchComponent>
>>
>> Â <!-- A request handler for demonstrating the clustering component
>>
>> Â Â Â Â This is purely as an example.
>>
>> Â Â Â Â In reality you will likely want to add the component to your
>> Â Â Â Â already specified request handlers.
>> Â Â -->
>> Â <requestHandler name="/clustering"
>> Â Â Â Â Â Â Â Â Â startup="lazy"
>> Â Â Â Â Â Â Â Â Â enable="${solr.clustering.enabled:false}"
>> Â Â Â Â Â Â Â Â Â class="solr.SearchHandler">
>> Â Â <lst name="defaults">
>> Â Â Â <bool name="clustering">true</bool>
>> Â Â Â <str name="clustering.engine">default</str>
>> Â Â Â <bool name="clustering.results">true</bool>
>> Â Â Â <!-- The title field -->
>> Â Â Â <str name="carrot.title">name</str>
>> Â Â Â <str name="carrot.url">id</str>
>> Â Â Â <!-- The field to cluster on -->
>> Â Â Â Â <str name="carrot.snippet">features</str>
>> Â Â Â Â <!-- produce summaries -->
>> Â Â Â Â <bool name="carrot.produceSummary">true</bool>
>> Â Â Â Â <!-- the maximum number of labels per cluster -->
>> Â Â Â Â <!--<int name="carrot.numDescriptions">5</int>-->
>> Â Â Â Â <!-- produce sub clusters -->
>> Â Â Â Â <bool name="carrot.outputSubClusters">false</bool>
>>
>> Â Â Â Â <str name="defType">edismax</str>
>> Â Â Â Â <str name="qf">
>> Â Â Â Â Â text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
>> Â Â Â Â </str>
>> Â Â Â Â <str name="q.alt">*:*</str>
>> Â Â Â Â <str name="rows">10</str>
>> Â Â Â Â <str name="fl">*,score</str>
>> Â Â </lst>
>> Â Â <arr name="last-components">
>> Â Â Â <str>clustering</str>
>> Â Â </arr>
>> Â </requestHandler>
>>
>> Â <!-- Terms Component
>>
>> Â Â Â Â http://wiki.apache.org/solr/TermsComponent
>>
>> Â Â Â Â A component to return terms and document frequency of those
>> Â Â Â Â terms
>> Â Â -->
>> Â <searchComponent name="terms" class="solr.TermsComponent"/>
>>
>> Â <!-- A request handler for demonstrating the terms component -->
>> Â <requestHandler name="/terms" class="solr.SearchHandler" startup="lazy">
>> Â Â Â <lst name="defaults">
>> Â Â Â <bool name="terms">true</bool>
>> Â Â </lst>
>> Â Â <arr name="components">
>> Â Â Â <str>terms</str>
>> Â Â </arr>
>> Â </requestHandler>
>>
>>
>> Â <!-- Query Elevation Component
>>
>> Â Â Â Â http://wiki.apache.org/solr/QueryElevationComponent
>>
>> Â Â Â Â a search component that enables you to configure the top
>> Â Â Â Â results for a given query regardless of the normal lucene
>> Â Â Â Â scoring.
>> Â Â -->
>> Â <!--
>> Â <searchComponent name="elevator" class="solr.QueryElevationComponent" >
>> Â Â <str name="queryFieldType">string</str>
>> Â Â <str name="config-file">elevate.xml</str>
>> Â </searchComponent>
>> Â -->
>>
>> Â <!-- A request handler for demonstrating the elevator component -->
>> Â <requestHandler name="/elevate" class="solr.SearchHandler"
>> startup="lazy">
>> Â Â <lst name="defaults">
>> Â Â Â <str name="echoParams">explicit</str>
>> Â Â </lst>
>> Â Â <arr name="last-components">
>> Â Â Â <str>elevator</str>
>> Â Â </arr>
>> Â </requestHandler>
>>
>> Â <!-- Highlighting Component
>>
>> Â Â Â Â http://wiki.apache.org/solr/HighlightingParameters
>> Â Â -->
>> Â <searchComponent class="solr.HighlightComponent" name="highlight">
>> Â Â <highlighting>
>> Â Â Â <!-- Configure the standard fragmenter -->
>> Â Â Â <!-- This could most likely be commented out in the "default" case
>> -->
>> Â Â Â <fragmenter name="gap"
>> Â Â Â Â Â Â Â Â Â default="true"
>> Â Â Â Â Â Â Â Â Â class="solr.highlight.GapFragmenter">
>> Â Â Â Â <lst name="defaults">
>> Â Â Â Â Â <int name="hl.fragsize">100</int>
>> Â Â Â Â </lst>
>> Â Â Â </fragmenter>
>>
>> Â Â Â <!-- A regular-expression-based fragmenter
>> Â Â Â Â Â Â (for sentence extraction)
>> Â Â Â Â -->
>> Â Â Â <fragmenter name="regex"
>> Â Â Â Â Â Â Â Â Â class="solr.highlight.RegexFragmenter">
>> Â Â Â Â <lst name="defaults">
>> Â Â Â Â Â <!-- slightly smaller fragsizes work better because of slop -->
>> Â Â Â Â Â <int name="hl.fragsize">70</int>
>> Â Â Â Â Â <!-- allow 50% slop on fragment sizes -->
>> Â Â Â Â Â <float name="hl.regex.slop">0.5</float>
>> Â Â Â Â Â <!-- a basic sentence pattern -->
>> Â Â Â Â Â <str name="hl.regex.pattern">[-\w
>> ,/\n\"']{20,200}</str>
>> Â Â Â Â </lst>
>> Â Â Â </fragmenter>
>>
>> Â Â Â <!-- Configure the standard formatter -->
>> Â Â Â <formatter name="html"
>> Â Â Â Â Â Â Â Â Â default="true"
>> Â Â Â Â Â Â Â Â Â class="solr.highlight.HtmlFormatter">
>> Â Â Â Â <lst name="defaults">
>> Â Â Â Â Â <str name="hl.simple.pre"><![CDATA[<em>]]></str>
>> Â Â Â Â Â <str name="hl.simple.post"><![CDATA[</em>]]></str>
>> Â Â Â Â </lst>
>> Â Â Â </formatter>
>>
>> Â Â Â <!-- Configure the standard encoder -->
>> Â Â Â <encoder name="html"
>> Â Â Â Â Â Â Â Â class="solr.highlight.HtmlEncoder" />
>>
>> Â Â Â <!-- Configure the standard fragListBuilder -->
>> Â Â Â <fragListBuilder name="simple"
>> Â Â Â Â Â Â Â Â Â Â Â Â default="true"
>> Â Â Â Â Â Â Â Â Â Â Â Â class="solr.highlight.SimpleFragListBuilder"/>
>>
>> Â Â Â <!-- Configure the single fragListBuilder -->
>> Â Â Â <fragListBuilder name="single"
>> Â Â Â Â Â Â Â Â Â Â Â Â class="solr.highlight.SingleFragListBuilder"/>
>>
>> Â Â Â <!-- default tag FragmentsBuilder -->
>> Â Â Â <fragmentsBuilder name="default"
>> Â Â Â Â Â Â Â Â Â Â Â Â default="true"
>> Â Â Â Â Â Â Â Â Â Â Â Â class="solr.highlight.ScoreOrderFragmentsBuilder">
>> Â Â Â Â <!--
>> Â Â Â Â <lst name="defaults">
>> Â Â Â Â Â <str name="hl.multiValuedSeparatorChar">/</str>
>> Â Â Â Â </lst>
>> Â Â Â Â -->
>> Â Â Â </fragmentsBuilder>
>>
>> Â Â Â <!-- multi-colored tag FragmentsBuilder -->
>> Â Â Â <fragmentsBuilder name="colored"
>> Â Â Â Â Â Â Â Â Â Â Â Â class="solr.highlight.ScoreOrderFragmentsBuilder">
>> Â Â Â Â <lst name="defaults">
>> Â Â Â Â Â <str name="hl.tag.pre"><![CDATA[
>> Â Â Â Â Â Â Â Â <b style="background:yellow">,<b
>> style="background:lawgreen">,
>> Â Â Â Â Â Â Â Â <b style="background:aquamarine">,<b
>> style="background:magenta">,
>> Â Â Â Â Â Â Â Â <b style="background:palegreen">,<b
>> style="background:coral">,
>> Â Â Â Â Â Â Â Â <b style="background:wheat">,<b style="background:khaki">,
>> Â Â Â Â Â Â Â Â <b style="background:lime">,<b
>> style="background:deepskyblue">]]></str>
>> Â Â Â Â Â <str name="hl.tag.post"><![CDATA[</b>]]></str>
>> Â Â Â Â </lst>
>> Â Â Â </fragmentsBuilder>
>>
>> Â Â Â <boundaryScanner name="default"
>> Â Â Â Â Â Â Â Â Â Â Â Â default="true"
>> Â Â Â Â Â Â Â Â Â Â Â Â class="solr.highlight.SimpleBoundaryScanner">
>> Â Â Â Â <lst name="defaults">
>> Â Â Â Â Â <str name="hl.bs.maxScan">10</str>
>> Â Â Â Â Â <str name="hl.bs.chars">.,!? 	 </str>
>> Â Â Â Â </lst>
>> Â Â Â </boundaryScanner>
>>
>> Â Â Â <boundaryScanner name="breakIterator"
>> Â Â Â Â Â Â Â Â Â Â Â Â class="solr.highlight.BreakIteratorBoundaryScanner">
>> Â Â Â Â <lst name="defaults">
>> Â Â Â Â Â <!-- type should be one of CHARACTER, WORD(default), LINE and
>> SENTENCE -->
>> Â Â Â Â Â <str name="hl.bs.type">WORD</str>
>> Â Â Â Â Â <!-- language and country are used when constructing Locale
>> object. Â -->
>> Â Â Â Â Â <!-- And the Locale object will be used when getting instance of
>> BreakIterator -->
>> Â Â Â Â Â <str name="hl.bs.language">en</str>
>> Â Â Â Â Â <str name="hl.bs.country">US</str>
>> Â Â Â Â </lst>
>> Â Â Â </boundaryScanner>
>> Â Â </highlighting>
>> Â </searchComponent>
>>
>> Â <!-- Update Processors
>>
>> Â Â Â Â Chains of Update Processor Factories for dealing with Update
>> Â Â Â Â Requests can be declared, and then used by name in Update
>> Â Â Â Â Request Processors
>>
>> Â Â Â Â http://wiki.apache.org/solr/UpdateRequestProcessor
>>
>> Â Â -->
>> Â <!-- Deduplication
>>
>> Â Â Â Â An example dedup update processor that creates the "id" field
>> Â Â Â Â on the fly based on the hash code of some other fields. Â This
>> Â Â Â Â example has overwriteDupes set to false since we are using the
>> Â Â Â Â id field as the signatureField and Solr will maintain
>> Â Â Â Â uniqueness based on that anyway.
>>
>> Â Â -->
>> Â <!--
>> Â Â Â <updateRequestProcessorChain name="dedupe">
>> Â Â Â Â <processor class="solr.processor.SignatureUpdateProcessorFactory">
>> Â Â Â Â Â <bool name="enabled">true</bool>
>> Â Â Â Â Â <str name="signatureField">id</str>
>> Â Â Â Â Â <bool name="overwriteDupes">false</bool>
>> Â Â Â Â Â <str name="fields">name,features,cat</str>
>> Â Â Â Â Â <str name="signatureClass">solr.processor.Lookup3Signature</str>
>> Â Â Â Â </processor>
>> Â Â Â Â <processor class="solr.LogUpdateProcessorFactory" />
>> Â Â Â Â <processor class="solr.RunUpdateProcessorFactory" />
>> Â Â Â </updateRequestProcessorChain>
>> Â Â -->
>>
>> Â Â <!--
>> Â Â Â Â This example update chain identifies the language of the incoming
>> Â Â Â Â documents using the langid contrib. The detected language is
>> Â Â Â Â written to field language_s. No field name mapping is done.
>> Â Â Â Â The fields used for detection are text, title, subject and
>> description,
>> Â Â Â Â making this example suitable for detecting languages form full-text
>> Â Â Â Â rich documents injected via ExtractingRequestHandler.
>> Â Â Â Â See more about langId at
>> http://wiki.apache.org/solr/LanguageDetection
>> Â Â -->
>> Â Â <!--
>> Â Â Â <updateRequestProcessorChain name="langid">
>> Â Â Â Â <processor
>> class="org.apache.solr.update.processor.TikaLanguageIdentifierUpdateProcessorFactory">
>> Â Â Â Â Â <str name="langid.fl">text,title,subject,description</str>
>> Â Â Â Â Â <str name="langid.langField">language_s</str>
>> Â Â Â Â Â <str name="langid.fallback">en</str>
>> Â Â Â Â </processor>
>> Â Â Â Â <processor class="solr.LogUpdateProcessorFactory" />
>> Â Â Â Â <processor class="solr.RunUpdateProcessorFactory" />
>> Â Â Â </updateRequestProcessorChain>
>> Â Â -->
>>
>> Â <!-- Response Writers
>>
>> Â Â Â Â http://wiki.apache.org/solr/QueryResponseWriter
>>
>> Â Â Â Â Request responses will be written using the writer specified by
>> Â Â Â Â the 'wt' request parameter matching the name of a registered
>> Â Â Â Â writer.
>>
>> Â Â Â Â The "default" writer is the default and will be used if 'wt' is
>> Â Â Â Â not specified in the request.
>> Â Â -->
>> Â <!-- The following response writers are implicitly configured unless
>> Â Â Â Â overridden...
>> Â Â -->
>> Â <!--
>> Â Â Â <queryResponseWriter name="xml"
>> Â Â Â Â Â Â Â Â Â Â Â Â Â default="true"
>> Â Â Â Â Â Â Â Â Â Â Â Â Â class="solr.XMLResponseWriter" />
>> Â Â Â <queryResponseWriter name="json" class="solr.JSONResponseWriter"/>
>> Â Â Â <queryResponseWriter name="python" class="solr.PythonResponseWriter"/>
>> Â Â Â <queryResponseWriter name="ruby" class="solr.RubyResponseWriter"/>
>> Â Â Â <queryResponseWriter name="php" class="solr.PHPResponseWriter"/>
>> Â Â Â <queryResponseWriter name="phps"
>> class="solr.PHPSerializedResponseWriter"/>
>> Â Â Â <queryResponseWriter name="csv" class="solr.CSVResponseWriter"/>
>> Â Â -->
>>
>> Â <queryResponseWriter name="json" class="solr.JSONResponseWriter">
>> Â Â Â <!-- For the purposes of the tutorial, JSON responses are written as
>> Â Â Â plain text so that they are easy to read in *any* browser.
>> Â Â Â If you expect a MIME type of "application/json" just remove this
>> override.
>> Â Â Â -->
>> Â Â <str name="content-type">text/plain; charset=UTF-8</str>
>> Â </queryResponseWriter>
>>
>> Â <!--
>> Â Â Â Custom response writers can be declared as needed...
>> Â Â -->
>> Â Â <!-- The solr.velocity.enabled flag is used by Solr's test cases so
>> that this response writer is not
>> Â Â Â Â Â loaded (causing an error if contrib/velocity has not been built
>> fully) -->
>> Â Â <!--
>> Â Â <queryResponseWriter name="velocity"
>> class="solr.VelocityResponseWriter" enable="${solr.velocity.enabled:true}"/>
>> Â Â -->
>>
>>
>> Â <!-- XSLT response writer transforms the XML output by any xslt file
>> found
>> Â Â Â Â in Solr's conf/xslt directory. Â Changes to xslt files are checked
>> for
>> Â Â Â Â every xsltCacheLifetimeSeconds.
>> Â Â -->
>> Â <queryResponseWriter name="xslt" class="solr.XSLTResponseWriter">
>> Â Â <int name="xsltCacheLifetimeSeconds">5</int>
>> Â </queryResponseWriter>
>>
>> Â <!-- Query Parsers
>>
>> Â Â Â Â http://wiki.apache.org/solr/SolrQuerySyntax
>>
>> Â Â Â Â Multiple QParserPlugins can be registered by name, and then
>> Â Â Â Â used in either the "defType" param for the QueryComponent (used
>> Â Â Â Â by SearchHandler) or in LocalParams
>> Â Â -->
>> Â <!-- example of registering a query parser -->
>> Â <!--
>> Â Â Â <queryParser name="myparser" class="com.mycompany.MyQParserPlugin"/>
>> Â Â -->
>>
>> Â <!-- Function Parsers
>>
>> Â Â Â Â http://wiki.apache.org/solr/FunctionQuery
>>
>> Â Â Â Â Multiple ValueSourceParsers can be registered by name, and then
>> Â Â Â Â used as function names when using the "func" QParser.
>> Â Â -->
>>  <!-- example of registering a custom function parser  -->
>> Â <!--
>> Â Â Â <valueSourceParser name="myfunc"
>> Â Â Â Â Â Â Â Â Â Â Â Â class="com.mycompany.MyValueSourceParser" />
>> Â Â -->
>>
>> Â <!-- Legacy config for the admin interface -->
>> Â <admin>
>> Â Â <defaultQuery>*:*</defaultQuery>
>>
>> Â Â <!-- configure a healthcheck file for servers behind a
>> Â Â Â Â Â loadbalancer
>> Â Â Â -->
>> Â Â <!--
>> Â Â Â Â <healthcheck type="file">server-enabled</healthcheck>
>> Â Â Â -->
>> Â </admin>
>>
>> Â <requestHandler name="/replication" class="solr.ReplicationHandler" >
>> Â Â <lst name="master">
>> Â Â Â Â <!--Replicate on 'startup' and 'commit'. 'optimize' is also a
>> valid value for replicateAfter. -->
>> Â Â Â Â <str name="replicateAfter">startup</str>
>> Â Â Â Â <str name="replicateAfter">commit</str>
>>
>> Â Â Â Â <!--Create a backup after 'optimize'. Other values can be
>> 'commit', 'startup'. It is possible to have multiple entries of this config
>> string. Â Note that this is just for backup, replication does not require
>> this. -->
>> Â Â Â Â <!-- <str name="backupAfter">optimize</str> -->
>>
>> Â Â Â Â <!--If configuration files need to be replicated give the names
>> here, separated by comma -->
>> Â Â Â Â <str name="confFiles">schema.xml,stopwords.txt,elevate.xml</str>
>> Â Â Â Â <!--The default value of reservation is 10 secs.See the
>> documentation below . Normally , you should not need to specify this -->
>> Â Â Â Â <str name="commitReserveDuration">00:01:00</str>
>> Â Â </lst>
>> Â </requestHandler>
>>
>> </config>
>>
>>
>>
>>
>>
>> On Mon, Feb 13, 2012 at 12:36 PM, Li Li <fa...@gmail.com> wrote:
>>
>>> can you post your config file?
>>> I found there are 2 places to config ramBufferSizeMB in latest svn of
>>> 3.6's
>>> example solrconfig.xml. trying to modify them both?
>>>
>>> Â <indexDefaults>
>>>
>>> Â Â <useCompoundFile>false</useCompoundFile>
>>>
>>> Â Â <mergeFactor>10</mergeFactor>
>>> Â Â <!-- Sets the amount of RAM that may be used by Lucene indexing
>>> Â Â Â Â for buffering added documents and deletions before they are
>>> Â Â Â Â flushed to the Directory. Â -->
>>> Â Â <ramBufferSizeMB>32</ramBufferSizeMB>
>>> Â Â <!-- If both ramBufferSizeMB and maxBufferedDocs is set, then
>>> Â Â Â Â Lucene will flush based on whichever limit is hit first.
>>> Â Â Â -->
>>> Â Â <!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
>>>
>>> Â Â <maxFieldLength>10000</maxFieldLength>
>>> Â Â <writeLockTimeout>1000</writeLockTimeout>
>>>
>>> Â Â .............
>>> Â Â <!-- <termIndexInterval>256</termIndexInterval> -->
>>> Â </indexDefaults>
>>>
>>> Â <!-- Main Index
>>>
>>> Â Â Â Values here override the values in the <indexDefaults> section
>>> Â Â Â for the main on disk index.
>>> Â Â -->
>>> Â <mainIndex>
>>>
>>> Â Â <useCompoundFile>false</useCompoundFile>
>>> Â Â <ramBufferSizeMB>32</ramBufferSizeMB>
>>> Â Â <mergeFactor>10</mergeFactor>
>>> Â ........
>>> Â </mainIndex>
>>>
>>> On Tue, Feb 14, 2012 at 1:10 AM, Huy Le <hu...@springpartners.com> wrote:
>>>
>>> > Hi,
>>> >
>>> > I am using solr 3.5. Â As I understood it, NRT is a solr 4 feature, but
>>> solr
>>> > 4 is not released yet.
>>> >
>>> > I understand commit after adding each document is expensive, but the
>>> > application requires that documents be available after adding to the
>>> index.
>>> >
>>> > What I don't understand is why new segment files are created so often.
>>> > Are the commit calls triggering new segment files being created? Â I
>>> don't
>>> > see this behavior in another environment of the same version of solr.
>>> >
>>> > Huy
>>> >
>>> > On Mon, Feb 13, 2012 at 11:55 AM, Li Li <fa...@gmail.com> wrote:
>>> >
>>> > > Â Commit is called
>>> > > after adding each document
>>> > >
>>> > >
>>> > > Â you should add enough documents and then calling a commit. commit is
>>> a
>>> > > cost operation.
>>> > > Â if you want to get latest feeded documents, you could use NRT
>>> > >
>>> > > On Tue, Feb 14, 2012 at 12:47 AM, Huy Le <hu...@springpartners.com>
>>> > wrote:
>>> > >
>>> > > > Hi,
>>> > > >
>>> > > > I am using solr 3.5. Â I seeing solr keeps creating new segment files
>>> > > (<1MB
>>> > > > files) so often that it triggers segment merge about every one
>>> minute.
>>> > I
>>> > > > search the news archive, but could not find any info on this issue.
>>> Â I
>>> > am
>>> > > > indexing about 10 docs of less 2KB each every second. Â Commit is
>>> called
>>> > > > after adding each document. Relevant config params are:
>>> > > >
>>> > > > <mergeFactor>10</mergeFactor>
>>> > > > <ramBufferSizeMB>1024</ramBufferSizeMB>
>>> > > > <maxMergeDocs>2147483647</maxMergeDocs>
>>> > > >
>>> > > > What might be triggering this frequent new segment files creation?
>>> > > Â Thanks!
>>> > > >
>>> > > > Huy
>>> > > >
>>> > > > --
>>> > > > Huy Le
>>> > > > Spring Partners, Inc.
>>> > > > http://springpadit.com
>>> > > >
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > Huy Le
>>> > Spring Partners, Inc.
>>> > http://springpadit.com
>>> >
>>>
>>
>>
>>
>> --
>> Huy Le
>> Spring Partners, Inc.
>> http://springpadit.com
>>
>
>
>
> --
> Huy Le
> Spring Partners, Inc.
> http://springpadit.com
Re: New segment file created too often
Posted by Huy Le <hu...@springpartners.com>.
Hi,
This turned out to be a two-issue question. Issue one is that segment
files get merged too frequent. Issue two is that new segment file is
created on commit call.
Looks like issue one is the behavior of tieredMergePolicy. This
environment is newly setup, and the segment files are small. And
mergeFactor is set at 10. Because the segment files are small, they are
all of a similar file size, this trigger merge call by tier merge policy
when there are 10 segments. So this is not an issue; it works as designed.
Issue two, however, I don't understand why new segment is created per
commit call? Does this have anything to do with replication setup I have
(<str name="replicateAfter">commit</str>)?
Huy
On Mon, Feb 13, 2012 at 1:18 PM, Huy Le <hu...@springpartners.com> wrote:
> I do have the indexDefault and mainIndex sections configured with the
> desired values. As trying to get to the root cause of this issue, I have
> the application disable commit calls, and segment files no longer that
> created/merged so often. I then manually call commit (curl '
> http://localhost:8983/solr/sbr01/update?commit=true'), segment files get
> merged immediately. This suggest somethings to do with commits.
>
> Here is content of the config file:
>
>
> <?xml version="1.0" encoding="UTF-8" ?>
> <!--
> Licensed to the Apache Software Foundation (ASF) under one or more
> contributor license agreements. See the NOTICE file distributed with
> this work for additional information regarding copyright ownership.
> The ASF licenses this file to You under the Apache License, Version 2.0
> (the "License"); you may not use this file except in compliance with
> the License. You may obtain a copy of the License at
>
> http://www.apache.org/licenses/LICENSE-2.0
>
> Unless required by applicable law or agreed to in writing, software
> distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the License for the specific language governing permissions and
> limitations under the License.
> -->
>
> <!--
> For more details about configurations options that may appear in
> this file, see http://wiki.apache.org/solr/SolrConfigXml.
> -->
> <config>
> <!-- In all configuration below, a prefix of "solr." for class names
> is an alias that causes solr to search appropriate packages,
> including org.apache.solr.(search|update|request|core|analysis)
>
> You may also specify a fully qualified Java classname if you
> have your own custom plugins.
> -->
>
> <!-- Set this to 'false' if you want solr to continue working after
> it has encountered an severe configuration error. In a
> production environment, you may want solr to keep working even
> if one handler is mis-configured.
>
> You may also set this to false using by setting the system
> property:
>
> -Dsolr.abortOnConfigurationError=false
> -->
>
> <abortOnConfigurationError>${solr.abortOnConfigurationError:true}</abortOnConfigurationError>
>
> <!-- Controls what version of Lucene various components of Solr
> adhere to. Generally, you want to use the latest version to
> get all bug fixes and improvements. It is highly recommended
> that you fully re-index after changing this setting as it can
> affect both how text is indexed and queried.
> -->
> <luceneMatchVersion>LUCENE_35</luceneMatchVersion>
>
> <!-- lib directives can be used to instruct Solr to load an Jars
> identified and use them to resolve any "plugins" specified in
> your solrconfig.xml or schema.xml (ie: Analyzers, Request
> Handlers, etc...).
>
> All directories and paths are resolved relative to the
> instanceDir.
>
> If a "./lib" directory exists in your instanceDir, all files
> found in it are included as if you had used the following
> syntax...
>
> <lib dir="./lib" />
> -->
>
> <!-- A dir option by itself adds any files found in the directory to
> the classpath, this is useful for including all jars in a
> directory.
> -->
> <lib dir="../../contrib/extraction/lib" />
> <lib dir="../../contrib/clustering/lib/" />
> <lib dir="../../contrib/velocity/lib" />
>
> <!-- When a regex is specified in addition to a directory, only the
> files in that directory which completely match the regex
> (anchored on both ends) will be included.
> -->
> <lib dir="../../dist/" regex="apache-solr-cell-\d.*\.jar" />
> <lib dir="../../dist/" regex="apache-solr-clustering-\d.*\.jar" />
> <lib dir="../../dist/" regex="apache-solr-dataimporthandler-\d.*\.jar" />
> <lib dir="../../dist/" regex="apache-solr-langid-\d.*\.jar" />
> <lib dir="../../dist/" regex="apache-solr-velocity-\d.*\.jar" />
>
> <!-- If a dir option (with or without a regex) is used and nothing
> is found that matches, it will be ignored
> -->
> <lib dir="../../contrib/clustering/lib/" />
> <lib dir="/total/crap/dir/ignored" />
>
> <!-- an exact path can be used to specify a specific file. This
> will cause a serious error to be logged if it can't be loaded.
> -->
> <!--
> <lib path="../a-jar-that-does-not-exist.jar" />
> -->
>
> <!-- Data Directory
>
> Used to specify an alternate directory to hold all index data
> other than the default ./data under the Solr home. If
> replication is in use, this should match the replication
> configuration.
> -->
> <dataDir>${solr.data.dir:/u01/solr/sbr01/data}</dataDir>
>
>
> <!-- The DirectoryFactory to use for indexes.
>
> solr.StandardDirectoryFactory, the default, is filesystem
> based and tries to pick the best implementation for the current
> JVM and platform. One can force a particular implementation
> via solr.MMapDirectoryFactory, solr.NIOFSDirectoryFactory, or
> solr.SimpleFSDirectoryFactory.
>
> solr.RAMDirectoryFactory is memory based, not
> persistent, and doesn't work with replication.
> -->
> <directoryFactory name="DirectoryFactory"
>
> class="${solr.directoryFactory:solr.StandardDirectoryFactory}"/>
>
>
> <!-- Index Defaults
>
> Values here affect all index writers and act as a default
> unless overridden.
>
> WARNING: See also the <mainIndex> section below for parameters
> that overfor Solr's main Lucene index.
> -->
> <indexDefaults>
>
> <useCompoundFile>true</useCompoundFile>
>
>
> <mergeFactor>10</mergeFactor>
> <!-- Sets the amount of RAM that may be used by Lucene indexing
> for buffering added documents and deletions before they are
> flushed to the Directory. -->
> <ramBufferSizeMB>1024</ramBufferSizeMB>
>
> <!-- If both ramBufferSizeMB and maxBufferedDocs is set, then
> Lucene will flush based on whichever limit is hit first.
> -->
> <!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
> <!-- springpad configure -->
> <maxMergeDocs>2147483647</maxMergeDocs>
>
> <!-- end springpad configure -->
>
>
> <maxFieldLength>10000</maxFieldLength>
> <writeLockTimeout>1000</writeLockTimeout>
>
> <!-- Expert: Merge Policy
>
> The Merge Policy in Lucene controls how merging is handled by
> Lucene. The default in Solr 3.3 is TieredMergePolicy.
>
> The default in 2.3 was the LogByteSizeMergePolicy,
> previous versions used LogDocMergePolicy.
>
> LogByteSizeMergePolicy chooses segments to merge based on
> their size. The Lucene 2.2 default, LogDocMergePolicy chose
> when to merge based on number of documents
>
> Other implementations of MergePolicy must have a no-argument
> constructor
> -->
> <!--
> <mergePolicy class="org.apache.lucene.index.TieredMergePolicy"/>
> -->
>
> <!-- Expert: Merge Scheduler
>
> The Merge Scheduler in Lucene controls how merges are
> performed. The ConcurrentMergeScheduler (Lucene 2.3 default)
> can perform merges in the background using separate threads.
> The SerialMergeScheduler (Lucene 2.2 default) does not.
> -->
> <!--
> <mergeScheduler
> class="org.apache.lucene.index.ConcurrentMergeScheduler"/>
> -->
>
> <!-- LockFactory
>
> This option specifies which Lucene LockFactory implementation
> to use.
>
> single = SingleInstanceLockFactory - suggested for a
> read-only index or when there is no possibility of
> another process trying to modify the index.
> native = NativeFSLockFactory - uses OS native file locking.
> Do not use when multiple solr webapps in the same
> JVM are attempting to share a single index.
> simple = SimpleFSLockFactory - uses a plain file for locking
>
> (For backwards compatibility with Solr 1.2, 'simple' is the
> default if not specified.)
>
> More details on the nuances of each LockFactory...
> http://wiki.apache.org/lucene-java/AvailableLockFactories
> -->
> <lockType>native</lockType>
>
> <!-- Expert: Controls how often Lucene loads terms into memory
> Default is 128 and is likely good for most everyone.
> -->
>
> <!-- <termIndexInterval>256</termIndexInterval> -->
> </indexDefaults>
>
> <!-- Main Index
>
> Values here override the values in the <indexDefaults> section
> for the main on disk index.
> -->
> <mainIndex>
>
> <maxMergeDocs>2147483647</maxMergeDocs>
>
> <useCompoundFile>true</useCompoundFile>
> <ramBufferSizeMB>1024</ramBufferSizeMB>
> <mergeFactor>30</mergeFactor>
>
> <!-- Unlock On Startup
>
> If true, unlock any held write or commit locks on startup.
> This defeats the locking mechanism that allows multiple
> processes to safely access a lucene index, and should be used
> with care.
>
> This is not needed if lock type is 'none' or 'single'
> -->
> <unlockOnStartup>false</unlockOnStartup>
>
> <!-- If true, IndexReaders will be reopened (often more efficient)
> instead of closed and then opened.
> -->
> <reopenReaders>true</reopenReaders>
>
> <!-- Commit Deletion Policy
>
> Custom deletion policies can specified here. The class must
> implement org.apache.lucene.index.IndexDeletionPolicy.
>
>
> http://lucene.apache.org/java/2_9_1/api/all/org/apache/lucene/index/IndexDeletionPolicy.html
>
> The standard Solr IndexDeletionPolicy implementation supports
> deleting index commit points on number of commits, age of
> commit point and optimized status.
>
> The latest commit point should always be preserved regardless
> of the criteria.
> -->
> <deletionPolicy class="solr.SolrDeletionPolicy">
> <!-- The number of commit points to be kept -->
> <str name="maxCommitsToKeep">1</str>
> <!-- The number of optimized commit points to be kept -->
> <str name="maxOptimizedCommitsToKeep">0</str>
> <!--
> Delete all commit points once they have reached the given age.
> Supports DateMathParser syntax e.g.
> -->
> <!--
> <str name="maxCommitAge">30MINUTES</str>
> <str name="maxCommitAge">1DAY</str>
> -->
> </deletionPolicy>
>
> <!-- Lucene Infostream
>
> To aid in advanced debugging, Lucene provides an "InfoStream"
> of detailed information when indexing.
>
> Setting The value to true will instruct the underlying Lucene
> IndexWriter to write its debugging info the specified file
> -->
> <infoStream file="INFOSTREAM.txt">false</infoStream>
>
> </mainIndex>
>
> <!-- JMX
>
> This example enables JMX if and only if an existing MBeanServer
> is found, use this if you want to configure JMX through JVM
> parameters. Remove this to disable exposing Solr configuration
> and statistics to JMX.
>
> For more details see http://wiki.apache.org/solr/SolrJmx
> -->
> <jmx />
> <!-- If you want to connect to a particular server, specify the
> agentId
> -->
> <!-- <jmx agentId="myAgent" /> -->
> <!-- If you want to start a new MBeanServer, specify the serviceUrl -->
> <!-- <jmx
> serviceUrl="service:jmx:rmi:///jndi/rmi://localhost:9999/solr"/>
> -->
>
> <!-- The default high-performance update handler -->
> <updateHandler class="solr.DirectUpdateHandler2">
>
> <!-- AutoCommit
>
> Perform a <commit/> automatically under certain conditions.
> Instead of enabling autoCommit, consider using "commitWithin"
> when adding documents.
>
> http://wiki.apache.org/solr/UpdateXmlMessages
>
> maxDocs - Maximum number of documents to add since the last
> commit before automatically triggering a new commit.
>
> maxTime - Maximum amount of time that is allowed to pass
> since a document was added before automaticly
> triggering a new commit.
> -->
> <autoCommit>
> <maxDocs>10000</maxDocs>
> <maxTime>300000</maxTime>
> </autoCommit>
>
> <!-- Update Related Event Listeners
>
> Various IndexWriter related events can trigger Listeners to
> take actions.
>
> postCommit - fired after every commit or optimize command
> postOptimize - fired after every optimize command
> -->
> <!-- The RunExecutableListener executes an external command from a
> hook such as postCommit or postOptimize.
>
> exe - the name of the executable to run
> dir - dir to use as the current working directory. (default=".")
> wait - the calling thread waits until the executable returns.
> (default="true")
> args - the arguments to pass to the program. (default is none)
> env - environment variables to set. (default is none)
> -->
> <!-- This example shows how RunExecutableListener could be used
> with the script based replication...
> http://wiki.apache.org/solr/CollectionDistribution
> -->
> <!--
> <listener event="postCommit" class="solr.RunExecutableListener">
> <str name="exe">solr/bin/snapshooter</str>
> <str name="dir">.</str>
> <bool name="wait">true</bool>
> <arr name="args"> <str>arg1</str> <str>arg2</str> </arr>
> <arr name="env"> <str>MYVAR=val1</str> </arr>
> </listener>
> -->
> </updateHandler>
>
> <!-- IndexReaderFactory
>
> Use the following format to specify a custom IndexReaderFactory,
> which allows for alternate IndexReader implementations.
>
> ** Experimental Feature **
>
> Please note - Using a custom IndexReaderFactory may prevent
> certain other features from working. The API to
> IndexReaderFactory may change without warning or may even be
> removed from future releases if the problems cannot be
> resolved.
>
>
> ** Features that may not work with custom IndexReaderFactory **
>
> The ReplicationHandler assumes a disk-resident index. Using a
> custom IndexReader implementation may cause incompatibility
> with ReplicationHandler and may cause replication to not work
> correctly. See SOLR-1366 for details.
>
> -->
> <!--
> <indexReaderFactory name="IndexReaderFactory" class="package.class">
> <str name="someArg">Some Value</str>
> </indexReaderFactory >
> -->
> <!-- By explicitly declaring the Factory, the termIndexDivisor can
> be specified.
> -->
> <!--
> <indexReaderFactory name="IndexReaderFactory"
> class="solr.StandardIndexReaderFactory">
> <int name="setTermIndexDivisor">12</int>
> </indexReaderFactory >
> -->
>
>
> <query>
> <!-- Max Boolean Clauses
>
> Maximum number of clauses in each BooleanQuery, an exception
> is thrown if exceeded.
>
> ** WARNING **
>
> This option actually modifies a global Lucene property that
> will affect all SolrCores. If multiple solrconfig.xml files
> disagree on this property, the value at any given moment will
> be based on the last SolrCore to be initialized.
>
> -->
> <maxBooleanClauses>1024</maxBooleanClauses>
>
>
> <!-- Solr Internal Query Caches
>
> There are two implementations of cache available for Solr,
> LRUCache, based on a synchronized LinkedHashMap, and
> FastLRUCache, based on a ConcurrentHashMap.
>
> FastLRUCache has faster gets and slower puts in single
> threaded operation and thus is generally faster than LRUCache
> when the hit ratio of the cache is high (> 75%), and may be
> faster under other scenarios on multi-cpu systems.
> -->
>
> <!-- Filter Cache
>
> Cache used by SolrIndexSearcher for filters (DocSets),
> unordered sets of *all* documents that match a query. When a
> new searcher is opened, its caches may be prepopulated or
> "autowarmed" using data from caches in the old searcher.
> autowarmCount is the number of items to prepopulate. For
> LRUCache, the autowarmed items will be the most recently
> accessed items.
>
> Parameters:
> class - the SolrCache implementation LRUCache or
> (LRUCache or FastLRUCache)
> size - the maximum number of entries in the cache
> initialSize - the initial capacity (number of entries) of
> the cache. (see java.util.HashMap)
> autowarmCount - the number of entries to prepopulate from
> and old cache.
> -->
> <filterCache class="solr.FastLRUCache"
> size="20000"
> initialSize="1000"
> autowarmCount="0"/>
>
> <!-- Query Result Cache
>
> Caches results of searches - ordered lists of document ids
> (DocList) based on a query, a sort, and the range of documents
> requested.
> -->
> <queryResultCache class="solr.LRUCache"
> size="10000"
> initialSize="1000"
> autowarmCount="0"/>
>
> <!-- Document Cache
>
> Caches Lucene Document objects (the stored fields for each
> document). Since Lucene internal document ids are transient,
> this cache will not be autowarmed.
> -->
> <documentCache class="solr.LRUCache"
> size="15000"
> initialSize="1000"
> autowarmCount="0"/>
>
> <!-- Field Value Cache
>
> Cache used to hold field values that are quickly accessible
> by document id. The fieldValueCache is created by default
> even if not configured here.
> -->
> <!--
> <fieldValueCache class="solr.FastLRUCache"
> size="512"
> autowarmCount="128"
> showItems="32" />
> -->
>
> <!-- Custom Cache
>
> Example of a generic cache. These caches may be accessed by
> name through SolrIndexSearcher.getCache(),cacheLookup(), and
> cacheInsert(). The purpose is to enable easy caching of
> user/application level data. The regenerator argument should
> be specified as an implementation of solr.CacheRegenerator
> if autowarming is desired.
> -->
> <!--
> <cache name="myUserCache"
> class="solr.LRUCache"
> size="4096"
> initialSize="1024"
> autowarmCount="1024"
> regenerator="com.mycompany.MyRegenerator"
> />
> -->
>
>
> <!-- Lazy Field Loading
>
> If true, stored fields that are not requested will be loaded
> lazily. This can result in a significant speed improvement
> if the usual case is to not load all stored fields,
> especially if the skipped fields are large compressed text
> fields.
> -->
> <enableLazyFieldLoading>true</enableLazyFieldLoading>
>
> <!-- Use Filter For Sorted Query
>
> A possible optimization that attempts to use a filter to
> satisfy a search. If the requested sort does not include
> score, then the filterCache will be checked for a filter
> matching the query. If found, the filter will be used as the
> source of document ids, and then the sort will be applied to
> that.
>
> For most situations, this will not be useful unless you
> frequently get the same search repeatedly with different sort
> options, and none of them ever use "score"
> -->
> <!--
> <useFilterForSortedQuery>true</useFilterForSortedQuery>
> -->
>
> <!-- Result Window Size
>
> An optimization for use with the queryResultCache. When a search
> is requested, a superset of the requested number of document ids
> are collected. For example, if a search for a particular query
> requests matching documents 10 through 19, and queryWindowSize is
> 50,
> then documents 0 through 49 will be collected and cached. Any
> further
> requests in that range can be satisfied via the cache.
> -->
> <queryResultWindowSize>50</queryResultWindowSize>
>
> <!-- Maximum number of documents to cache for any entry in the
> queryResultCache.
> -->
> <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
>
> <!-- Query Related Event Listeners
>
> Various IndexSearcher related events can trigger Listeners to
> take actions.
>
> newSearcher - fired whenever a new searcher is being prepared
> and there is a current searcher handling requests (aka
> registered). It can be used to prime certain caches to
> prevent long request times for certain requests.
>
> firstSearcher - fired whenever a new searcher is being
> prepared but there is no current registered searcher to handle
> requests or to gain autowarming data from.
>
>
> -->
> <!-- QuerySenderListener takes an array of NamedList and executes a
> local query request for each NamedList in sequence.
> -->
> <listener event="newSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <!--
> <lst><str name="q">solr</str><str name="sort">price
> asc</str></lst>
> <lst><str name="q">rocks</str><str name="sort">weight
> asc</str></lst>
> -->
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst>
> <str name="q">static firstSearcher warming in
> solrconfig.xml</str>
> </lst>
> </arr>
> </listener>
>
> <!-- Use Cold Searcher
>
> If a search request comes in and there is no current
> registered searcher, then immediately register the still
> warming searcher and use it. If "false" then all requests
> will block until the first searcher is done warming.
> -->
> <useColdSearcher>true</useColdSearcher>
>
> <!-- Max Warming Searchers
>
> Maximum number of searchers that may be warming in the
> background concurrently. An error is returned if this limit
> is exceeded.
>
> Recommend values of 1-2 for read-only slaves, higher for
> masters w/o cache warming.
> -->
> <maxWarmingSearchers>100</maxWarmingSearchers>
>
> </query>
>
>
> <!-- Request Dispatcher
>
> This section contains instructions for how the SolrDispatchFilter
> should behave when processing requests for this SolrCore.
>
> handleSelect affects the behavior of requests such as /select?qt=XXX
>
> handleSelect="true" will cause the SolrDispatchFilter to process
> the request and will result in consistent error handling and
> formatting for all types of requests.
>
> handleSelect="false" will cause the SolrDispatchFilter to
> ignore "/select" requests and fallback to using the legacy
> SolrServlet and it's Solr 1.1 style error formatting
> -->
> <requestDispatcher handleSelect="true" >
> <!-- Request Parsing
>
> These settings indicate how Solr Requests may be parsed, and
> what restrictions may be placed on the ContentStreams from
> those requests
>
> enableRemoteStreaming - enables use of the stream.file
> and stream.url parameters for specifying remote streams.
>
> multipartUploadLimitInKB - specifies the max size of
> Multipart File Uploads that Solr will allow in a Request.
>
> *** WARNING ***
> The settings below authorize Solr to fetch remote files, You
> should make sure your system has some authentication before
> using enableRemoteStreaming="true"
>
> -->
> <requestParsers enableRemoteStreaming="true"
> multipartUploadLimitInKB="2048000" />
>
> <!-- HTTP Caching
>
> Set HTTP caching related parameters (for proxy caches and
> clients).
>
> The options below instruct Solr not to output any HTTP Caching
> related headers
> -->
> <httpCaching never304="true" />
> <!-- If you include a <cacheControl> directive, it will be used to
> generate a Cache-Control header (as well as an Expires header
> if the value contains "max-age=")
>
> By default, no Cache-Control header is generated.
>
> You can use the <cacheControl> option even if you have set
> never304="true"
> -->
> <!--
> <httpCaching never304="true" >
> <cacheControl>max-age=30, public</cacheControl>
> </httpCaching>
> -->
> <!-- To enable Solr to respond with automatically generated HTTP
> Caching headers, and to response to Cache Validation requests
> correctly, set the value of never304="false"
>
> This will cause Solr to generate Last-Modified and ETag
> headers based on the properties of the Index.
>
> The following options can also be specified to affect the
> values of these headers...
>
> lastModFrom - the default value is "openTime" which means the
> Last-Modified value (and validation against If-Modified-Since
> requests) will all be relative to when the current Searcher
> was opened. You can change it to lastModFrom="dirLastMod" if
> you want the value to exactly correspond to when the physical
> index was last modified.
>
> etagSeed="..." is an option you can change to force the ETag
> header (and validation against If-None-Match requests) to be
> different even if the index has not changed (ie: when making
> significant changes to your config file)
>
> (lastModifiedFrom and etagSeed are both ignored if you use
> the never304="true" option)
> -->
> <!--
> <httpCaching lastModifiedFrom="openTime"
> etagSeed="Solr">
> <cacheControl>max-age=30, public</cacheControl>
> </httpCaching>
> -->
> </requestDispatcher>
>
> <!-- Request Handlers
>
> http://wiki.apache.org/solr/SolrRequestHandler
>
> incoming queries will be dispatched to the correct handler
> based on the path or the qt (query type) param.
>
> Names starting with a '/' are accessed with the a path equal to
> the registered name. Names without a leading '/' are accessed
> with: http://host/app/[core/]select?qt=name
>
> If a /select request is processed with out a qt param
> specified, the requestHandler that declares default="true" will
> be used.
>
> If a Request Handler is declared with startup="lazy", then it will
> not be initialized until the first request that uses it.
>
> -->
> <!-- SearchHandler
>
> http://wiki.apache.org/solr/SearchHandler
>
> For processing Search Queries, the primary Request Handler
> provided with Solr is "SearchHandler" It delegates to a sequent
> of SearchComponents (see below) and supports distributed
> queries across multiple shards
> -->
> <requestHandler name="search" class="solr.SearchHandler" default="true">
> <!-- default values for query parameters can be specified, these
> will be overridden by parameters in the request
> -->
> <lst name="defaults">
> <str name="echoParams">explicit</str>
> <int name="rows">10</int>
> </lst>
> <!-- In addition to defaults, "appends" params can be specified
> to identify values which should be appended to the list of
> multi-val params from the query (or the existing "defaults").
> -->
> <!-- In this example, the param "fq=instock:true" would be appended to
> any query time fq params the user may specify, as a mechanism for
> partitioning the index, independent of any user selected filtering
> that may also be desired (perhaps as a result of faceted
> searching).
>
> NOTE: there is *absolutely* nothing a client can do to prevent
> these
> "appends" values from being used, so don't use this mechanism
> unless you are sure you always want it.
> -->
> <!--
> <lst name="appends">
> <str name="fq">inStock:true</str>
> </lst>
> -->
> <!-- "invariants" are a way of letting the Solr maintainer lock down
> the options available to Solr clients. Any params values
> specified here are used regardless of what values may be specified
> in either the query, the "defaults", or the "appends" params.
>
> In this example, the facet.field and facet.query params would
> be fixed, limiting the facets clients can use. Faceting is
> not turned on by default - but if the client does specify
> facet=true in the request, these are the only facets they
> will be able to see counts for; regardless of what other
> facet.field or facet.query params they may specify.
>
> NOTE: there is *absolutely* nothing a client can do to prevent
> these
> "invariants" values from being used, so don't use this mechanism
> unless you are sure you always want it.
> -->
> <!--
> <lst name="invariants">
> <str name="facet.field">cat</str>
> <str name="facet.field">manu_exact</str>
> <str name="facet.query">price:[* TO 500]</str>
> <str name="facet.query">price:[500 TO *]</str>
> </lst>
> -->
> <!-- If the default list of SearchComponents is not desired, that
> list can either be overridden completely, or components can be
> prepended or appended to the default list. (see below)
> -->
> <!--
> <arr name="components">
> <str>nameOfCustomComponent1</str>
> <str>nameOfCustomComponent2</str>
> </arr>
> -->
> </requestHandler>
>
> <!-- A Robust Example
>
> This example SearchHandler declaration shows off usage of the
> SearchHandler with many defaults declared
>
> Note that multiple instances of the same Request Handler
> (SearchHandler) can be registered multiple times with different
> names (and different init parameters)
> -->
> <requestHandler name="/browse" class="solr.SearchHandler">
> <lst name="defaults">
> <str name="echoParams">explicit</str>
>
> <!-- VelocityResponseWriter settings -->
> <!---
> <str name="wt">velocity</str>
> -->
>
> <str name="v.template">browse</str>
> <str name="v.layout">layout</str>
> <str name="title">Solritas</str>
>
> <str name="defType">edismax</str>
> <str name="q.alt">*:*</str>
> <str name="rows">10</str>
> <str name="fl">*,score</str>
> <str name="mlt.qf">
> text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
> </str>
> <str name="mlt.fl">text,features,name,sku,id,manu,cat</str>
> <int name="mlt.count">3</int>
>
> <str name="qf">
> text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
> </str>
>
> <str name="facet">on</str>
> <str name="facet.field">cat</str>
> <str name="facet.field">manu_exact</str>
> <str name="facet.query">ipod</str>
> <str name="facet.query">GB</str>
> <str name="facet.mincount">1</str>
> <str name="facet.pivot">cat,inStock</str>
> <str name="facet.range.other">after</str>
> <str name="facet.range">price</str>
> <int name="f.price.facet.range.start">0</int>
> <int name="f.price.facet.range.end">600</int>
> <int name="f.price.facet.range.gap">50</int>
> <str name="facet.range">popularity</str>
> <int name="f.popularity.facet.range.start">0</int>
> <int name="f.popularity.facet.range.end">10</int>
> <int name="f.popularity.facet.range.gap">3</int>
> <str name="facet.range">manufacturedate_dt</str>
> <str
> name="f.manufacturedate_dt.facet.range.start">NOW/YEAR-10YEARS</str>
> <str name="f.manufacturedate_dt.facet.range.end">NOW</str>
> <str name="f.manufacturedate_dt.facet.range.gap">+1YEAR</str>
> <str name="f.manufacturedate_dt.facet.range.other">before</str>
> <str name="f.manufacturedate_dt.facet.range.other">after</str>
>
>
> <!-- Highlighting defaults -->
> <str name="hl">on</str>
> <str name="hl.fl">text features name</str>
> <str name="f.name.hl.fragsize">0</str>
> <str name="f.name.hl.alternateField">name</str>
> </lst>
> <arr name="last-components">
> <str>spellcheck</str>
> </arr>
> <!--
> <str name="url-scheme">httpx</str>
> -->
> </requestHandler>
>
> <!-- XML Update Request Handler.
>
> http://wiki.apache.org/solr/UpdateXmlMessages
>
> The canonical Request Handler for Modifying the Index through
> commands specified using XML.
>
> Note: Since solr1.1 requestHandlers requires a valid content
> type header if posted in the body. For example, curl now
> requires: -H 'Content-type:text/xml; charset=utf-8'
> -->
> <requestHandler name="/update"
> class="solr.XmlUpdateRequestHandler">
> <!-- See below for information on defining
> updateRequestProcessorChains that can be used by name
> on each Update Request
> -->
> <!--
> <lst name="defaults">
> <str name="update.chain">dedupe</str>
> </lst>
> -->
> </requestHandler>
> <!-- Binary Update Request Handler
> http://wiki.apache.org/solr/javabin
> -->
> <requestHandler name="/update/javabin"
> class="solr.BinaryUpdateRequestHandler" />
>
> <!-- CSV Update Request Handler
> http://wiki.apache.org/solr/UpdateCSV
> -->
> <requestHandler name="/update/csv"
> class="solr.CSVRequestHandler"
> startup="lazy" />
>
> <!-- JSON Update Request Handler
> http://wiki.apache.org/solr/UpdateJSON
> -->
> <requestHandler name="/update/json"
> class="solr.JsonUpdateRequestHandler"
> startup="lazy" />
>
> <!-- Solr Cell Update Request Handler
>
> http://wiki.apache.org/solr/ExtractingRequestHandler
>
> -->
> <requestHandler name="/update/extract"
> startup="lazy"
> class="solr.extraction.ExtractingRequestHandler" >
> <lst name="defaults">
> <!-- All the main content goes into "text"... if you need to return
> the extracted text or do highlighting, use a stored field. -->
> <str name="fmap.content">text</str>
> <str name="lowernames">true</str>
> <str name="uprefix">ignored_</str>
>
> <!-- capture link hrefs but ignore div attributes -->
> <str name="captureAttr">true</str>
> <str name="fmap.a">links</str>
> <str name="fmap.div">ignored_</str>
> </lst>
> </requestHandler>
>
> <!-- XSLT Update Request Handler
> Transforms incoming XML with stylesheet identified by tr=
> -->
> <requestHandler name="/update/xslt"
> startup="lazy"
> class="solr.XsltUpdateRequestHandler"/>
>
> <!-- Field Analysis Request Handler
>
> RequestHandler that provides much the same functionality as
> analysis.jsp. Provides the ability to specify multiple field
> types and field names in the same request and outputs
> index-time and query-time analysis for each of them.
>
> Request parameters are:
> analysis.fieldname - field name whose analyzers are to be used
>
> analysis.fieldtype - field type whose analyzers are to be used
> analysis.fieldvalue - text for index-time analysis
> q (or analysis.q) - text for query time analysis
> analysis.showmatch (true|false) - When set to true and when
> query analysis is performed, the produced tokens of the
> field value analysis will be marked as "matched" for every
> token that is produces by the query analysis
> -->
> <requestHandler name="/analysis/field"
> startup="lazy"
> class="solr.FieldAnalysisRequestHandler" />
>
>
> <!-- Document Analysis Handler
>
> http://wiki.apache.org/solr/AnalysisRequestHandler
>
> An analysis handler that provides a breakdown of the analysis
> process of provided docuemnts. This handler expects a (single)
> content stream with the following format:
>
> <docs>
> <doc>
> <field name="id">1</field>
> <field name="name">The Name</field>
> <field name="text">The Text Value</field>
> </doc>
> <doc>...</doc>
> <doc>...</doc>
> ...
> </docs>
>
> Note: Each document must contain a field which serves as the
> unique key. This key is used in the returned response to associate
> an analysis breakdown to the analyzed document.
>
> Like the FieldAnalysisRequestHandler, this handler also supports
> query analysis by sending either an "analysis.query" or "q"
> request parameter that holds the query text to be analyzed. It
> also supports the "analysis.showmatch" parameter which when set to
> true, all field tokens that match the query tokens will be marked
> as a "match".
> -->
> <requestHandler name="/analysis/document"
> class="solr.DocumentAnalysisRequestHandler"
> startup="lazy" />
>
> <!-- Admin Handlers
>
> Admin Handlers - This will register all the standard admin
> RequestHandlers.
> -->
> <requestHandler name="/admin/"
> class="solr.admin.AdminHandlers" />
> <!-- This single handler is equivalent to the following... -->
> <!--
> <requestHandler name="/admin/luke"
> class="solr.admin.LukeRequestHandler" />
> <requestHandler name="/admin/system"
> class="solr.admin.SystemInfoHandler" />
> <requestHandler name="/admin/plugins"
> class="solr.admin.PluginInfoHandler" />
> <requestHandler name="/admin/threads"
> class="solr.admin.ThreadDumpHandler" />
> <requestHandler name="/admin/properties"
> class="solr.admin.PropertiesRequestHandler" />
> <requestHandler name="/admin/file"
> class="solr.admin.ShowFileRequestHandler" >
> -->
> <!-- If you wish to hide files under ${solr.home}/conf, explicitly
> register the ShowFileRequestHandler using:
> -->
> <!--
> <requestHandler name="/admin/file"
> class="solr.admin.ShowFileRequestHandler" >
> <lst name="invariants">
> <str name="hidden">synonyms.txt</str>
> <str name="hidden">anotherfile.txt</str>
> </lst>
> </requestHandler>
> -->
>
> <!-- ping/healthcheck -->
> <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
> <lst name="invariants">
> <str name="qt">search</str>
> <str name="q">solrpingquery</str>
> </lst>
> <lst name="defaults">
> <str name="echoParams">all</str>
> </lst>
> </requestHandler>
>
> <!-- Echo the request contents back to the client -->
> <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
> <lst name="defaults">
> <str name="echoParams">explicit</str>
> <str name="echoHandler">true</str>
> </lst>
> </requestHandler>
>
> <!-- Solr Replication
>
> The SolrReplicationHandler supports replicating indexes from a
> "master" used for indexing and "salves" used for queries.
>
> http://wiki.apache.org/solr/SolrReplication
>
> In the example below, remove the <lst name="master"> section if
> this is just a slave and remove the <lst name="slave"> section
> if this is just a master.
> -->
> <!--
> <requestHandler name="/replication" class="solr.ReplicationHandler" >
> <lst name="master">
> <str name="replicateAfter">commit</str>
> <str name="replicateAfter">startup</str>
> <str name="confFiles">schema.xml,stopwords.txt</str>
> </lst>
> <lst name="slave">
> <str name="masterUrl">http://localhost:8983/solr/replication
> </str>
> <str name="pollInterval">00:00:60</str>
> </lst>
> </requestHandler>
> -->
>
> <!-- Search Components
>
> Search components are registered to SolrCore and used by
> instances of SearchHandler (which can access them by name)
>
> By default, the following components are available:
>
> <searchComponent name="query" class="solr.QueryComponent" />
> <searchComponent name="facet" class="solr.FacetComponent" />
> <searchComponent name="mlt"
> class="solr.MoreLikeThisComponent" />
> <searchComponent name="highlight" class="solr.HighlightComponent" />
> <searchComponent name="stats" class="solr.StatsComponent" />
> <searchComponent name="debug" class="solr.DebugComponent" />
>
> Default configuration in a requestHandler would look like:
>
> <arr name="components">
> <str>query</str>
> <str>facet</str>
> <str>mlt</str>
> <str>highlight</str>
> <str>stats</str>
> <str>debug</str>
> </arr>
>
> If you register a searchComponent to one of the standard names,
> that will be used instead of the default.
>
> To insert components before or after the 'standard' components, use:
>
> <arr name="first-components">
> <str>myFirstComponentName</str>
> </arr>
>
> <arr name="last-components">
> <str>myLastComponentName</str>
> </arr>
>
> NOTE: The component registered with the name "debug" will
> always be executed after the "last-components"
>
> -->
>
> <!-- Spell Check
>
> The spell check component can return a list of alternative spelling
> suggestions.
>
> http://wiki.apache.org/solr/SpellCheckComponent
> -->
> <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
>
> <str name="queryAnalyzerFieldType">textSpell</str>
>
> <!-- Multiple "Spell Checkers" can be declared and used by this
> component
> -->
>
> <!-- a spellchecker built from a field of the main index, and
> written to disk
> -->
> <lst name="spellchecker">
> <str name="name">default</str>
> <str name="field">name</str>
> <str name="spellcheckIndexDir">spellchecker</str>
> <!-- uncomment this to require terms to occur in 1% of the documents
> in order to be included in the dictionary
> <float name="thresholdTokenFrequency">.01</float>
> -->
> </lst>
>
> <!-- a spellchecker that uses a different distance measure -->
> <!--
> <lst name="spellchecker">
> <str name="name">jarowinkler</str>
> <str name="field">spell</str>
> <str name="distanceMeasure">
> org.apache.lucene.search.spell.JaroWinklerDistance
> </str>
> <str name="spellcheckIndexDir">spellcheckerJaro</str>
> </lst>
> -->
>
> <!-- a spellchecker that use an alternate comparator
>
> comparatorClass be one of:
> 1. score (default)
> 2. freq (Frequency first, then score)
> 3. A fully qualified class name
> -->
> <!--
> <lst name="spellchecker">
> <str name="name">freq</str>
> <str name="field">lowerfilt</str>
> <str name="spellcheckIndexDir">spellcheckerFreq</str>
> <str name="comparatorClass">freq</str>
> <str name="buildOnCommit">true</str>
> -->
>
> <!-- A spellchecker that reads the list of words from a file -->
> <!--
> <lst name="spellchecker">
> <str name="classname">solr.FileBasedSpellChecker</str>
> <str name="name">file</str>
> <str name="sourceLocation">spellings.txt</str>
> <str name="characterEncoding">UTF-8</str>
> <str name="spellcheckIndexDir">spellcheckerFile</str>
> </lst>
> -->
> </searchComponent>
>
> <!-- A request handler for demonstrating the spellcheck component.
>
> NOTE: This is purely as an example. The whole purpose of the
> SpellCheckComponent is to hook it into the request handler that
> handles your normal user queries so that a separate request is
> not needed to get suggestions.
>
> IN OTHER WORDS, THERE IS REALLY GOOD CHANCE THE SETUP BELOW IS
> NOT WHAT YOU WANT FOR YOUR PRODUCTION SYSTEM!
>
> See http://wiki.apache.org/solr/SpellCheckComponent for details
> on the request parameters.
> -->
> <requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
> <lst name="defaults">
> <str name="spellcheck.onlyMorePopular">false</str>
> <str name="spellcheck.extendedResults">false</str>
> <str name="spellcheck.count">1</str>
> </lst>
> <arr name="last-components">
> <str>spellcheck</str>
> </arr>
> </requestHandler>
>
> <!-- Term Vector Component
>
> http://wiki.apache.org/solr/TermVectorComponent
> -->
> <searchComponent name="tvComponent" class="solr.TermVectorComponent"/>
>
> <!-- A request handler for demonstrating the term vector component
>
> This is purely as an example.
>
> In reality you will likely want to add the component to your
> already specified request handlers.
> -->
> <requestHandler name="tvrh" class="solr.SearchHandler" startup="lazy">
> <lst name="defaults">
> <bool name="tv">true</bool>
> </lst>
> <arr name="last-components">
> <str>tvComponent</str>
> </arr>
> </requestHandler>
>
> <!-- Clustering Component
>
> http://wiki.apache.org/solr/ClusteringComponent
>
> This relies on third party jars which are notincluded in the
> release. To use this component (and the "/clustering" handler)
> Those jars will need to be downloaded, and you'll need to set
> the solr.cluster.enabled system property when running solr...
>
> java -Dsolr.clustering.enabled=true -jar start.jar
> -->
> <searchComponent name="clustering"
> enable="${solr.clustering.enabled:false}"
> class="solr.clustering.ClusteringComponent" >
> <!-- Declare an engine -->
> <lst name="engine">
> <!-- The name, only one can be named "default" -->
> <str name="name">default</str>
>
> <!-- Class name of Carrot2 clustering algorithm.
>
> Currently available algorithms are:
>
> * org.carrot2.clustering.lingo.LingoClusteringAlgorithm
> * org.carrot2.clustering.stc.STCClusteringAlgorithm
> *
> org.carrot2.clustering.kmeans.BisectingKMeansClusteringAlgorithm
>
> See http://project.carrot2.org/algorithms.html for the
> algorithm's characteristics.
> -->
> <str
> name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm</str>
>
> <!-- Overriding values for Carrot2 default algorithm attributes.
>
> For a description of all available attributes, see:
> http://download.carrot2.org/stable/manual/#chapter.components.
> Use attribute key as name attribute of str elements
> below. These can be further overridden for individual
> requests by specifying attribute key as request parameter
> name and attribute value as parameter value.
> -->
> <str name="LingoClusteringAlgorithm.desiredClusterCountBase">20</str>
>
> <!-- Location of Carrot2 lexical resources.
>
> A directory from which to load Carrot2-specific stop words
> and stop labels. Absolute or relative to Solr config directory.
> If a specific resource (e.g. stopwords.en) is present in the
> specified dir, it will completely override the corresponding
> default one that ships with Carrot2.
>
> For an overview of Carrot2 lexical resources, see:
>
> http://download.carrot2.org/head/manual/#chapter.lexical-resources
> -->
> <str name="carrot.lexicalResourcesDir">clustering/carrot2</str>
>
> <!-- The language to assume for the documents.
>
> For a list of allowed values, see:
>
> http://download.carrot2.org/stable/manual/#section.attribute.lingo.MultilingualClustering.defaultLanguage
> -->
> <str name="MultilingualClustering.defaultLanguage">ENGLISH</str>
> </lst>
> <lst name="engine">
> <str name="name">stc</str>
> <str
> name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm</str>
> </lst>
> </searchComponent>
>
> <!-- A request handler for demonstrating the clustering component
>
> This is purely as an example.
>
> In reality you will likely want to add the component to your
> already specified request handlers.
> -->
> <requestHandler name="/clustering"
> startup="lazy"
> enable="${solr.clustering.enabled:false}"
> class="solr.SearchHandler">
> <lst name="defaults">
> <bool name="clustering">true</bool>
> <str name="clustering.engine">default</str>
> <bool name="clustering.results">true</bool>
> <!-- The title field -->
> <str name="carrot.title">name</str>
> <str name="carrot.url">id</str>
> <!-- The field to cluster on -->
> <str name="carrot.snippet">features</str>
> <!-- produce summaries -->
> <bool name="carrot.produceSummary">true</bool>
> <!-- the maximum number of labels per cluster -->
> <!--<int name="carrot.numDescriptions">5</int>-->
> <!-- produce sub clusters -->
> <bool name="carrot.outputSubClusters">false</bool>
>
> <str name="defType">edismax</str>
> <str name="qf">
> text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
> </str>
> <str name="q.alt">*:*</str>
> <str name="rows">10</str>
> <str name="fl">*,score</str>
> </lst>
> <arr name="last-components">
> <str>clustering</str>
> </arr>
> </requestHandler>
>
> <!-- Terms Component
>
> http://wiki.apache.org/solr/TermsComponent
>
> A component to return terms and document frequency of those
> terms
> -->
> <searchComponent name="terms" class="solr.TermsComponent"/>
>
> <!-- A request handler for demonstrating the terms component -->
> <requestHandler name="/terms" class="solr.SearchHandler" startup="lazy">
> <lst name="defaults">
> <bool name="terms">true</bool>
> </lst>
> <arr name="components">
> <str>terms</str>
> </arr>
> </requestHandler>
>
>
> <!-- Query Elevation Component
>
> http://wiki.apache.org/solr/QueryElevationComponent
>
> a search component that enables you to configure the top
> results for a given query regardless of the normal lucene
> scoring.
> -->
> <!--
> <searchComponent name="elevator" class="solr.QueryElevationComponent" >
> <str name="queryFieldType">string</str>
> <str name="config-file">elevate.xml</str>
> </searchComponent>
> -->
>
> <!-- A request handler for demonstrating the elevator component -->
> <requestHandler name="/elevate" class="solr.SearchHandler"
> startup="lazy">
> <lst name="defaults">
> <str name="echoParams">explicit</str>
> </lst>
> <arr name="last-components">
> <str>elevator</str>
> </arr>
> </requestHandler>
>
> <!-- Highlighting Component
>
> http://wiki.apache.org/solr/HighlightingParameters
> -->
> <searchComponent class="solr.HighlightComponent" name="highlight">
> <highlighting>
> <!-- Configure the standard fragmenter -->
> <!-- This could most likely be commented out in the "default" case
> -->
> <fragmenter name="gap"
> default="true"
> class="solr.highlight.GapFragmenter">
> <lst name="defaults">
> <int name="hl.fragsize">100</int>
> </lst>
> </fragmenter>
>
> <!-- A regular-expression-based fragmenter
> (for sentence extraction)
> -->
> <fragmenter name="regex"
> class="solr.highlight.RegexFragmenter">
> <lst name="defaults">
> <!-- slightly smaller fragsizes work better because of slop -->
> <int name="hl.fragsize">70</int>
> <!-- allow 50% slop on fragment sizes -->
> <float name="hl.regex.slop">0.5</float>
> <!-- a basic sentence pattern -->
> <str name="hl.regex.pattern">[-\w
> ,/\n\"']{20,200}</str>
> </lst>
> </fragmenter>
>
> <!-- Configure the standard formatter -->
> <formatter name="html"
> default="true"
> class="solr.highlight.HtmlFormatter">
> <lst name="defaults">
> <str name="hl.simple.pre"><![CDATA[<em>]]></str>
> <str name="hl.simple.post"><![CDATA[</em>]]></str>
> </lst>
> </formatter>
>
> <!-- Configure the standard encoder -->
> <encoder name="html"
> class="solr.highlight.HtmlEncoder" />
>
> <!-- Configure the standard fragListBuilder -->
> <fragListBuilder name="simple"
> default="true"
> class="solr.highlight.SimpleFragListBuilder"/>
>
> <!-- Configure the single fragListBuilder -->
> <fragListBuilder name="single"
> class="solr.highlight.SingleFragListBuilder"/>
>
> <!-- default tag FragmentsBuilder -->
> <fragmentsBuilder name="default"
> default="true"
> class="solr.highlight.ScoreOrderFragmentsBuilder">
> <!--
> <lst name="defaults">
> <str name="hl.multiValuedSeparatorChar">/</str>
> </lst>
> -->
> </fragmentsBuilder>
>
> <!-- multi-colored tag FragmentsBuilder -->
> <fragmentsBuilder name="colored"
> class="solr.highlight.ScoreOrderFragmentsBuilder">
> <lst name="defaults">
> <str name="hl.tag.pre"><![CDATA[
> <b style="background:yellow">,<b
> style="background:lawgreen">,
> <b style="background:aquamarine">,<b
> style="background:magenta">,
> <b style="background:palegreen">,<b
> style="background:coral">,
> <b style="background:wheat">,<b style="background:khaki">,
> <b style="background:lime">,<b
> style="background:deepskyblue">]]></str>
> <str name="hl.tag.post"><![CDATA[</b>]]></str>
> </lst>
> </fragmentsBuilder>
>
> <boundaryScanner name="default"
> default="true"
> class="solr.highlight.SimpleBoundaryScanner">
> <lst name="defaults">
> <str name="hl.bs.maxScan">10</str>
> <str name="hl.bs.chars">.,!? 	 </str>
> </lst>
> </boundaryScanner>
>
> <boundaryScanner name="breakIterator"
> class="solr.highlight.BreakIteratorBoundaryScanner">
> <lst name="defaults">
> <!-- type should be one of CHARACTER, WORD(default), LINE and
> SENTENCE -->
> <str name="hl.bs.type">WORD</str>
> <!-- language and country are used when constructing Locale
> object. -->
> <!-- And the Locale object will be used when getting instance of
> BreakIterator -->
> <str name="hl.bs.language">en</str>
> <str name="hl.bs.country">US</str>
> </lst>
> </boundaryScanner>
> </highlighting>
> </searchComponent>
>
> <!-- Update Processors
>
> Chains of Update Processor Factories for dealing with Update
> Requests can be declared, and then used by name in Update
> Request Processors
>
> http://wiki.apache.org/solr/UpdateRequestProcessor
>
> -->
> <!-- Deduplication
>
> An example dedup update processor that creates the "id" field
> on the fly based on the hash code of some other fields. This
> example has overwriteDupes set to false since we are using the
> id field as the signatureField and Solr will maintain
> uniqueness based on that anyway.
>
> -->
> <!--
> <updateRequestProcessorChain name="dedupe">
> <processor class="solr.processor.SignatureUpdateProcessorFactory">
> <bool name="enabled">true</bool>
> <str name="signatureField">id</str>
> <bool name="overwriteDupes">false</bool>
> <str name="fields">name,features,cat</str>
> <str name="signatureClass">solr.processor.Lookup3Signature</str>
> </processor>
> <processor class="solr.LogUpdateProcessorFactory" />
> <processor class="solr.RunUpdateProcessorFactory" />
> </updateRequestProcessorChain>
> -->
>
> <!--
> This example update chain identifies the language of the incoming
> documents using the langid contrib. The detected language is
> written to field language_s. No field name mapping is done.
> The fields used for detection are text, title, subject and
> description,
> making this example suitable for detecting languages form full-text
> rich documents injected via ExtractingRequestHandler.
> See more about langId at
> http://wiki.apache.org/solr/LanguageDetection
> -->
> <!--
> <updateRequestProcessorChain name="langid">
> <processor
> class="org.apache.solr.update.processor.TikaLanguageIdentifierUpdateProcessorFactory">
> <str name="langid.fl">text,title,subject,description</str>
> <str name="langid.langField">language_s</str>
> <str name="langid.fallback">en</str>
> </processor>
> <processor class="solr.LogUpdateProcessorFactory" />
> <processor class="solr.RunUpdateProcessorFactory" />
> </updateRequestProcessorChain>
> -->
>
> <!-- Response Writers
>
> http://wiki.apache.org/solr/QueryResponseWriter
>
> Request responses will be written using the writer specified by
> the 'wt' request parameter matching the name of a registered
> writer.
>
> The "default" writer is the default and will be used if 'wt' is
> not specified in the request.
> -->
> <!-- The following response writers are implicitly configured unless
> overridden...
> -->
> <!--
> <queryResponseWriter name="xml"
> default="true"
> class="solr.XMLResponseWriter" />
> <queryResponseWriter name="json" class="solr.JSONResponseWriter"/>
> <queryResponseWriter name="python" class="solr.PythonResponseWriter"/>
> <queryResponseWriter name="ruby" class="solr.RubyResponseWriter"/>
> <queryResponseWriter name="php" class="solr.PHPResponseWriter"/>
> <queryResponseWriter name="phps"
> class="solr.PHPSerializedResponseWriter"/>
> <queryResponseWriter name="csv" class="solr.CSVResponseWriter"/>
> -->
>
> <queryResponseWriter name="json" class="solr.JSONResponseWriter">
> <!-- For the purposes of the tutorial, JSON responses are written as
> plain text so that they are easy to read in *any* browser.
> If you expect a MIME type of "application/json" just remove this
> override.
> -->
> <str name="content-type">text/plain; charset=UTF-8</str>
> </queryResponseWriter>
>
> <!--
> Custom response writers can be declared as needed...
> -->
> <!-- The solr.velocity.enabled flag is used by Solr's test cases so
> that this response writer is not
> loaded (causing an error if contrib/velocity has not been built
> fully) -->
> <!--
> <queryResponseWriter name="velocity"
> class="solr.VelocityResponseWriter" enable="${solr.velocity.enabled:true}"/>
> -->
>
>
> <!-- XSLT response writer transforms the XML output by any xslt file
> found
> in Solr's conf/xslt directory. Changes to xslt files are checked
> for
> every xsltCacheLifetimeSeconds.
> -->
> <queryResponseWriter name="xslt" class="solr.XSLTResponseWriter">
> <int name="xsltCacheLifetimeSeconds">5</int>
> </queryResponseWriter>
>
> <!-- Query Parsers
>
> http://wiki.apache.org/solr/SolrQuerySyntax
>
> Multiple QParserPlugins can be registered by name, and then
> used in either the "defType" param for the QueryComponent (used
> by SearchHandler) or in LocalParams
> -->
> <!-- example of registering a query parser -->
> <!--
> <queryParser name="myparser" class="com.mycompany.MyQParserPlugin"/>
> -->
>
> <!-- Function Parsers
>
> http://wiki.apache.org/solr/FunctionQuery
>
> Multiple ValueSourceParsers can be registered by name, and then
> used as function names when using the "func" QParser.
> -->
> <!-- example of registering a custom function parser -->
> <!--
> <valueSourceParser name="myfunc"
> class="com.mycompany.MyValueSourceParser" />
> -->
>
> <!-- Legacy config for the admin interface -->
> <admin>
> <defaultQuery>*:*</defaultQuery>
>
> <!-- configure a healthcheck file for servers behind a
> loadbalancer
> -->
> <!--
> <healthcheck type="file">server-enabled</healthcheck>
> -->
> </admin>
>
> <requestHandler name="/replication" class="solr.ReplicationHandler" >
> <lst name="master">
> <!--Replicate on 'startup' and 'commit'. 'optimize' is also a
> valid value for replicateAfter. -->
> <str name="replicateAfter">startup</str>
> <str name="replicateAfter">commit</str>
>
> <!--Create a backup after 'optimize'. Other values can be
> 'commit', 'startup'. It is possible to have multiple entries of this config
> string. Note that this is just for backup, replication does not require
> this. -->
> <!-- <str name="backupAfter">optimize</str> -->
>
> <!--If configuration files need to be replicated give the names
> here, separated by comma -->
> <str name="confFiles">schema.xml,stopwords.txt,elevate.xml</str>
> <!--The default value of reservation is 10 secs.See the
> documentation below . Normally , you should not need to specify this -->
> <str name="commitReserveDuration">00:01:00</str>
> </lst>
> </requestHandler>
>
> </config>
>
>
>
>
>
> On Mon, Feb 13, 2012 at 12:36 PM, Li Li <fa...@gmail.com> wrote:
>
>> can you post your config file?
>> I found there are 2 places to config ramBufferSizeMB in latest svn of
>> 3.6's
>> example solrconfig.xml. trying to modify them both?
>>
>> <indexDefaults>
>>
>> <useCompoundFile>false</useCompoundFile>
>>
>> <mergeFactor>10</mergeFactor>
>> <!-- Sets the amount of RAM that may be used by Lucene indexing
>> for buffering added documents and deletions before they are
>> flushed to the Directory. -->
>> <ramBufferSizeMB>32</ramBufferSizeMB>
>> <!-- If both ramBufferSizeMB and maxBufferedDocs is set, then
>> Lucene will flush based on whichever limit is hit first.
>> -->
>> <!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
>>
>> <maxFieldLength>10000</maxFieldLength>
>> <writeLockTimeout>1000</writeLockTimeout>
>>
>> .............
>> <!-- <termIndexInterval>256</termIndexInterval> -->
>> </indexDefaults>
>>
>> <!-- Main Index
>>
>> Values here override the values in the <indexDefaults> section
>> for the main on disk index.
>> -->
>> <mainIndex>
>>
>> <useCompoundFile>false</useCompoundFile>
>> <ramBufferSizeMB>32</ramBufferSizeMB>
>> <mergeFactor>10</mergeFactor>
>> ........
>> </mainIndex>
>>
>> On Tue, Feb 14, 2012 at 1:10 AM, Huy Le <hu...@springpartners.com> wrote:
>>
>> > Hi,
>> >
>> > I am using solr 3.5. As I understood it, NRT is a solr 4 feature, but
>> solr
>> > 4 is not released yet.
>> >
>> > I understand commit after adding each document is expensive, but the
>> > application requires that documents be available after adding to the
>> index.
>> >
>> > What I don't understand is why new segment files are created so often.
>> > Are the commit calls triggering new segment files being created? I
>> don't
>> > see this behavior in another environment of the same version of solr.
>> >
>> > Huy
>> >
>> > On Mon, Feb 13, 2012 at 11:55 AM, Li Li <fa...@gmail.com> wrote:
>> >
>> > > Commit is called
>> > > after adding each document
>> > >
>> > >
>> > > you should add enough documents and then calling a commit. commit is
>> a
>> > > cost operation.
>> > > if you want to get latest feeded documents, you could use NRT
>> > >
>> > > On Tue, Feb 14, 2012 at 12:47 AM, Huy Le <hu...@springpartners.com>
>> > wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > I am using solr 3.5. I seeing solr keeps creating new segment files
>> > > (<1MB
>> > > > files) so often that it triggers segment merge about every one
>> minute.
>> > I
>> > > > search the news archive, but could not find any info on this issue.
>> I
>> > am
>> > > > indexing about 10 docs of less 2KB each every second. Commit is
>> called
>> > > > after adding each document. Relevant config params are:
>> > > >
>> > > > <mergeFactor>10</mergeFactor>
>> > > > <ramBufferSizeMB>1024</ramBufferSizeMB>
>> > > > <maxMergeDocs>2147483647</maxMergeDocs>
>> > > >
>> > > > What might be triggering this frequent new segment files creation?
>> > > Thanks!
>> > > >
>> > > > Huy
>> > > >
>> > > > --
>> > > > Huy Le
>> > > > Spring Partners, Inc.
>> > > > http://springpadit.com
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Huy Le
>> > Spring Partners, Inc.
>> > http://springpadit.com
>> >
>>
>
>
>
> --
> Huy Le
> Spring Partners, Inc.
> http://springpadit.com
>
--
Huy Le
Spring Partners, Inc.
http://springpadit.com
Re: New segment file created too often
Posted by Huy Le <hu...@springpartners.com>.
I do have the indexDefault and mainIndex sections configured with the
desired values. As trying to get to the root cause of this issue, I have
the application disable commit calls, and segment files no longer that
created/merged so often. I then manually call commit (curl '
http://localhost:8983/solr/sbr01/update?commit=true'), segment files get
merged immediately. This suggest somethings to do with commits.
Here is content of the config file:
<?xml version="1.0" encoding="UTF-8" ?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!--
For more details about configurations options that may appear in
this file, see http://wiki.apache.org/solr/SolrConfigXml.
-->
<config>
<!-- In all configuration below, a prefix of "solr." for class names
is an alias that causes solr to search appropriate packages,
including org.apache.solr.(search|update|request|core|analysis)
You may also specify a fully qualified Java classname if you
have your own custom plugins.
-->
<!-- Set this to 'false' if you want solr to continue working after
it has encountered an severe configuration error. In a
production environment, you may want solr to keep working even
if one handler is mis-configured.
You may also set this to false using by setting the system
property:
-Dsolr.abortOnConfigurationError=false
-->
<abortOnConfigurationError>${solr.abortOnConfigurationError:true}</abortOnConfigurationError>
<!-- Controls what version of Lucene various components of Solr
adhere to. Generally, you want to use the latest version to
get all bug fixes and improvements. It is highly recommended
that you fully re-index after changing this setting as it can
affect both how text is indexed and queried.
-->
<luceneMatchVersion>LUCENE_35</luceneMatchVersion>
<!-- lib directives can be used to instruct Solr to load an Jars
identified and use them to resolve any "plugins" specified in
your solrconfig.xml or schema.xml (ie: Analyzers, Request
Handlers, etc...).
All directories and paths are resolved relative to the
instanceDir.
If a "./lib" directory exists in your instanceDir, all files
found in it are included as if you had used the following
syntax...
<lib dir="./lib" />
-->
<!-- A dir option by itself adds any files found in the directory to
the classpath, this is useful for including all jars in a
directory.
-->
<lib dir="../../contrib/extraction/lib" />
<lib dir="../../contrib/clustering/lib/" />
<lib dir="../../contrib/velocity/lib" />
<!-- When a regex is specified in addition to a directory, only the
files in that directory which completely match the regex
(anchored on both ends) will be included.
-->
<lib dir="../../dist/" regex="apache-solr-cell-\d.*\.jar" />
<lib dir="../../dist/" regex="apache-solr-clustering-\d.*\.jar" />
<lib dir="../../dist/" regex="apache-solr-dataimporthandler-\d.*\.jar" />
<lib dir="../../dist/" regex="apache-solr-langid-\d.*\.jar" />
<lib dir="../../dist/" regex="apache-solr-velocity-\d.*\.jar" />
<!-- If a dir option (with or without a regex) is used and nothing
is found that matches, it will be ignored
-->
<lib dir="../../contrib/clustering/lib/" />
<lib dir="/total/crap/dir/ignored" />
<!-- an exact path can be used to specify a specific file. This
will cause a serious error to be logged if it can't be loaded.
-->
<!--
<lib path="../a-jar-that-does-not-exist.jar" />
-->
<!-- Data Directory
Used to specify an alternate directory to hold all index data
other than the default ./data under the Solr home. If
replication is in use, this should match the replication
configuration.
-->
<dataDir>${solr.data.dir:/u01/solr/sbr01/data}</dataDir>
<!-- The DirectoryFactory to use for indexes.
solr.StandardDirectoryFactory, the default, is filesystem
based and tries to pick the best implementation for the current
JVM and platform. One can force a particular implementation
via solr.MMapDirectoryFactory, solr.NIOFSDirectoryFactory, or
solr.SimpleFSDirectoryFactory.
solr.RAMDirectoryFactory is memory based, not
persistent, and doesn't work with replication.
-->
<directoryFactory name="DirectoryFactory"
class="${solr.directoryFactory:solr.StandardDirectoryFactory}"/>
<!-- Index Defaults
Values here affect all index writers and act as a default
unless overridden.
WARNING: See also the <mainIndex> section below for parameters
that overfor Solr's main Lucene index.
-->
<indexDefaults>
<useCompoundFile>true</useCompoundFile>
<mergeFactor>10</mergeFactor>
<!-- Sets the amount of RAM that may be used by Lucene indexing
for buffering added documents and deletions before they are
flushed to the Directory. -->
<ramBufferSizeMB>1024</ramBufferSizeMB>
<!-- If both ramBufferSizeMB and maxBufferedDocs is set, then
Lucene will flush based on whichever limit is hit first.
-->
<!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
<!-- springpad configure -->
<maxMergeDocs>2147483647</maxMergeDocs>
<!-- end springpad configure -->
<maxFieldLength>10000</maxFieldLength>
<writeLockTimeout>1000</writeLockTimeout>
<!-- Expert: Merge Policy
The Merge Policy in Lucene controls how merging is handled by
Lucene. The default in Solr 3.3 is TieredMergePolicy.
The default in 2.3 was the LogByteSizeMergePolicy,
previous versions used LogDocMergePolicy.
LogByteSizeMergePolicy chooses segments to merge based on
their size. The Lucene 2.2 default, LogDocMergePolicy chose
when to merge based on number of documents
Other implementations of MergePolicy must have a no-argument
constructor
-->
<!--
<mergePolicy class="org.apache.lucene.index.TieredMergePolicy"/>
-->
<!-- Expert: Merge Scheduler
The Merge Scheduler in Lucene controls how merges are
performed. The ConcurrentMergeScheduler (Lucene 2.3 default)
can perform merges in the background using separate threads.
The SerialMergeScheduler (Lucene 2.2 default) does not.
-->
<!--
<mergeScheduler
class="org.apache.lucene.index.ConcurrentMergeScheduler"/>
-->
<!-- LockFactory
This option specifies which Lucene LockFactory implementation
to use.
single = SingleInstanceLockFactory - suggested for a
read-only index or when there is no possibility of
another process trying to modify the index.
native = NativeFSLockFactory - uses OS native file locking.
Do not use when multiple solr webapps in the same
JVM are attempting to share a single index.
simple = SimpleFSLockFactory - uses a plain file for locking
(For backwards compatibility with Solr 1.2, 'simple' is the
default if not specified.)
More details on the nuances of each LockFactory...
http://wiki.apache.org/lucene-java/AvailableLockFactories
-->
<lockType>native</lockType>
<!-- Expert: Controls how often Lucene loads terms into memory
Default is 128 and is likely good for most everyone.
-->
<!-- <termIndexInterval>256</termIndexInterval> -->
</indexDefaults>
<!-- Main Index
Values here override the values in the <indexDefaults> section
for the main on disk index.
-->
<mainIndex>
<maxMergeDocs>2147483647</maxMergeDocs>
<useCompoundFile>true</useCompoundFile>
<ramBufferSizeMB>1024</ramBufferSizeMB>
<mergeFactor>30</mergeFactor>
<!-- Unlock On Startup
If true, unlock any held write or commit locks on startup.
This defeats the locking mechanism that allows multiple
processes to safely access a lucene index, and should be used
with care.
This is not needed if lock type is 'none' or 'single'
-->
<unlockOnStartup>false</unlockOnStartup>
<!-- If true, IndexReaders will be reopened (often more efficient)
instead of closed and then opened.
-->
<reopenReaders>true</reopenReaders>
<!-- Commit Deletion Policy
Custom deletion policies can specified here. The class must
implement org.apache.lucene.index.IndexDeletionPolicy.
http://lucene.apache.org/java/2_9_1/api/all/org/apache/lucene/index/IndexDeletionPolicy.html
The standard Solr IndexDeletionPolicy implementation supports
deleting index commit points on number of commits, age of
commit point and optimized status.
The latest commit point should always be preserved regardless
of the criteria.
-->
<deletionPolicy class="solr.SolrDeletionPolicy">
<!-- The number of commit points to be kept -->
<str name="maxCommitsToKeep">1</str>
<!-- The number of optimized commit points to be kept -->
<str name="maxOptimizedCommitsToKeep">0</str>
<!--
Delete all commit points once they have reached the given age.
Supports DateMathParser syntax e.g.
-->
<!--
<str name="maxCommitAge">30MINUTES</str>
<str name="maxCommitAge">1DAY</str>
-->
</deletionPolicy>
<!-- Lucene Infostream
To aid in advanced debugging, Lucene provides an "InfoStream"
of detailed information when indexing.
Setting The value to true will instruct the underlying Lucene
IndexWriter to write its debugging info the specified file
-->
<infoStream file="INFOSTREAM.txt">false</infoStream>
</mainIndex>
<!-- JMX
This example enables JMX if and only if an existing MBeanServer
is found, use this if you want to configure JMX through JVM
parameters. Remove this to disable exposing Solr configuration
and statistics to JMX.
For more details see http://wiki.apache.org/solr/SolrJmx
-->
<jmx />
<!-- If you want to connect to a particular server, specify the
agentId
-->
<!-- <jmx agentId="myAgent" /> -->
<!-- If you want to start a new MBeanServer, specify the serviceUrl -->
<!-- <jmx serviceUrl="service:jmx:rmi:///jndi/rmi://localhost:9999/solr"/>
-->
<!-- The default high-performance update handler -->
<updateHandler class="solr.DirectUpdateHandler2">
<!-- AutoCommit
Perform a <commit/> automatically under certain conditions.
Instead of enabling autoCommit, consider using "commitWithin"
when adding documents.
http://wiki.apache.org/solr/UpdateXmlMessages
maxDocs - Maximum number of documents to add since the last
commit before automatically triggering a new commit.
maxTime - Maximum amount of time that is allowed to pass
since a document was added before automaticly
triggering a new commit.
-->
<autoCommit>
<maxDocs>10000</maxDocs>
<maxTime>300000</maxTime>
</autoCommit>
<!-- Update Related Event Listeners
Various IndexWriter related events can trigger Listeners to
take actions.
postCommit - fired after every commit or optimize command
postOptimize - fired after every optimize command
-->
<!-- The RunExecutableListener executes an external command from a
hook such as postCommit or postOptimize.
exe - the name of the executable to run
dir - dir to use as the current working directory. (default=".")
wait - the calling thread waits until the executable returns.
(default="true")
args - the arguments to pass to the program. (default is none)
env - environment variables to set. (default is none)
-->
<!-- This example shows how RunExecutableListener could be used
with the script based replication...
http://wiki.apache.org/solr/CollectionDistribution
-->
<!--
<listener event="postCommit" class="solr.RunExecutableListener">
<str name="exe">solr/bin/snapshooter</str>
<str name="dir">.</str>
<bool name="wait">true</bool>
<arr name="args"> <str>arg1</str> <str>arg2</str> </arr>
<arr name="env"> <str>MYVAR=val1</str> </arr>
</listener>
-->
</updateHandler>
<!-- IndexReaderFactory
Use the following format to specify a custom IndexReaderFactory,
which allows for alternate IndexReader implementations.
** Experimental Feature **
Please note - Using a custom IndexReaderFactory may prevent
certain other features from working. The API to
IndexReaderFactory may change without warning or may even be
removed from future releases if the problems cannot be
resolved.
** Features that may not work with custom IndexReaderFactory **
The ReplicationHandler assumes a disk-resident index. Using a
custom IndexReader implementation may cause incompatibility
with ReplicationHandler and may cause replication to not work
correctly. See SOLR-1366 for details.
-->
<!--
<indexReaderFactory name="IndexReaderFactory" class="package.class">
<str name="someArg">Some Value</str>
</indexReaderFactory >
-->
<!-- By explicitly declaring the Factory, the termIndexDivisor can
be specified.
-->
<!--
<indexReaderFactory name="IndexReaderFactory"
class="solr.StandardIndexReaderFactory">
<int name="setTermIndexDivisor">12</int>
</indexReaderFactory >
-->
<query>
<!-- Max Boolean Clauses
Maximum number of clauses in each BooleanQuery, an exception
is thrown if exceeded.
** WARNING **
This option actually modifies a global Lucene property that
will affect all SolrCores. If multiple solrconfig.xml files
disagree on this property, the value at any given moment will
be based on the last SolrCore to be initialized.
-->
<maxBooleanClauses>1024</maxBooleanClauses>
<!-- Solr Internal Query Caches
There are two implementations of cache available for Solr,
LRUCache, based on a synchronized LinkedHashMap, and
FastLRUCache, based on a ConcurrentHashMap.
FastLRUCache has faster gets and slower puts in single
threaded operation and thus is generally faster than LRUCache
when the hit ratio of the cache is high (> 75%), and may be
faster under other scenarios on multi-cpu systems.
-->
<!-- Filter Cache
Cache used by SolrIndexSearcher for filters (DocSets),
unordered sets of *all* documents that match a query. When a
new searcher is opened, its caches may be prepopulated or
"autowarmed" using data from caches in the old searcher.
autowarmCount is the number of items to prepopulate. For
LRUCache, the autowarmed items will be the most recently
accessed items.
Parameters:
class - the SolrCache implementation LRUCache or
(LRUCache or FastLRUCache)
size - the maximum number of entries in the cache
initialSize - the initial capacity (number of entries) of
the cache. (see java.util.HashMap)
autowarmCount - the number of entries to prepopulate from
and old cache.
-->
<filterCache class="solr.FastLRUCache"
size="20000"
initialSize="1000"
autowarmCount="0"/>
<!-- Query Result Cache
Caches results of searches - ordered lists of document ids
(DocList) based on a query, a sort, and the range of documents
requested.
-->
<queryResultCache class="solr.LRUCache"
size="10000"
initialSize="1000"
autowarmCount="0"/>
<!-- Document Cache
Caches Lucene Document objects (the stored fields for each
document). Since Lucene internal document ids are transient,
this cache will not be autowarmed.
-->
<documentCache class="solr.LRUCache"
size="15000"
initialSize="1000"
autowarmCount="0"/>
<!-- Field Value Cache
Cache used to hold field values that are quickly accessible
by document id. The fieldValueCache is created by default
even if not configured here.
-->
<!--
<fieldValueCache class="solr.FastLRUCache"
size="512"
autowarmCount="128"
showItems="32" />
-->
<!-- Custom Cache
Example of a generic cache. These caches may be accessed by
name through SolrIndexSearcher.getCache(),cacheLookup(), and
cacheInsert(). The purpose is to enable easy caching of
user/application level data. The regenerator argument should
be specified as an implementation of solr.CacheRegenerator
if autowarming is desired.
-->
<!--
<cache name="myUserCache"
class="solr.LRUCache"
size="4096"
initialSize="1024"
autowarmCount="1024"
regenerator="com.mycompany.MyRegenerator"
/>
-->
<!-- Lazy Field Loading
If true, stored fields that are not requested will be loaded
lazily. This can result in a significant speed improvement
if the usual case is to not load all stored fields,
especially if the skipped fields are large compressed text
fields.
-->
<enableLazyFieldLoading>true</enableLazyFieldLoading>
<!-- Use Filter For Sorted Query
A possible optimization that attempts to use a filter to
satisfy a search. If the requested sort does not include
score, then the filterCache will be checked for a filter
matching the query. If found, the filter will be used as the
source of document ids, and then the sort will be applied to
that.
For most situations, this will not be useful unless you
frequently get the same search repeatedly with different sort
options, and none of them ever use "score"
-->
<!--
<useFilterForSortedQuery>true</useFilterForSortedQuery>
-->
<!-- Result Window Size
An optimization for use with the queryResultCache. When a search
is requested, a superset of the requested number of document ids
are collected. For example, if a search for a particular query
requests matching documents 10 through 19, and queryWindowSize is
50,
then documents 0 through 49 will be collected and cached. Any
further
requests in that range can be satisfied via the cache.
-->
<queryResultWindowSize>50</queryResultWindowSize>
<!-- Maximum number of documents to cache for any entry in the
queryResultCache.
-->
<queryResultMaxDocsCached>200</queryResultMaxDocsCached>
<!-- Query Related Event Listeners
Various IndexSearcher related events can trigger Listeners to
take actions.
newSearcher - fired whenever a new searcher is being prepared
and there is a current searcher handling requests (aka
registered). It can be used to prime certain caches to
prevent long request times for certain requests.
firstSearcher - fired whenever a new searcher is being
prepared but there is no current registered searcher to handle
requests or to gain autowarming data from.
-->
<!-- QuerySenderListener takes an array of NamedList and executes a
local query request for each NamedList in sequence.
-->
<listener event="newSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<!--
<lst><str name="q">solr</str><str name="sort">price
asc</str></lst>
<lst><str name="q">rocks</str><str name="sort">weight
asc</str></lst>
-->
</arr>
</listener>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst>
<str name="q">static firstSearcher warming in solrconfig.xml</str>
</lst>
</arr>
</listener>
<!-- Use Cold Searcher
If a search request comes in and there is no current
registered searcher, then immediately register the still
warming searcher and use it. If "false" then all requests
will block until the first searcher is done warming.
-->
<useColdSearcher>true</useColdSearcher>
<!-- Max Warming Searchers
Maximum number of searchers that may be warming in the
background concurrently. An error is returned if this limit
is exceeded.
Recommend values of 1-2 for read-only slaves, higher for
masters w/o cache warming.
-->
<maxWarmingSearchers>100</maxWarmingSearchers>
</query>
<!-- Request Dispatcher
This section contains instructions for how the SolrDispatchFilter
should behave when processing requests for this SolrCore.
handleSelect affects the behavior of requests such as /select?qt=XXX
handleSelect="true" will cause the SolrDispatchFilter to process
the request and will result in consistent error handling and
formatting for all types of requests.
handleSelect="false" will cause the SolrDispatchFilter to
ignore "/select" requests and fallback to using the legacy
SolrServlet and it's Solr 1.1 style error formatting
-->
<requestDispatcher handleSelect="true" >
<!-- Request Parsing
These settings indicate how Solr Requests may be parsed, and
what restrictions may be placed on the ContentStreams from
those requests
enableRemoteStreaming - enables use of the stream.file
and stream.url parameters for specifying remote streams.
multipartUploadLimitInKB - specifies the max size of
Multipart File Uploads that Solr will allow in a Request.
*** WARNING ***
The settings below authorize Solr to fetch remote files, You
should make sure your system has some authentication before
using enableRemoteStreaming="true"
-->
<requestParsers enableRemoteStreaming="true"
multipartUploadLimitInKB="2048000" />
<!-- HTTP Caching
Set HTTP caching related parameters (for proxy caches and clients).
The options below instruct Solr not to output any HTTP Caching
related headers
-->
<httpCaching never304="true" />
<!-- If you include a <cacheControl> directive, it will be used to
generate a Cache-Control header (as well as an Expires header
if the value contains "max-age=")
By default, no Cache-Control header is generated.
You can use the <cacheControl> option even if you have set
never304="true"
-->
<!--
<httpCaching never304="true" >
<cacheControl>max-age=30, public</cacheControl>
</httpCaching>
-->
<!-- To enable Solr to respond with automatically generated HTTP
Caching headers, and to response to Cache Validation requests
correctly, set the value of never304="false"
This will cause Solr to generate Last-Modified and ETag
headers based on the properties of the Index.
The following options can also be specified to affect the
values of these headers...
lastModFrom - the default value is "openTime" which means the
Last-Modified value (and validation against If-Modified-Since
requests) will all be relative to when the current Searcher
was opened. You can change it to lastModFrom="dirLastMod" if
you want the value to exactly correspond to when the physical
index was last modified.
etagSeed="..." is an option you can change to force the ETag
header (and validation against If-None-Match requests) to be
different even if the index has not changed (ie: when making
significant changes to your config file)
(lastModifiedFrom and etagSeed are both ignored if you use
the never304="true" option)
-->
<!--
<httpCaching lastModifiedFrom="openTime"
etagSeed="Solr">
<cacheControl>max-age=30, public</cacheControl>
</httpCaching>
-->
</requestDispatcher>
<!-- Request Handlers
http://wiki.apache.org/solr/SolrRequestHandler
incoming queries will be dispatched to the correct handler
based on the path or the qt (query type) param.
Names starting with a '/' are accessed with the a path equal to
the registered name. Names without a leading '/' are accessed
with: http://host/app/[core/]select?qt=name
If a /select request is processed with out a qt param
specified, the requestHandler that declares default="true" will
be used.
If a Request Handler is declared with startup="lazy", then it will
not be initialized until the first request that uses it.
-->
<!-- SearchHandler
http://wiki.apache.org/solr/SearchHandler
For processing Search Queries, the primary Request Handler
provided with Solr is "SearchHandler" It delegates to a sequent
of SearchComponents (see below) and supports distributed
queries across multiple shards
-->
<requestHandler name="search" class="solr.SearchHandler" default="true">
<!-- default values for query parameters can be specified, these
will be overridden by parameters in the request
-->
<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10</int>
</lst>
<!-- In addition to defaults, "appends" params can be specified
to identify values which should be appended to the list of
multi-val params from the query (or the existing "defaults").
-->
<!-- In this example, the param "fq=instock:true" would be appended to
any query time fq params the user may specify, as a mechanism for
partitioning the index, independent of any user selected filtering
that may also be desired (perhaps as a result of faceted
searching).
NOTE: there is *absolutely* nothing a client can do to prevent
these
"appends" values from being used, so don't use this mechanism
unless you are sure you always want it.
-->
<!--
<lst name="appends">
<str name="fq">inStock:true</str>
</lst>
-->
<!-- "invariants" are a way of letting the Solr maintainer lock down
the options available to Solr clients. Any params values
specified here are used regardless of what values may be specified
in either the query, the "defaults", or the "appends" params.
In this example, the facet.field and facet.query params would
be fixed, limiting the facets clients can use. Faceting is
not turned on by default - but if the client does specify
facet=true in the request, these are the only facets they
will be able to see counts for; regardless of what other
facet.field or facet.query params they may specify.
NOTE: there is *absolutely* nothing a client can do to prevent
these
"invariants" values from being used, so don't use this mechanism
unless you are sure you always want it.
-->
<!--
<lst name="invariants">
<str name="facet.field">cat</str>
<str name="facet.field">manu_exact</str>
<str name="facet.query">price:[* TO 500]</str>
<str name="facet.query">price:[500 TO *]</str>
</lst>
-->
<!-- If the default list of SearchComponents is not desired, that
list can either be overridden completely, or components can be
prepended or appended to the default list. (see below)
-->
<!--
<arr name="components">
<str>nameOfCustomComponent1</str>
<str>nameOfCustomComponent2</str>
</arr>
-->
</requestHandler>
<!-- A Robust Example
This example SearchHandler declaration shows off usage of the
SearchHandler with many defaults declared
Note that multiple instances of the same Request Handler
(SearchHandler) can be registered multiple times with different
names (and different init parameters)
-->
<requestHandler name="/browse" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<!-- VelocityResponseWriter settings -->
<!---
<str name="wt">velocity</str>
-->
<str name="v.template">browse</str>
<str name="v.layout">layout</str>
<str name="title">Solritas</str>
<str name="defType">edismax</str>
<str name="q.alt">*:*</str>
<str name="rows">10</str>
<str name="fl">*,score</str>
<str name="mlt.qf">
text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
</str>
<str name="mlt.fl">text,features,name,sku,id,manu,cat</str>
<int name="mlt.count">3</int>
<str name="qf">
text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
</str>
<str name="facet">on</str>
<str name="facet.field">cat</str>
<str name="facet.field">manu_exact</str>
<str name="facet.query">ipod</str>
<str name="facet.query">GB</str>
<str name="facet.mincount">1</str>
<str name="facet.pivot">cat,inStock</str>
<str name="facet.range.other">after</str>
<str name="facet.range">price</str>
<int name="f.price.facet.range.start">0</int>
<int name="f.price.facet.range.end">600</int>
<int name="f.price.facet.range.gap">50</int>
<str name="facet.range">popularity</str>
<int name="f.popularity.facet.range.start">0</int>
<int name="f.popularity.facet.range.end">10</int>
<int name="f.popularity.facet.range.gap">3</int>
<str name="facet.range">manufacturedate_dt</str>
<str
name="f.manufacturedate_dt.facet.range.start">NOW/YEAR-10YEARS</str>
<str name="f.manufacturedate_dt.facet.range.end">NOW</str>
<str name="f.manufacturedate_dt.facet.range.gap">+1YEAR</str>
<str name="f.manufacturedate_dt.facet.range.other">before</str>
<str name="f.manufacturedate_dt.facet.range.other">after</str>
<!-- Highlighting defaults -->
<str name="hl">on</str>
<str name="hl.fl">text features name</str>
<str name="f.name.hl.fragsize">0</str>
<str name="f.name.hl.alternateField">name</str>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
<!--
<str name="url-scheme">httpx</str>
-->
</requestHandler>
<!-- XML Update Request Handler.
http://wiki.apache.org/solr/UpdateXmlMessages
The canonical Request Handler for Modifying the Index through
commands specified using XML.
Note: Since solr1.1 requestHandlers requires a valid content
type header if posted in the body. For example, curl now
requires: -H 'Content-type:text/xml; charset=utf-8'
-->
<requestHandler name="/update"
class="solr.XmlUpdateRequestHandler">
<!-- See below for information on defining
updateRequestProcessorChains that can be used by name
on each Update Request
-->
<!--
<lst name="defaults">
<str name="update.chain">dedupe</str>
</lst>
-->
</requestHandler>
<!-- Binary Update Request Handler
http://wiki.apache.org/solr/javabin
-->
<requestHandler name="/update/javabin"
class="solr.BinaryUpdateRequestHandler" />
<!-- CSV Update Request Handler
http://wiki.apache.org/solr/UpdateCSV
-->
<requestHandler name="/update/csv"
class="solr.CSVRequestHandler"
startup="lazy" />
<!-- JSON Update Request Handler
http://wiki.apache.org/solr/UpdateJSON
-->
<requestHandler name="/update/json"
class="solr.JsonUpdateRequestHandler"
startup="lazy" />
<!-- Solr Cell Update Request Handler
http://wiki.apache.org/solr/ExtractingRequestHandler
-->
<requestHandler name="/update/extract"
startup="lazy"
class="solr.extraction.ExtractingRequestHandler" >
<lst name="defaults">
<!-- All the main content goes into "text"... if you need to return
the extracted text or do highlighting, use a stored field. -->
<str name="fmap.content">text</str>
<str name="lowernames">true</str>
<str name="uprefix">ignored_</str>
<!-- capture link hrefs but ignore div attributes -->
<str name="captureAttr">true</str>
<str name="fmap.a">links</str>
<str name="fmap.div">ignored_</str>
</lst>
</requestHandler>
<!-- XSLT Update Request Handler
Transforms incoming XML with stylesheet identified by tr=
-->
<requestHandler name="/update/xslt"
startup="lazy"
class="solr.XsltUpdateRequestHandler"/>
<!-- Field Analysis Request Handler
RequestHandler that provides much the same functionality as
analysis.jsp. Provides the ability to specify multiple field
types and field names in the same request and outputs
index-time and query-time analysis for each of them.
Request parameters are:
analysis.fieldname - field name whose analyzers are to be used
analysis.fieldtype - field type whose analyzers are to be used
analysis.fieldvalue - text for index-time analysis
q (or analysis.q) - text for query time analysis
analysis.showmatch (true|false) - When set to true and when
query analysis is performed, the produced tokens of the
field value analysis will be marked as "matched" for every
token that is produces by the query analysis
-->
<requestHandler name="/analysis/field"
startup="lazy"
class="solr.FieldAnalysisRequestHandler" />
<!-- Document Analysis Handler
http://wiki.apache.org/solr/AnalysisRequestHandler
An analysis handler that provides a breakdown of the analysis
process of provided docuemnts. This handler expects a (single)
content stream with the following format:
<docs>
<doc>
<field name="id">1</field>
<field name="name">The Name</field>
<field name="text">The Text Value</field>
</doc>
<doc>...</doc>
<doc>...</doc>
...
</docs>
Note: Each document must contain a field which serves as the
unique key. This key is used in the returned response to associate
an analysis breakdown to the analyzed document.
Like the FieldAnalysisRequestHandler, this handler also supports
query analysis by sending either an "analysis.query" or "q"
request parameter that holds the query text to be analyzed. It
also supports the "analysis.showmatch" parameter which when set to
true, all field tokens that match the query tokens will be marked
as a "match".
-->
<requestHandler name="/analysis/document"
class="solr.DocumentAnalysisRequestHandler"
startup="lazy" />
<!-- Admin Handlers
Admin Handlers - This will register all the standard admin
RequestHandlers.
-->
<requestHandler name="/admin/"
class="solr.admin.AdminHandlers" />
<!-- This single handler is equivalent to the following... -->
<!--
<requestHandler name="/admin/luke"
class="solr.admin.LukeRequestHandler" />
<requestHandler name="/admin/system"
class="solr.admin.SystemInfoHandler" />
<requestHandler name="/admin/plugins"
class="solr.admin.PluginInfoHandler" />
<requestHandler name="/admin/threads"
class="solr.admin.ThreadDumpHandler" />
<requestHandler name="/admin/properties"
class="solr.admin.PropertiesRequestHandler" />
<requestHandler name="/admin/file"
class="solr.admin.ShowFileRequestHandler" >
-->
<!-- If you wish to hide files under ${solr.home}/conf, explicitly
register the ShowFileRequestHandler using:
-->
<!--
<requestHandler name="/admin/file"
class="solr.admin.ShowFileRequestHandler" >
<lst name="invariants">
<str name="hidden">synonyms.txt</str>
<str name="hidden">anotherfile.txt</str>
</lst>
</requestHandler>
-->
<!-- ping/healthcheck -->
<requestHandler name="/admin/ping" class="solr.PingRequestHandler">
<lst name="invariants">
<str name="qt">search</str>
<str name="q">solrpingquery</str>
</lst>
<lst name="defaults">
<str name="echoParams">all</str>
</lst>
</requestHandler>
<!-- Echo the request contents back to the client -->
<requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
<lst name="defaults">
<str name="echoParams">explicit</str>
<str name="echoHandler">true</str>
</lst>
</requestHandler>
<!-- Solr Replication
The SolrReplicationHandler supports replicating indexes from a
"master" used for indexing and "salves" used for queries.
http://wiki.apache.org/solr/SolrReplication
In the example below, remove the <lst name="master"> section if
this is just a slave and remove the <lst name="slave"> section
if this is just a master.
-->
<!--
<requestHandler name="/replication" class="solr.ReplicationHandler" >
<lst name="master">
<str name="replicateAfter">commit</str>
<str name="replicateAfter">startup</str>
<str name="confFiles">schema.xml,stopwords.txt</str>
</lst>
<lst name="slave">
<str name="masterUrl">http://localhost:8983/solr/replication</str>
<str name="pollInterval">00:00:60</str>
</lst>
</requestHandler>
-->
<!-- Search Components
Search components are registered to SolrCore and used by
instances of SearchHandler (which can access them by name)
By default, the following components are available:
<searchComponent name="query" class="solr.QueryComponent" />
<searchComponent name="facet" class="solr.FacetComponent" />
<searchComponent name="mlt" class="solr.MoreLikeThisComponent"
/>
<searchComponent name="highlight" class="solr.HighlightComponent" />
<searchComponent name="stats" class="solr.StatsComponent" />
<searchComponent name="debug" class="solr.DebugComponent" />
Default configuration in a requestHandler would look like:
<arr name="components">
<str>query</str>
<str>facet</str>
<str>mlt</str>
<str>highlight</str>
<str>stats</str>
<str>debug</str>
</arr>
If you register a searchComponent to one of the standard names,
that will be used instead of the default.
To insert components before or after the 'standard' components, use:
<arr name="first-components">
<str>myFirstComponentName</str>
</arr>
<arr name="last-components">
<str>myLastComponentName</str>
</arr>
NOTE: The component registered with the name "debug" will
always be executed after the "last-components"
-->
<!-- Spell Check
The spell check component can return a list of alternative spelling
suggestions.
http://wiki.apache.org/solr/SpellCheckComponent
-->
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<str name="queryAnalyzerFieldType">textSpell</str>
<!-- Multiple "Spell Checkers" can be declared and used by this
component
-->
<!-- a spellchecker built from a field of the main index, and
written to disk
-->
<lst name="spellchecker">
<str name="name">default</str>
<str name="field">name</str>
<str name="spellcheckIndexDir">spellchecker</str>
<!-- uncomment this to require terms to occur in 1% of the documents
in order to be included in the dictionary
<float name="thresholdTokenFrequency">.01</float>
-->
</lst>
<!-- a spellchecker that uses a different distance measure -->
<!--
<lst name="spellchecker">
<str name="name">jarowinkler</str>
<str name="field">spell</str>
<str name="distanceMeasure">
org.apache.lucene.search.spell.JaroWinklerDistance
</str>
<str name="spellcheckIndexDir">spellcheckerJaro</str>
</lst>
-->
<!-- a spellchecker that use an alternate comparator
comparatorClass be one of:
1. score (default)
2. freq (Frequency first, then score)
3. A fully qualified class name
-->
<!--
<lst name="spellchecker">
<str name="name">freq</str>
<str name="field">lowerfilt</str>
<str name="spellcheckIndexDir">spellcheckerFreq</str>
<str name="comparatorClass">freq</str>
<str name="buildOnCommit">true</str>
-->
<!-- A spellchecker that reads the list of words from a file -->
<!--
<lst name="spellchecker">
<str name="classname">solr.FileBasedSpellChecker</str>
<str name="name">file</str>
<str name="sourceLocation">spellings.txt</str>
<str name="characterEncoding">UTF-8</str>
<str name="spellcheckIndexDir">spellcheckerFile</str>
</lst>
-->
</searchComponent>
<!-- A request handler for demonstrating the spellcheck component.
NOTE: This is purely as an example. The whole purpose of the
SpellCheckComponent is to hook it into the request handler that
handles your normal user queries so that a separate request is
not needed to get suggestions.
IN OTHER WORDS, THERE IS REALLY GOOD CHANCE THE SETUP BELOW IS
NOT WHAT YOU WANT FOR YOUR PRODUCTION SYSTEM!
See http://wiki.apache.org/solr/SpellCheckComponent for details
on the request parameters.
-->
<requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="spellcheck.onlyMorePopular">false</str>
<str name="spellcheck.extendedResults">false</str>
<str name="spellcheck.count">1</str>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
</requestHandler>
<!-- Term Vector Component
http://wiki.apache.org/solr/TermVectorComponent
-->
<searchComponent name="tvComponent" class="solr.TermVectorComponent"/>
<!-- A request handler for demonstrating the term vector component
This is purely as an example.
In reality you will likely want to add the component to your
already specified request handlers.
-->
<requestHandler name="tvrh" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<bool name="tv">true</bool>
</lst>
<arr name="last-components">
<str>tvComponent</str>
</arr>
</requestHandler>
<!-- Clustering Component
http://wiki.apache.org/solr/ClusteringComponent
This relies on third party jars which are notincluded in the
release. To use this component (and the "/clustering" handler)
Those jars will need to be downloaded, and you'll need to set
the solr.cluster.enabled system property when running solr...
java -Dsolr.clustering.enabled=true -jar start.jar
-->
<searchComponent name="clustering"
enable="${solr.clustering.enabled:false}"
class="solr.clustering.ClusteringComponent" >
<!-- Declare an engine -->
<lst name="engine">
<!-- The name, only one can be named "default" -->
<str name="name">default</str>
<!-- Class name of Carrot2 clustering algorithm.
Currently available algorithms are:
* org.carrot2.clustering.lingo.LingoClusteringAlgorithm
* org.carrot2.clustering.stc.STCClusteringAlgorithm
*
org.carrot2.clustering.kmeans.BisectingKMeansClusteringAlgorithm
See http://project.carrot2.org/algorithms.html for the
algorithm's characteristics.
-->
<str
name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm</str>
<!-- Overriding values for Carrot2 default algorithm attributes.
For a description of all available attributes, see:
http://download.carrot2.org/stable/manual/#chapter.components.
Use attribute key as name attribute of str elements
below. These can be further overridden for individual
requests by specifying attribute key as request parameter
name and attribute value as parameter value.
-->
<str name="LingoClusteringAlgorithm.desiredClusterCountBase">20</str>
<!-- Location of Carrot2 lexical resources.
A directory from which to load Carrot2-specific stop words
and stop labels. Absolute or relative to Solr config directory.
If a specific resource (e.g. stopwords.en) is present in the
specified dir, it will completely override the corresponding
default one that ships with Carrot2.
For an overview of Carrot2 lexical resources, see:
http://download.carrot2.org/head/manual/#chapter.lexical-resources
-->
<str name="carrot.lexicalResourcesDir">clustering/carrot2</str>
<!-- The language to assume for the documents.
For a list of allowed values, see:
http://download.carrot2.org/stable/manual/#section.attribute.lingo.MultilingualClustering.defaultLanguage
-->
<str name="MultilingualClustering.defaultLanguage">ENGLISH</str>
</lst>
<lst name="engine">
<str name="name">stc</str>
<str
name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm</str>
</lst>
</searchComponent>
<!-- A request handler for demonstrating the clustering component
This is purely as an example.
In reality you will likely want to add the component to your
already specified request handlers.
-->
<requestHandler name="/clustering"
startup="lazy"
enable="${solr.clustering.enabled:false}"
class="solr.SearchHandler">
<lst name="defaults">
<bool name="clustering">true</bool>
<str name="clustering.engine">default</str>
<bool name="clustering.results">true</bool>
<!-- The title field -->
<str name="carrot.title">name</str>
<str name="carrot.url">id</str>
<!-- The field to cluster on -->
<str name="carrot.snippet">features</str>
<!-- produce summaries -->
<bool name="carrot.produceSummary">true</bool>
<!-- the maximum number of labels per cluster -->
<!--<int name="carrot.numDescriptions">5</int>-->
<!-- produce sub clusters -->
<bool name="carrot.outputSubClusters">false</bool>
<str name="defType">edismax</str>
<str name="qf">
text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
</str>
<str name="q.alt">*:*</str>
<str name="rows">10</str>
<str name="fl">*,score</str>
</lst>
<arr name="last-components">
<str>clustering</str>
</arr>
</requestHandler>
<!-- Terms Component
http://wiki.apache.org/solr/TermsComponent
A component to return terms and document frequency of those
terms
-->
<searchComponent name="terms" class="solr.TermsComponent"/>
<!-- A request handler for demonstrating the terms component -->
<requestHandler name="/terms" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<bool name="terms">true</bool>
</lst>
<arr name="components">
<str>terms</str>
</arr>
</requestHandler>
<!-- Query Elevation Component
http://wiki.apache.org/solr/QueryElevationComponent
a search component that enables you to configure the top
results for a given query regardless of the normal lucene
scoring.
-->
<!--
<searchComponent name="elevator" class="solr.QueryElevationComponent" >
<str name="queryFieldType">string</str>
<str name="config-file">elevate.xml</str>
</searchComponent>
-->
<!-- A request handler for demonstrating the elevator component -->
<requestHandler name="/elevate" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="echoParams">explicit</str>
</lst>
<arr name="last-components">
<str>elevator</str>
</arr>
</requestHandler>
<!-- Highlighting Component
http://wiki.apache.org/solr/HighlightingParameters
-->
<searchComponent class="solr.HighlightComponent" name="highlight">
<highlighting>
<!-- Configure the standard fragmenter -->
<!-- This could most likely be commented out in the "default" case -->
<fragmenter name="gap"
default="true"
class="solr.highlight.GapFragmenter">
<lst name="defaults">
<int name="hl.fragsize">100</int>
</lst>
</fragmenter>
<!-- A regular-expression-based fragmenter
(for sentence extraction)
-->
<fragmenter name="regex"
class="solr.highlight.RegexFragmenter">
<lst name="defaults">
<!-- slightly smaller fragsizes work better because of slop -->
<int name="hl.fragsize">70</int>
<!-- allow 50% slop on fragment sizes -->
<float name="hl.regex.slop">0.5</float>
<!-- a basic sentence pattern -->
<str name="hl.regex.pattern">[-\w ,/\n\"']{20,200}</str>
</lst>
</fragmenter>
<!-- Configure the standard formatter -->
<formatter name="html"
default="true"
class="solr.highlight.HtmlFormatter">
<lst name="defaults">
<str name="hl.simple.pre"><![CDATA[<em>]]></str>
<str name="hl.simple.post"><![CDATA[</em>]]></str>
</lst>
</formatter>
<!-- Configure the standard encoder -->
<encoder name="html"
class="solr.highlight.HtmlEncoder" />
<!-- Configure the standard fragListBuilder -->
<fragListBuilder name="simple"
default="true"
class="solr.highlight.SimpleFragListBuilder"/>
<!-- Configure the single fragListBuilder -->
<fragListBuilder name="single"
class="solr.highlight.SingleFragListBuilder"/>
<!-- default tag FragmentsBuilder -->
<fragmentsBuilder name="default"
default="true"
class="solr.highlight.ScoreOrderFragmentsBuilder">
<!--
<lst name="defaults">
<str name="hl.multiValuedSeparatorChar">/</str>
</lst>
-->
</fragmentsBuilder>
<!-- multi-colored tag FragmentsBuilder -->
<fragmentsBuilder name="colored"
class="solr.highlight.ScoreOrderFragmentsBuilder">
<lst name="defaults">
<str name="hl.tag.pre"><![CDATA[
<b style="background:yellow">,<b
style="background:lawgreen">,
<b style="background:aquamarine">,<b
style="background:magenta">,
<b style="background:palegreen">,<b
style="background:coral">,
<b style="background:wheat">,<b style="background:khaki">,
<b style="background:lime">,<b
style="background:deepskyblue">]]></str>
<str name="hl.tag.post"><![CDATA[</b>]]></str>
</lst>
</fragmentsBuilder>
<boundaryScanner name="default"
default="true"
class="solr.highlight.SimpleBoundaryScanner">
<lst name="defaults">
<str name="hl.bs.maxScan">10</str>
<str name="hl.bs.chars">.,!? 	 </str>
</lst>
</boundaryScanner>
<boundaryScanner name="breakIterator"
class="solr.highlight.BreakIteratorBoundaryScanner">
<lst name="defaults">
<!-- type should be one of CHARACTER, WORD(default), LINE and
SENTENCE -->
<str name="hl.bs.type">WORD</str>
<!-- language and country are used when constructing Locale
object. -->
<!-- And the Locale object will be used when getting instance of
BreakIterator -->
<str name="hl.bs.language">en</str>
<str name="hl.bs.country">US</str>
</lst>
</boundaryScanner>
</highlighting>
</searchComponent>
<!-- Update Processors
Chains of Update Processor Factories for dealing with Update
Requests can be declared, and then used by name in Update
Request Processors
http://wiki.apache.org/solr/UpdateRequestProcessor
-->
<!-- Deduplication
An example dedup update processor that creates the "id" field
on the fly based on the hash code of some other fields. This
example has overwriteDupes set to false since we are using the
id field as the signatureField and Solr will maintain
uniqueness based on that anyway.
-->
<!--
<updateRequestProcessorChain name="dedupe">
<processor class="solr.processor.SignatureUpdateProcessorFactory">
<bool name="enabled">true</bool>
<str name="signatureField">id</str>
<bool name="overwriteDupes">false</bool>
<str name="fields">name,features,cat</str>
<str name="signatureClass">solr.processor.Lookup3Signature</str>
</processor>
<processor class="solr.LogUpdateProcessorFactory" />
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
-->
<!--
This example update chain identifies the language of the incoming
documents using the langid contrib. The detected language is
written to field language_s. No field name mapping is done.
The fields used for detection are text, title, subject and
description,
making this example suitable for detecting languages form full-text
rich documents injected via ExtractingRequestHandler.
See more about langId at
http://wiki.apache.org/solr/LanguageDetection
-->
<!--
<updateRequestProcessorChain name="langid">
<processor
class="org.apache.solr.update.processor.TikaLanguageIdentifierUpdateProcessorFactory">
<str name="langid.fl">text,title,subject,description</str>
<str name="langid.langField">language_s</str>
<str name="langid.fallback">en</str>
</processor>
<processor class="solr.LogUpdateProcessorFactory" />
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
-->
<!-- Response Writers
http://wiki.apache.org/solr/QueryResponseWriter
Request responses will be written using the writer specified by
the 'wt' request parameter matching the name of a registered
writer.
The "default" writer is the default and will be used if 'wt' is
not specified in the request.
-->
<!-- The following response writers are implicitly configured unless
overridden...
-->
<!--
<queryResponseWriter name="xml"
default="true"
class="solr.XMLResponseWriter" />
<queryResponseWriter name="json" class="solr.JSONResponseWriter"/>
<queryResponseWriter name="python" class="solr.PythonResponseWriter"/>
<queryResponseWriter name="ruby" class="solr.RubyResponseWriter"/>
<queryResponseWriter name="php" class="solr.PHPResponseWriter"/>
<queryResponseWriter name="phps"
class="solr.PHPSerializedResponseWriter"/>
<queryResponseWriter name="csv" class="solr.CSVResponseWriter"/>
-->
<queryResponseWriter name="json" class="solr.JSONResponseWriter">
<!-- For the purposes of the tutorial, JSON responses are written as
plain text so that they are easy to read in *any* browser.
If you expect a MIME type of "application/json" just remove this
override.
-->
<str name="content-type">text/plain; charset=UTF-8</str>
</queryResponseWriter>
<!--
Custom response writers can be declared as needed...
-->
<!-- The solr.velocity.enabled flag is used by Solr's test cases so
that this response writer is not
loaded (causing an error if contrib/velocity has not been built
fully) -->
<!--
<queryResponseWriter name="velocity"
class="solr.VelocityResponseWriter" enable="${solr.velocity.enabled:true}"/>
-->
<!-- XSLT response writer transforms the XML output by any xslt file found
in Solr's conf/xslt directory. Changes to xslt files are checked for
every xsltCacheLifetimeSeconds.
-->
<queryResponseWriter name="xslt" class="solr.XSLTResponseWriter">
<int name="xsltCacheLifetimeSeconds">5</int>
</queryResponseWriter>
<!-- Query Parsers
http://wiki.apache.org/solr/SolrQuerySyntax
Multiple QParserPlugins can be registered by name, and then
used in either the "defType" param for the QueryComponent (used
by SearchHandler) or in LocalParams
-->
<!-- example of registering a query parser -->
<!--
<queryParser name="myparser" class="com.mycompany.MyQParserPlugin"/>
-->
<!-- Function Parsers
http://wiki.apache.org/solr/FunctionQuery
Multiple ValueSourceParsers can be registered by name, and then
used as function names when using the "func" QParser.
-->
<!-- example of registering a custom function parser -->
<!--
<valueSourceParser name="myfunc"
class="com.mycompany.MyValueSourceParser" />
-->
<!-- Legacy config for the admin interface -->
<admin>
<defaultQuery>*:*</defaultQuery>
<!-- configure a healthcheck file for servers behind a
loadbalancer
-->
<!--
<healthcheck type="file">server-enabled</healthcheck>
-->
</admin>
<requestHandler name="/replication" class="solr.ReplicationHandler" >
<lst name="master">
<!--Replicate on 'startup' and 'commit'. 'optimize' is also a valid
value for replicateAfter. -->
<str name="replicateAfter">startup</str>
<str name="replicateAfter">commit</str>
<!--Create a backup after 'optimize'. Other values can be 'commit',
'startup'. It is possible to have multiple entries of this config string.
Note that this is just for backup, replication does not require this. -->
<!-- <str name="backupAfter">optimize</str> -->
<!--If configuration files need to be replicated give the names
here, separated by comma -->
<str name="confFiles">schema.xml,stopwords.txt,elevate.xml</str>
<!--The default value of reservation is 10 secs.See the
documentation below . Normally , you should not need to specify this -->
<str name="commitReserveDuration">00:01:00</str>
</lst>
</requestHandler>
</config>
On Mon, Feb 13, 2012 at 12:36 PM, Li Li <fa...@gmail.com> wrote:
> can you post your config file?
> I found there are 2 places to config ramBufferSizeMB in latest svn of 3.6's
> example solrconfig.xml. trying to modify them both?
>
> <indexDefaults>
>
> <useCompoundFile>false</useCompoundFile>
>
> <mergeFactor>10</mergeFactor>
> <!-- Sets the amount of RAM that may be used by Lucene indexing
> for buffering added documents and deletions before they are
> flushed to the Directory. -->
> <ramBufferSizeMB>32</ramBufferSizeMB>
> <!-- If both ramBufferSizeMB and maxBufferedDocs is set, then
> Lucene will flush based on whichever limit is hit first.
> -->
> <!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
>
> <maxFieldLength>10000</maxFieldLength>
> <writeLockTimeout>1000</writeLockTimeout>
>
> .............
> <!-- <termIndexInterval>256</termIndexInterval> -->
> </indexDefaults>
>
> <!-- Main Index
>
> Values here override the values in the <indexDefaults> section
> for the main on disk index.
> -->
> <mainIndex>
>
> <useCompoundFile>false</useCompoundFile>
> <ramBufferSizeMB>32</ramBufferSizeMB>
> <mergeFactor>10</mergeFactor>
> ........
> </mainIndex>
>
> On Tue, Feb 14, 2012 at 1:10 AM, Huy Le <hu...@springpartners.com> wrote:
>
> > Hi,
> >
> > I am using solr 3.5. As I understood it, NRT is a solr 4 feature, but
> solr
> > 4 is not released yet.
> >
> > I understand commit after adding each document is expensive, but the
> > application requires that documents be available after adding to the
> index.
> >
> > What I don't understand is why new segment files are created so often.
> > Are the commit calls triggering new segment files being created? I don't
> > see this behavior in another environment of the same version of solr.
> >
> > Huy
> >
> > On Mon, Feb 13, 2012 at 11:55 AM, Li Li <fa...@gmail.com> wrote:
> >
> > > Commit is called
> > > after adding each document
> > >
> > >
> > > you should add enough documents and then calling a commit. commit is a
> > > cost operation.
> > > if you want to get latest feeded documents, you could use NRT
> > >
> > > On Tue, Feb 14, 2012 at 12:47 AM, Huy Le <hu...@springpartners.com>
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > I am using solr 3.5. I seeing solr keeps creating new segment files
> > > (<1MB
> > > > files) so often that it triggers segment merge about every one
> minute.
> > I
> > > > search the news archive, but could not find any info on this issue.
> I
> > am
> > > > indexing about 10 docs of less 2KB each every second. Commit is
> called
> > > > after adding each document. Relevant config params are:
> > > >
> > > > <mergeFactor>10</mergeFactor>
> > > > <ramBufferSizeMB>1024</ramBufferSizeMB>
> > > > <maxMergeDocs>2147483647</maxMergeDocs>
> > > >
> > > > What might be triggering this frequent new segment files creation?
> > > Thanks!
> > > >
> > > > Huy
> > > >
> > > > --
> > > > Huy Le
> > > > Spring Partners, Inc.
> > > > http://springpadit.com
> > > >
> > >
> >
> >
> >
> > --
> > Huy Le
> > Spring Partners, Inc.
> > http://springpadit.com
> >
>
--
Huy Le
Spring Partners, Inc.
http://springpadit.com
Re: New segment file created too often
Posted by Li Li <fa...@gmail.com>.
can you post your config file?
I found there are 2 places to config ramBufferSizeMB in latest svn of 3.6's
example solrconfig.xml. trying to modify them both?
<indexDefaults>
<useCompoundFile>false</useCompoundFile>
<mergeFactor>10</mergeFactor>
<!-- Sets the amount of RAM that may be used by Lucene indexing
for buffering added documents and deletions before they are
flushed to the Directory. -->
<ramBufferSizeMB>32</ramBufferSizeMB>
<!-- If both ramBufferSizeMB and maxBufferedDocs is set, then
Lucene will flush based on whichever limit is hit first.
-->
<!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
<maxFieldLength>10000</maxFieldLength>
<writeLockTimeout>1000</writeLockTimeout>
.............
<!-- <termIndexInterval>256</termIndexInterval> -->
</indexDefaults>
<!-- Main Index
Values here override the values in the <indexDefaults> section
for the main on disk index.
-->
<mainIndex>
<useCompoundFile>false</useCompoundFile>
<ramBufferSizeMB>32</ramBufferSizeMB>
<mergeFactor>10</mergeFactor>
........
</mainIndex>
On Tue, Feb 14, 2012 at 1:10 AM, Huy Le <hu...@springpartners.com> wrote:
> Hi,
>
> I am using solr 3.5. As I understood it, NRT is a solr 4 feature, but solr
> 4 is not released yet.
>
> I understand commit after adding each document is expensive, but the
> application requires that documents be available after adding to the index.
>
> What I don't understand is why new segment files are created so often.
> Are the commit calls triggering new segment files being created? I don't
> see this behavior in another environment of the same version of solr.
>
> Huy
>
> On Mon, Feb 13, 2012 at 11:55 AM, Li Li <fa...@gmail.com> wrote:
>
> > Commit is called
> > after adding each document
> >
> >
> > you should add enough documents and then calling a commit. commit is a
> > cost operation.
> > if you want to get latest feeded documents, you could use NRT
> >
> > On Tue, Feb 14, 2012 at 12:47 AM, Huy Le <hu...@springpartners.com>
> wrote:
> >
> > > Hi,
> > >
> > > I am using solr 3.5. I seeing solr keeps creating new segment files
> > (<1MB
> > > files) so often that it triggers segment merge about every one minute.
> I
> > > search the news archive, but could not find any info on this issue. I
> am
> > > indexing about 10 docs of less 2KB each every second. Commit is called
> > > after adding each document. Relevant config params are:
> > >
> > > <mergeFactor>10</mergeFactor>
> > > <ramBufferSizeMB>1024</ramBufferSizeMB>
> > > <maxMergeDocs>2147483647</maxMergeDocs>
> > >
> > > What might be triggering this frequent new segment files creation?
> > Thanks!
> > >
> > > Huy
> > >
> > > --
> > > Huy Le
> > > Spring Partners, Inc.
> > > http://springpadit.com
> > >
> >
>
>
>
> --
> Huy Le
> Spring Partners, Inc.
> http://springpadit.com
>
Re: New segment file created too often
Posted by Li Li <fa...@gmail.com>.
as far as I know, there are three situation it will be flushed to a new
segment: RAM buffer for posting data structure is used up; added doc
numbers are exceeding threshold and there are many deletions in a segment
but your configuration seems it is not likely to flush many small segments.
<ramBufferSizeMB>1024</ramBufferSizeMB>
<maxMergeDocs>2147483647</maxMergeDocs>
On Tue, Feb 14, 2012 at 1:10 AM, Huy Le <hu...@springpartners.com> wrote:
> Hi,
>
> I am using solr 3.5. As I understood it, NRT is a solr 4 feature, but solr
> 4 is not released yet.
>
> I understand commit after adding each document is expensive, but the
> application requires that documents be available after adding to the index.
>
> What I don't understand is why new segment files are created so often.
> Are the commit calls triggering new segment files being created? I don't
> see this behavior in another environment of the same version of solr.
>
> Huy
>
> On Mon, Feb 13, 2012 at 11:55 AM, Li Li <fa...@gmail.com> wrote:
>
> > Commit is called
> > after adding each document
> >
> >
> > you should add enough documents and then calling a commit. commit is a
> > cost operation.
> > if you want to get latest feeded documents, you could use NRT
> >
> > On Tue, Feb 14, 2012 at 12:47 AM, Huy Le <hu...@springpartners.com>
> wrote:
> >
> > > Hi,
> > >
> > > I am using solr 3.5. I seeing solr keeps creating new segment files
> > (<1MB
> > > files) so often that it triggers segment merge about every one minute.
> I
> > > search the news archive, but could not find any info on this issue. I
> am
> > > indexing about 10 docs of less 2KB each every second. Commit is called
> > > after adding each document. Relevant config params are:
> > >
> > > <mergeFactor>10</mergeFactor>
> > > <ramBufferSizeMB>1024</ramBufferSizeMB>
> > > <maxMergeDocs>2147483647</maxMergeDocs>
> > >
> > > What might be triggering this frequent new segment files creation?
> > Thanks!
> > >
> > > Huy
> > >
> > > --
> > > Huy Le
> > > Spring Partners, Inc.
> > > http://springpadit.com
> > >
> >
>
>
>
> --
> Huy Le
> Spring Partners, Inc.
> http://springpadit.com
>
Re: New segment file created too often
Posted by Huy Le <hu...@springpartners.com>.
Hi,
I am using solr 3.5. As I understood it, NRT is a solr 4 feature, but solr
4 is not released yet.
I understand commit after adding each document is expensive, but the
application requires that documents be available after adding to the index.
What I don't understand is why new segment files are created so often.
Are the commit calls triggering new segment files being created? I don't
see this behavior in another environment of the same version of solr.
Huy
On Mon, Feb 13, 2012 at 11:55 AM, Li Li <fa...@gmail.com> wrote:
> Commit is called
> after adding each document
>
>
> you should add enough documents and then calling a commit. commit is a
> cost operation.
> if you want to get latest feeded documents, you could use NRT
>
> On Tue, Feb 14, 2012 at 12:47 AM, Huy Le <hu...@springpartners.com> wrote:
>
> > Hi,
> >
> > I am using solr 3.5. I seeing solr keeps creating new segment files
> (<1MB
> > files) so often that it triggers segment merge about every one minute. I
> > search the news archive, but could not find any info on this issue. I am
> > indexing about 10 docs of less 2KB each every second. Commit is called
> > after adding each document. Relevant config params are:
> >
> > <mergeFactor>10</mergeFactor>
> > <ramBufferSizeMB>1024</ramBufferSizeMB>
> > <maxMergeDocs>2147483647</maxMergeDocs>
> >
> > What might be triggering this frequent new segment files creation?
> Thanks!
> >
> > Huy
> >
> > --
> > Huy Le
> > Spring Partners, Inc.
> > http://springpadit.com
> >
>
--
Huy Le
Spring Partners, Inc.
http://springpadit.com
Re: New segment file created too often
Posted by Li Li <fa...@gmail.com>.
Commit is called
after adding each document
you should add enough documents and then calling a commit. commit is a
cost operation.
if you want to get latest feeded documents, you could use NRT
On Tue, Feb 14, 2012 at 12:47 AM, Huy Le <hu...@springpartners.com> wrote:
> Hi,
>
> I am using solr 3.5. I seeing solr keeps creating new segment files (<1MB
> files) so often that it triggers segment merge about every one minute. I
> search the news archive, but could not find any info on this issue. I am
> indexing about 10 docs of less 2KB each every second. Commit is called
> after adding each document. Relevant config params are:
>
> <mergeFactor>10</mergeFactor>
> <ramBufferSizeMB>1024</ramBufferSizeMB>
> <maxMergeDocs>2147483647</maxMergeDocs>
>
> What might be triggering this frequent new segment files creation? Thanks!
>
> Huy
>
> --
> Huy Le
> Spring Partners, Inc.
> http://springpadit.com
>