You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by bu...@apache.org on 2013/07/23 18:09:28 UTC
svn commit: r870817 [2/3] - in /websites/staging/lucene/trunk/content: ./
core/ openrelevance/ solr/
Modified: websites/staging/lucene/trunk/content/mainnews.html
==============================================================================
--- websites/staging/lucene/trunk/content/mainnews.html (original)
+++ websites/staging/lucene/trunk/content/mainnews.html Tue Jul 23 16:09:27 2013
@@ -193,7 +193,155 @@
<h1 class="title"></h1>
<div><h1 id="lucenewzxhzdk0tmwzxhzdk1-news">Lucene<span style="vertical-align: super; font-size: xx-small">TM</span> News</h1>
-<h2 id="18-june-2013-apache-lucene-431-and-apache-solrwzxhzdk2tmwzxhzdk3-431-available">18 June 2013 - Apache Lucene 4.3.1 and Apache Solr<span style="vertical-align: super; font-size: xx-small">TM</span> 4.3.1 available</h2>
+<h2 id="23-july-2013-apache-lucene-44-and-apache-solrwzxhzdk2tmwzxhzdk3-44-available">23 July 2013 - Apache Lucene 4.4 and Apache Solr<span style="vertical-align: super; font-size: xx-small">TM</span> 4.4 available</h2>
+<p>The Lucene PMC is pleased to announce the availability
+of Apache Lucene 4.4 and Apache Solr 4.4.</p>
+<p>Lucene can be downloaded from <a href="http://lucene.apache.org/core/mirrors-core-latest-redir.html">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
+and Solr can be downloaded from <a href="http://lucene.apache.org/solr/mirrors-solr-latest-redir.html">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
+<p>See the <a href="/core/4_4_0/changes/Changes.html">Lucene CHANGES.txt</a> and
+<a href="/solr/4_4_0/changes/Changes.html">Solr CHANGES.txt</a> files included
+with the release for a full list of details.</p>
+<h3 id="highlights-of-the-lucene-release-include">Highlights of the Lucene release include:</h3>
+<ul>
+<li>
+<p>New Replicator module: replicate index revisions between server and
+ client. See http://shaierera.blogspot.com/2013/05/the-replicator.html</p>
+</li>
+<li>
+<p>New AnalyzingInfixSuggester: finds suggestions based on matches to any
+ tokens in the suggestion, not just based on pure prefix matching. See
+ http://blog.mikemccandless.com/2013/06/a-new-lucene-suggester-based-on-infix.html</p>
+</li>
+<li>
+<p>New PatternCaptureGroupTokenFilter: emit multiple tokens, one for each
+ capture group in one or more Java regexes.</p>
+</li>
+<li>
+<p>New Lucene Facet module features: </p>
+</li>
+<li>Added dynamic (no taxonomy index used) numeric range faceting (see
+ http://blog.mikemccandless.com/2013/05/dynamic-faceting-with-lucene.html )</li>
+<li>Arbitrary Querys are now allowed for per-dimension drill-down on
+ DrillDownQuery and DrillSideways, to support future dynamic faceting.</li>
+<li>
+<p>New FacetResult.mergeHierarchies: merge multiple FacetResult of the
+ same dimension into a single one with the reconstructed hierarchy.</p>
+</li>
+<li>
+<p>FST's Builder can now handle more than 2.1 billion "tail nodes" while
+ building a minimal FST.</p>
+</li>
+<li>
+<p>FieldCache Ints and Longs now use bit-packing to save memory. String fields
+ have more efficient compression if there are many unique terms.</p>
+</li>
+<li>
+<p>Improved compression for NumericDocValues for dates and fields with very
+ small numbers of unique values.</p>
+</li>
+<li>
+<p>New IndexWriter.hasUncommittedChanges(): returns true if there are changes
+ that have not been committed.</p>
+</li>
+<li>
+<p>multiValuedSeparator in PostingsHighlighter is now configurable, for cases
+ where you want a different logical separator between field values.</p>
+</li>
+<li>
+<p>NorwegianLightStemFilter and NorwegianMinimalStemFilter have been extended
+ to handle "nynorsk".</p>
+</li>
+<li>
+<p>New ScandinavianFoldingFilter and ScandinavianNormalizationFilter.</p>
+</li>
+<li>
+<p>Easier compressed norms: Lucene42NormsFormat now takes an overhead
+ parameter, allowing for values other than PackedInts.FASTEST.</p>
+</li>
+<li>
+<p>Analyzer now has an additional tokenStream(String fieldName, String text)
+ method, so wrapping by StringReader for common use is no longer needed.</p>
+</li>
+<li>
+<p>New SimpleMergedSegmentWarmer: just ensures that data structures
+ (terms, norms, docvalues, etc.) are initialized.</p>
+</li>
+<li>
+<p>IndexWriter flushes segments to the compound file format by default.</p>
+</li>
+</ul>
+<p>Lucene 4.4 also includes numerous optimizations and bugfixes.</p>
+<h3 id="highlights-of-the-solr-release-include">Highlights of the Solr release include:</h3>
+<ul>
+<li>
+<p>Solr indexes and transaction logs may stored in HDFS with full read/write
+ capability.</p>
+</li>
+<li>
+<p>Schemaless mode: Added support for a mode that requires no up-front schema
+ modifications, in which previously unknown fields' types are guessed based
+ on the values in added/updated documents, and are then added to the schema
+ prior to processing the update. Note that the below-described features
+ are also useful independently from schemaless mode operation. </p>
+</li>
+<li>New Parse{Date,Integer,Long,Float,Double,Boolean}UpdateProcessorFactory
+ classes parse/guess the field value class for String-valued and unknown
+ fields.</li>
+<li>New AddSchemaFieldsUpdateProcessor: Automatically add new fields to the
+ schema when adding/updating documents with unknown fields. Custom rules
+ map field value class(es) to schema fieldTypes.</li>
+<li>
+<p>A new schemaless mode example configuration, using the above-described
+ field-value-class-guessing and unknown-field-schema-addition features,
+ is provided at solr/example/example-schemaless/.</p>
+</li>
+<li>
+<p>Core Discovery mode: A new solr.xml format which does not store core
+ information, but instead searches for files named 'core.properties' in
+ the filesystem which tell Solr all the details about that core. The main
+ example and the schemaless example both use this new format.</p>
+</li>
+<li>
+<p>Schema REST API: Add support for creating copy fields.</p>
+</li>
+<li>
+<p>A merged segment warmer may now be plugged into solrconfig.xml. </p>
+</li>
+<li>
+<p>New MaxScoreQParserPlugin: Return max() instead of sum() of terms.</p>
+</li>
+<li>
+<p>Binary files are now supported in ZooKeeper.</p>
+</li>
+<li>
+<p>SolrJ's SolrPing object has new methods for ping, enable, and disable.</p>
+</li>
+<li>
+<p>The Admin UI now supports adding documents to Solr.</p>
+</li>
+<li>
+<p>Added a PUT command to the Solr ZkCli tool.</p>
+</li>
+<li>
+<p>New "deleteshard" collections API that unloads all replicas of a given
+ shard and then removes it from the cluster state. It will remove only
+ those shards which are INACTIVE or have no range.</p>
+</li>
+<li>
+<p>The Overseer can now optionally assign generic node names so that
+ new addresses can host shards without naming confusion.</p>
+</li>
+<li>
+<p>The CSV Update Handler now supports optionally adding the line number/
+ row id to a document.</p>
+</li>
+<li>
+<p>Added a new system wide info admin handler that exposes the system info
+ that could previously only be retrieved using a SolrCore.</p>
+</li>
+</ul>
+<p>Solr 4.4 also includes numerous optimizations and bugfixes.</p>
+<h2 id="18-june-2013-apache-lucene-431-and-apache-solr-431-available">18 June 2013 - Apache Lucene 4.3.1 and Apache Solr 4.3.1 available</h2>
<p>The Lucene PMC is pleased to announce the availability
of Apache Lucene 4.3.1 and Apache Solr 4.3.1.</p>
<p>Lucene can be downloaded from <a href="http://lucene.apache.org/core/mirrors-core-latest-redir.html">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
@@ -201,12 +349,12 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_3_1/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_3_1/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_1">Highlights of the Lucene release include:</h3>
<ul>
<li>Lucene 4.3.1 includes 12 bug fixes and 1 optimization, including fixes
for a serious bug that can cause deadlock.</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_1">Highlights of the Solr release include:</h3>
<ul>
<li>
<p>Solr 4.3.1 includes 24 bug fixes. The list includes a lot of SolrCloud
@@ -224,7 +372,7 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_3_0/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_3_0/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include_1">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_2">Highlights of the Lucene release include:</h3>
<ul>
<li>
<p>Significant performance improvements for minShouldMatch BooleanQuery due to
@@ -271,7 +419,7 @@ with the release for a full list of deta
<p>Various bugfixes and optimizations since the 4.2.1 release.</p>
</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include_1">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_2">Highlights of the Solr release include:</h3>
<ul>
<li>
<p>Tired of maintaining core information in solr.xml? Now you can configure
@@ -319,12 +467,12 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_2_1/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_2_1/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include_2">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_3">Highlights of the Lucene release include:</h3>
<ul>
<li>Lucene 4.2.1 includes 9 bug fixes and 3 optimizations, including a fix
for a serious bug that could result in the loss of an index.</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include_2">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_3">Highlights of the Solr release include:</h3>
<ul>
<li>
<p>Solr 4.2.1 includes 38 bug fixes and 2 optimizations. The list includes
@@ -344,7 +492,7 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_2_0/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_2_0/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include_3">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_4">Highlights of the Lucene release include:</h3>
<ul>
<li>
<p>Lucene 4.2 has a new default codec (Lucene42Codec) with a more
@@ -393,7 +541,7 @@ with the release for a full list of deta
<p>Various bugfixes and optimizations since the 4.1 release.</p>
</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include_3">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_4">Highlights of the Solr release include:</h3>
<ul>
<li>
<p>A read side REST API for the schema. Always wanted to introspect the schema over http?
@@ -440,7 +588,7 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_1_0/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_1_0/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include_4">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_5">Highlights of the Lucene release include:</h3>
<ul>
<li>
<p>Lucene 4.1 has a new default codec (Lucene41Codec) based on the
@@ -495,7 +643,7 @@ with the release for a full list of deta
<p>Several bugfixes and optimizations since the 4.0 release.</p>
</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include_4">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_5">Highlights of the Solr release include:</h3>
<h4 id="solrcloud-enhancements-see-httpwikiapacheorgsolrsolrcloud">SolrCloud enhancements (see <a href="http://wiki.apache.org/solr/SolrCloud">http://wiki.apache.org/solr/SolrCloud</a>):</h4>
<ul>
<li>Simple multi-tenancy through enhanced document routing:<ul>
@@ -656,319 +804,6 @@ details.</p>
<li>Various Admin UI improvements.</li>
<li>SolrCloud related performance optimizations in writing the the transaction log, PeerSync recovery, Leader election, and ClusterState caching.</li>
<li>Numerous bug fixes and optimizations.</li>
-</ul>
-<h2 id="14-august-2012-lucene-core-40-beta-and-solr-40-beta-available">14 August 2012 - Lucene Core 4.0-BETA and Solr 4.0-BETA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-BETA and Apache Solr 4.0-BETA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p><a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/index/IndexWriter.html#tryDeleteDocument%28org.apache.lucene.index.IndexReader,%20int%29">
- IndexWriter.tryDeleteDocument</a> can sometimes delete by document ID,
- for higher performance in some applications.</p>
-</li>
-<li>
-<p>New experimental postings formats: <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/bloom/BloomFilteringPostingsFormat.html">
- BloomFilteringPostingsFormat</a> uses a bloom filter to sometimes avoid
- disk seeks when looking up terms,
- <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/memory/DirectPostingsFormat.html">
- DirectPostingsFormat</a> holds all postings as simple byte[] and int[]
- for very fast performance at the cost of very high RAM consumption.</p>
-</li>
-<li>
-<p>CJK analysis improvements: <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-kuromoji/org/apache/lucene/analysis/ja/JapaneseIterationMarkCharFilter.html">
- JapaneseIterationMarkCharFilter</a> normalizes Japanese iteration marks,
- added unigram+bigram support to <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/cjk/CJKBigramFilter.html">
- CJKBigramFilter</a>.</p>
-</li>
-<li>
-<p>Improvements to Scorer navigation API (<a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/search/Scorer.html#getChildren%28%29">
- Scorer.getChildren</a>) to support all queries, useful for determining
- which portions of the query matched.</p>
-</li>
-<li>
-<p>Analysis improvements: factories for creating <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenizerFactory.html">
- Tokenizer</a>, <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenFilterFactory.html">
- TokenFilter</a>, and <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/CharFilterFactory.html">
- CharFilter</a> have been moved from Solr to Lucene's analysis module,
- less memory overhead for StandardTokenizer and Snowball filters.</p>
-</li>
-<li>
-<p>Improved highlighting for multi-valued fields.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<ul>
-<li>
-<p>Added a Collection management API for <a href="http://wiki.apache.org/solr/SolrCloud/">Solr Cloud</a>.</p>
-</li>
-<li>
-<p>Solr Admin UI now clearly displays failures related to initializing SolrCores</p>
-</li>
-<li>
-<p>Updatable documents can create a document if it doesn't already exist,
- or you can force that the document must already exist.</p>
-</li>
-<li>
-<p>Full delete-by-query support for Solr Cloud.</p>
-</li>
-<li>
-<p>Default to <a href="http://lucene.apache.org/solr/api-4_0_0-BETA/org/apache/solr/core/NRTCachingDirectoryFactory.html">
- NRTCachingDirectory</a> for improved near-realtime performance.</p>
-</li>
-<li>
-<p>Improved <a href="http://wiki.apache.org/solr/Solrj">Solrj</a> client performance
- with Solr Cloud: updates are only sent to leaders by default.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<h2 id="22-july-2012-apache-lucene-361-and-apache-solr-361-available">22 July 2012 - Apache Lucene 3.6.1 and Apache Solr 3.6.1 available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 3.6.1 and Apache Solr 3.6.1.</p>
-<p>This release is a bug fix release for version 3.6.0. It contains numerous
-bug fixes, optimizations, and improvements, some of which are highlighted
-below.</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-3x-redir.html?">http://lucene.apache.org/core/mirrors-core-3x-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-3x-redir.html?">http://lucene.apache.org/solr/mirrors-solr-3x-redir.html</a></p>
-<p>See the CHANGES.txt file included with the release for a full list of
-details.</p>
-<p>Lucene 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapIndexInput.clone() was improved, which caused
- a performance regression in comparison to Lucene 3.5.0.</p>
-</li>
-<li>
-<p>MappingCharFilter was fixed to return correct final token positions.</p>
-</li>
-<li>
-<p>QueryParser now supports +/- operators with any amount of whitespace.</p>
-</li>
-<li>
-<p>DisjunctionMaxScorer now implements visitSubScorers().</p>
-</li>
-<li>
-<p>Changed the visibility of Scorer#visitSubScorers() to
- public, otherwise it's impossible to implement Scorers outside
- the Lucene package. This is a small backwards break, affecting a few
- users who implemented custom Scorers.</p>
-</li>
-<li>
-<p>Various analyzer bugs where fixed: Kuromoji to not produce invalid
- token graph due to UNK with punctuation being decompounded, invalid
- position length in SynonymFilter, loading of Hunspell dictionaries that
- use aliasing, be consistent with closing streams when loading
- Hunspell affix files.</p>
-</li>
-<li>
-<p>Various bugs in FST components were fixed: Offline sorter minimum
- buffer size, integer overflow in sorter, FSTCompletionLookup missed
- to close its sorter.</p>
-</li>
-<li>
-<p>Fixed a synchronization bug in handling taxonomies in facet module.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed: BytesRef/CharsRef copy methods
- with nonzero offsets and subSequence off-by-one, TieredMergePolicy
- returned wrong-scaled floor segment setting.</p>
-</li>
-</ul>
-<p>Solr 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapDirectory was improved, which caused
- a performance regression in comparison to Solr 3.5.0. This affected
- users with 64bit platforms (Linux, Solaris, Windows) or those
- explicitely using MMapDirectoryFactory.</p>
-</li>
-<li>
-<p>ReplicationHandler "maxNumberOfBackups" was fixed to work if backups are
- triggered on commit.</p>
-</li>
-<li>
-<p>Charset problems were fixed with HttpSolrServer, caused by an upgrade to
- a new Commons HttpClient version in 3.6.0.</p>
-</li>
-<li>
-<p>Grouping was fixed to return correct count when not all shards are
- queried in the second pass. Solr no longer throws Exception when using
- result grouping with main=true and using wt=javabin.</p>
-</li>
-<li>
-<p>Config file replication was made less error prone.</p>
-</li>
-<li>
-<p>Data Import Handler threading fixes.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed.</p>
-</li>
-</ul>
-<h2 id="3-july-2012-lucene-core-40-alpha-and-solr-40-alpha-available">3 July 2012 - Lucene Core 4.0-ALPHA and Solr 4.0-ALPHA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-ALPHA and Apache Solr 4.0-ALPHA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p>The index formats for terms, postings lists, stored fields, term vectors, etc
- are pluggable via the Codec api. You can select from the provided
- implementations or customize the index format with your own Codec to meet your needs.</p>
-</li>
-<li>
-<p>Similarity has been decoupled from the vector space model (TF/IDF). Additional models
- such as BM25, Divergence from Randomness, Language Models, and Information-based models
- are provided (see <a href="http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4">http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4</a>).</p>
-</li>
-<li>
-<p>Added support for per-document values (DocValues). DocValues can be used for custom
- scoring factors (accessible via Similarity), for pre-sorted Sort values, and more.</p>
-</li>
-<li>
-<p>When indexing via multiple threads, each IndexWriter thread now flushes its own segment
- to disk concurrently, resulting in substantial performance improvements
- (see <a href="http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html">http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html</a>).</p>
-</li>
-<li>
-<p>Per-document normalization factors ("norms") are no longer limited to a single byte.
- Similarity implementations can use any DocValues type to store norms.</p>
-</li>
-<li>
-<p>Added index statistics such as the number of tokens for a term or field, number of postings
- for a field, and number of documents with a posting for a field: these support additional
- scoring models (see
- <a href="http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html">http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html</a>).</p>
-</li>
-<li>
-<p>Implemented a new default term dictionary/index (BlockTree) that indexes shared prefixes
- instead of every n'th term. This is not only more time- and space- efficient, but can
- also sometimes avoid going to disk at all for terms that do not exist. Alternative term
- dictionary implementions are provided and pluggable via the Codec api.</p>
-</li>
-<li>
-<p>Indexed terms are no longer UTF-16 char sequences, instead terms can be any binary
- value encoded as byte arrays. By default, text terms are now encoded as UTF-8
- bytes. Sort order of terms is now defined by their binary value, which is identical
- to UTF-8 sort order.</p>
-</li>
-<li>
-<p>Substantially faster performance when using a Filter during searching.</p>
-</li>
-<li>
-<p>File-system based directories can rate-limit the IO (MB/sec) of merge
- threads, to reduce IO contention between merging and searching threads.</p>
-</li>
-<li>
-<p>Added a number of alternative Codecs and components for different use-cases: "Appending"
- works with append-only filesystems (such as Hadoop DFS), "Memory" writes the entire
- terms+postings as an FST read into RAM (see
- <a href="http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html">http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html</a>),
- "Pulsing" inlines the postings for low-frequency terms into the term dictionary (see
- <a href="http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html">http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html</a>),
- "SimpleText" writes all files in plain-text for easy debugging/transparency (see
- <a href="http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html">http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html</a>), among others.</p>
-</li>
-<li>
-<p>Term offsets can be optionally encoded into the postings lists and can be retrieved
- per-position.</p>
-</li>
-<li>
-<p>A new AutomatonQuery returns all documents containing any term matching a provided
- finite-state automaton (see <a href="http://www.slideshare.net/otisg/finite-state-queries-in-lucene">http://www.slideshare.net/otisg/finite-state-queries-in-lucene</a>).</p>
-</li>
-<li>
-<p>FuzzyQuery is 100-200 times faster than in past releases (see
- <a href="http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html">http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html</a>).</p>
-</li>
-<li>
-<p>A new spell checker, DirectSpellChecker, finds possible corrections directly against the
- main search index without requiring a separate index.</p>
-</li>
-<li>
-<p>Various in-memory data structures such as the term dictionary and FieldCache are represented
- more efficiently with less object overhead (see <a href="http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html">http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html</a>).</p>
-</li>
-<li>
-<p>All search logic is now required to work per segment, IndexReader was therefore refactored to
- differentiate between atomic and composite readers
- (see <a href="http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html">http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html</a>).</p>
-</li>
-<li>
-<p>Lucene 4.0 provides a modular API, consolidating components such as Analyzers and Queries
- that were previously scattered across Lucene core, contrib, and Solr. These modules also
- include additional functionality such as UIMA analyzer integration and a completely reworked
- spatial search implementation.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<p>The largest set of features goes by the development code-name âSolr Cloudâ and involves bringing easy scalability to Solr. See <a href="http://wiki.apache.org/solr/SolrCloud">http://wiki.apache.org/solr/SolrCloud</a> for more details.</p>
-<ul>
-<li>
-<p>Distributed indexing designed from the ground up for near real-time (NRT) and NoSQL features such as realtime-get, optimistic locking, and durable updates.</p>
-</li>
-<li>
-<p>High availability with no single points of failure.</p>
-</li>
-<li>
-<p>Apache Zookeeper integration for distributed coordination and cluster metadata and configuration storage.</p>
-</li>
-<li>
-<p>Immunity to split-brain issues due to Zookeeper's Paxos distributed consensus protocols.</p>
-</li>
-<li>
-<p>Updates sent to any node in the cluster and are automatically forwarded to the correct shard and replicated to multiple nodes for redundancy.</p>
-</li>
-<li>
-<p>Queries sent to any node automatically perform a full distributed search across the cluster with load balancing and fail-over.</p>
-</li>
-</ul>
-<p>Solr 4.0-alpha includes more NoSQL features for those using Solr as a primary data store:</p>
-<ul>
-<li>
-<p>Update durability â A transaction log ensures that even uncommitted documents are never lost.</p>
-</li>
-<li>
-<p>Real-time Get â The ability to quickly retrieve the latest version of a document, without the need to commit or open a new searcher</p>
-</li>
-<li>
-<p>Versioning and Optimistic Locking â combined with real-time get, this allows read-update-write functionality that ensures no conflicting changes were made concurrently by other clients.</p>
-</li>
-<li>
-<p>Atomic updates - the ability to add, remove, change, and increment fields of an existing document without having to send in the complete document again.</p>
-</li>
-</ul>
-<p>There are many other features coming in Solr 4, such as</p>
-<ul>
-<li>
-<p>Pivot Faceting â Multi-level or hierarchical faceting where the top constraints for one field are found for each top constraint of a different field.</p>
-</li>
-<li>
-<p>Pseudo-fields â The ability to alias fields, or to add metadata along with returned documents, such as function query values and results of spatial distance calculations.</p>
-</li>
-<li>
-<p>A spell checker implementation that can work directly from the main index instead of creating a sidecar index.</p>
-</li>
-<li>
-<p>Pseudo-Join functionality â The ability to select a set of documents based on their relationship to a second set of documents.</p>
-</li>
-<li>
-<p>Function query enhancements including conditional function queries and relevancy functions.</p>
-</li>
-<li>
-<p>New update processors to facilitate modifying documents prior to indexing.</p>
-</li>
-<li>
-<p>A brand new web admin interface, including support for SolrCloud.</p>
-</li>
</ul></div>
@@ -981,27 +816,15 @@ and Solr can be downloaded from <a href=
<p><div class="button-green">
<a href="core/mirrors-core-latest-redir.html?">Download</a>
<div class="flap top">Click to begin</div>
- <div class="flap bottom">of Apache Lucene 4.3.1</div>
+ <div class="flap bottom">of Apache Lucene 4.4</div>
</div>
- <div class="download-desc">Apache Lucene 4.3.1</div>
+ <div class="download-desc">Apache Lucene 4.4</div>
<div class="button-orange">
<a href="solr/mirrors-solr-latest-redir.html?">Download</a>
<div class="flap top">Click to begin</div>
- <div class="flap bottom">of Apache Solr 4.3.1</div>
- </div>
- <div class="download-desc">Apache Solr 4.3.1</div>
- <div class="button-green">
- <a href="core/mirrors-core-3x-redir.html?">Download</a>
- <div class="flap top">Click to begin</div>
- <div class="flap bottom">of Apache Lucene 3.6.2</div>
- </div>
- <div class="download-desc">Apache Lucene 3.6.2</div>
- <div class="button-orange">
- <a href="solr/mirrors-solr-3x-redir.html?">Download</a>
- <div class="flap top">Click to begin</div>
- <div class="flap bottom">of Apache Solr 3.6.2</div>
+ <div class="flap bottom">of Apache Solr 4.4</div>
</div>
- <div class="download-desc" style="margin-bottom: 4em">Apache Solr 3.6.2</div></p>
+ <div class="download-desc">Apache Solr 4.4</div></p>
<h1 id="projects">Projects</h1>
<ul>
<li><a href="./core/">Lucene Core (Java)</a></li>
Modified: websites/staging/lucene/trunk/content/openrelevance/mailing-lists.html
==============================================================================
--- websites/staging/lucene/trunk/content/openrelevance/mailing-lists.html (original)
+++ websites/staging/lucene/trunk/content/openrelevance/mailing-lists.html Tue Jul 23 16:09:27 2013
@@ -146,7 +146,155 @@ to how to consume ORP content or how to
-<a href="http://mail-archives.apache.org/mod_mbox/lucene-openrelevance-dev/">Archive</a></p></div>
<div><h1 id="lucenewzxhzdk0tmwzxhzdk1-news">Lucene<span style="vertical-align: super; font-size: xx-small">TM</span> News</h1>
-<h2 id="18-june-2013-apache-lucene-431-and-apache-solrwzxhzdk2tmwzxhzdk3-431-available">18 June 2013 - Apache Lucene 4.3.1 and Apache Solr<span style="vertical-align: super; font-size: xx-small">TM</span> 4.3.1 available</h2>
+<h2 id="23-july-2013-apache-lucene-44-and-apache-solrwzxhzdk2tmwzxhzdk3-44-available">23 July 2013 - Apache Lucene 4.4 and Apache Solr<span style="vertical-align: super; font-size: xx-small">TM</span> 4.4 available</h2>
+<p>The Lucene PMC is pleased to announce the availability
+of Apache Lucene 4.4 and Apache Solr 4.4.</p>
+<p>Lucene can be downloaded from <a href="http://lucene.apache.org/core/mirrors-core-latest-redir.html">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
+and Solr can be downloaded from <a href="http://lucene.apache.org/solr/mirrors-solr-latest-redir.html">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
+<p>See the <a href="/core/4_4_0/changes/Changes.html">Lucene CHANGES.txt</a> and
+<a href="/solr/4_4_0/changes/Changes.html">Solr CHANGES.txt</a> files included
+with the release for a full list of details.</p>
+<h3 id="highlights-of-the-lucene-release-include">Highlights of the Lucene release include:</h3>
+<ul>
+<li>
+<p>New Replicator module: replicate index revisions between server and
+ client. See http://shaierera.blogspot.com/2013/05/the-replicator.html</p>
+</li>
+<li>
+<p>New AnalyzingInfixSuggester: finds suggestions based on matches to any
+ tokens in the suggestion, not just based on pure prefix matching. See
+ http://blog.mikemccandless.com/2013/06/a-new-lucene-suggester-based-on-infix.html</p>
+</li>
+<li>
+<p>New PatternCaptureGroupTokenFilter: emit multiple tokens, one for each
+ capture group in one or more Java regexes.</p>
+</li>
+<li>
+<p>New Lucene Facet module features: </p>
+</li>
+<li>Added dynamic (no taxonomy index used) numeric range faceting (see
+ http://blog.mikemccandless.com/2013/05/dynamic-faceting-with-lucene.html )</li>
+<li>Arbitrary Querys are now allowed for per-dimension drill-down on
+ DrillDownQuery and DrillSideways, to support future dynamic faceting.</li>
+<li>
+<p>New FacetResult.mergeHierarchies: merge multiple FacetResult of the
+ same dimension into a single one with the reconstructed hierarchy.</p>
+</li>
+<li>
+<p>FST's Builder can now handle more than 2.1 billion "tail nodes" while
+ building a minimal FST.</p>
+</li>
+<li>
+<p>FieldCache Ints and Longs now use bit-packing to save memory. String fields
+ have more efficient compression if there are many unique terms.</p>
+</li>
+<li>
+<p>Improved compression for NumericDocValues for dates and fields with very
+ small numbers of unique values.</p>
+</li>
+<li>
+<p>New IndexWriter.hasUncommittedChanges(): returns true if there are changes
+ that have not been committed.</p>
+</li>
+<li>
+<p>multiValuedSeparator in PostingsHighlighter is now configurable, for cases
+ where you want a different logical separator between field values.</p>
+</li>
+<li>
+<p>NorwegianLightStemFilter and NorwegianMinimalStemFilter have been extended
+ to handle "nynorsk".</p>
+</li>
+<li>
+<p>New ScandinavianFoldingFilter and ScandinavianNormalizationFilter.</p>
+</li>
+<li>
+<p>Easier compressed norms: Lucene42NormsFormat now takes an overhead
+ parameter, allowing for values other than PackedInts.FASTEST.</p>
+</li>
+<li>
+<p>Analyzer now has an additional tokenStream(String fieldName, String text)
+ method, so wrapping by StringReader for common use is no longer needed.</p>
+</li>
+<li>
+<p>New SimpleMergedSegmentWarmer: just ensures that data structures
+ (terms, norms, docvalues, etc.) are initialized.</p>
+</li>
+<li>
+<p>IndexWriter flushes segments to the compound file format by default.</p>
+</li>
+</ul>
+<p>Lucene 4.4 also includes numerous optimizations and bugfixes.</p>
+<h3 id="highlights-of-the-solr-release-include">Highlights of the Solr release include:</h3>
+<ul>
+<li>
+<p>Solr indexes and transaction logs may stored in HDFS with full read/write
+ capability.</p>
+</li>
+<li>
+<p>Schemaless mode: Added support for a mode that requires no up-front schema
+ modifications, in which previously unknown fields' types are guessed based
+ on the values in added/updated documents, and are then added to the schema
+ prior to processing the update. Note that the below-described features
+ are also useful independently from schemaless mode operation. </p>
+</li>
+<li>New Parse{Date,Integer,Long,Float,Double,Boolean}UpdateProcessorFactory
+ classes parse/guess the field value class for String-valued and unknown
+ fields.</li>
+<li>New AddSchemaFieldsUpdateProcessor: Automatically add new fields to the
+ schema when adding/updating documents with unknown fields. Custom rules
+ map field value class(es) to schema fieldTypes.</li>
+<li>
+<p>A new schemaless mode example configuration, using the above-described
+ field-value-class-guessing and unknown-field-schema-addition features,
+ is provided at solr/example/example-schemaless/.</p>
+</li>
+<li>
+<p>Core Discovery mode: A new solr.xml format which does not store core
+ information, but instead searches for files named 'core.properties' in
+ the filesystem which tell Solr all the details about that core. The main
+ example and the schemaless example both use this new format.</p>
+</li>
+<li>
+<p>Schema REST API: Add support for creating copy fields.</p>
+</li>
+<li>
+<p>A merged segment warmer may now be plugged into solrconfig.xml. </p>
+</li>
+<li>
+<p>New MaxScoreQParserPlugin: Return max() instead of sum() of terms.</p>
+</li>
+<li>
+<p>Binary files are now supported in ZooKeeper.</p>
+</li>
+<li>
+<p>SolrJ's SolrPing object has new methods for ping, enable, and disable.</p>
+</li>
+<li>
+<p>The Admin UI now supports adding documents to Solr.</p>
+</li>
+<li>
+<p>Added a PUT command to the Solr ZkCli tool.</p>
+</li>
+<li>
+<p>New "deleteshard" collections API that unloads all replicas of a given
+ shard and then removes it from the cluster state. It will remove only
+ those shards which are INACTIVE or have no range.</p>
+</li>
+<li>
+<p>The Overseer can now optionally assign generic node names so that
+ new addresses can host shards without naming confusion.</p>
+</li>
+<li>
+<p>The CSV Update Handler now supports optionally adding the line number/
+ row id to a document.</p>
+</li>
+<li>
+<p>Added a new system wide info admin handler that exposes the system info
+ that could previously only be retrieved using a SolrCore.</p>
+</li>
+</ul>
+<p>Solr 4.4 also includes numerous optimizations and bugfixes.</p>
+<h2 id="18-june-2013-apache-lucene-431-and-apache-solr-431-available">18 June 2013 - Apache Lucene 4.3.1 and Apache Solr 4.3.1 available</h2>
<p>The Lucene PMC is pleased to announce the availability
of Apache Lucene 4.3.1 and Apache Solr 4.3.1.</p>
<p>Lucene can be downloaded from <a href="http://lucene.apache.org/core/mirrors-core-latest-redir.html">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
@@ -154,12 +302,12 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_3_1/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_3_1/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_1">Highlights of the Lucene release include:</h3>
<ul>
<li>Lucene 4.3.1 includes 12 bug fixes and 1 optimization, including fixes
for a serious bug that can cause deadlock.</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_1">Highlights of the Solr release include:</h3>
<ul>
<li>
<p>Solr 4.3.1 includes 24 bug fixes. The list includes a lot of SolrCloud
@@ -177,7 +325,7 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_3_0/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_3_0/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include_1">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_2">Highlights of the Lucene release include:</h3>
<ul>
<li>
<p>Significant performance improvements for minShouldMatch BooleanQuery due to
@@ -224,7 +372,7 @@ with the release for a full list of deta
<p>Various bugfixes and optimizations since the 4.2.1 release.</p>
</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include_1">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_2">Highlights of the Solr release include:</h3>
<ul>
<li>
<p>Tired of maintaining core information in solr.xml? Now you can configure
@@ -272,12 +420,12 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_2_1/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_2_1/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include_2">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_3">Highlights of the Lucene release include:</h3>
<ul>
<li>Lucene 4.2.1 includes 9 bug fixes and 3 optimizations, including a fix
for a serious bug that could result in the loss of an index.</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include_2">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_3">Highlights of the Solr release include:</h3>
<ul>
<li>
<p>Solr 4.2.1 includes 38 bug fixes and 2 optimizations. The list includes
@@ -297,7 +445,7 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_2_0/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_2_0/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include_3">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_4">Highlights of the Lucene release include:</h3>
<ul>
<li>
<p>Lucene 4.2 has a new default codec (Lucene42Codec) with a more
@@ -346,7 +494,7 @@ with the release for a full list of deta
<p>Various bugfixes and optimizations since the 4.1 release.</p>
</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include_3">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_4">Highlights of the Solr release include:</h3>
<ul>
<li>
<p>A read side REST API for the schema. Always wanted to introspect the schema over http?
@@ -393,7 +541,7 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_1_0/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_1_0/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include_4">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_5">Highlights of the Lucene release include:</h3>
<ul>
<li>
<p>Lucene 4.1 has a new default codec (Lucene41Codec) based on the
@@ -448,7 +596,7 @@ with the release for a full list of deta
<p>Several bugfixes and optimizations since the 4.0 release.</p>
</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include_4">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_5">Highlights of the Solr release include:</h3>
<h4 id="solrcloud-enhancements-see-httpwikiapacheorgsolrsolrcloud">SolrCloud enhancements (see <a href="http://wiki.apache.org/solr/SolrCloud">http://wiki.apache.org/solr/SolrCloud</a>):</h4>
<ul>
<li>Simple multi-tenancy through enhanced document routing:<ul>
@@ -609,319 +757,6 @@ details.</p>
<li>Various Admin UI improvements.</li>
<li>SolrCloud related performance optimizations in writing the the transaction log, PeerSync recovery, Leader election, and ClusterState caching.</li>
<li>Numerous bug fixes and optimizations.</li>
-</ul>
-<h2 id="14-august-2012-lucene-core-40-beta-and-solr-40-beta-available">14 August 2012 - Lucene Core 4.0-BETA and Solr 4.0-BETA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-BETA and Apache Solr 4.0-BETA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p><a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/index/IndexWriter.html#tryDeleteDocument%28org.apache.lucene.index.IndexReader,%20int%29">
- IndexWriter.tryDeleteDocument</a> can sometimes delete by document ID,
- for higher performance in some applications.</p>
-</li>
-<li>
-<p>New experimental postings formats: <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/bloom/BloomFilteringPostingsFormat.html">
- BloomFilteringPostingsFormat</a> uses a bloom filter to sometimes avoid
- disk seeks when looking up terms,
- <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/memory/DirectPostingsFormat.html">
- DirectPostingsFormat</a> holds all postings as simple byte[] and int[]
- for very fast performance at the cost of very high RAM consumption.</p>
-</li>
-<li>
-<p>CJK analysis improvements: <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-kuromoji/org/apache/lucene/analysis/ja/JapaneseIterationMarkCharFilter.html">
- JapaneseIterationMarkCharFilter</a> normalizes Japanese iteration marks,
- added unigram+bigram support to <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/cjk/CJKBigramFilter.html">
- CJKBigramFilter</a>.</p>
-</li>
-<li>
-<p>Improvements to Scorer navigation API (<a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/search/Scorer.html#getChildren%28%29">
- Scorer.getChildren</a>) to support all queries, useful for determining
- which portions of the query matched.</p>
-</li>
-<li>
-<p>Analysis improvements: factories for creating <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenizerFactory.html">
- Tokenizer</a>, <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenFilterFactory.html">
- TokenFilter</a>, and <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/CharFilterFactory.html">
- CharFilter</a> have been moved from Solr to Lucene's analysis module,
- less memory overhead for StandardTokenizer and Snowball filters.</p>
-</li>
-<li>
-<p>Improved highlighting for multi-valued fields.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<ul>
-<li>
-<p>Added a Collection management API for <a href="http://wiki.apache.org/solr/SolrCloud/">Solr Cloud</a>.</p>
-</li>
-<li>
-<p>Solr Admin UI now clearly displays failures related to initializing SolrCores</p>
-</li>
-<li>
-<p>Updatable documents can create a document if it doesn't already exist,
- or you can force that the document must already exist.</p>
-</li>
-<li>
-<p>Full delete-by-query support for Solr Cloud.</p>
-</li>
-<li>
-<p>Default to <a href="http://lucene.apache.org/solr/api-4_0_0-BETA/org/apache/solr/core/NRTCachingDirectoryFactory.html">
- NRTCachingDirectory</a> for improved near-realtime performance.</p>
-</li>
-<li>
-<p>Improved <a href="http://wiki.apache.org/solr/Solrj">Solrj</a> client performance
- with Solr Cloud: updates are only sent to leaders by default.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<h2 id="22-july-2012-apache-lucene-361-and-apache-solr-361-available">22 July 2012 - Apache Lucene 3.6.1 and Apache Solr 3.6.1 available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 3.6.1 and Apache Solr 3.6.1.</p>
-<p>This release is a bug fix release for version 3.6.0. It contains numerous
-bug fixes, optimizations, and improvements, some of which are highlighted
-below.</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-3x-redir.html?">http://lucene.apache.org/core/mirrors-core-3x-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-3x-redir.html?">http://lucene.apache.org/solr/mirrors-solr-3x-redir.html</a></p>
-<p>See the CHANGES.txt file included with the release for a full list of
-details.</p>
-<p>Lucene 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapIndexInput.clone() was improved, which caused
- a performance regression in comparison to Lucene 3.5.0.</p>
-</li>
-<li>
-<p>MappingCharFilter was fixed to return correct final token positions.</p>
-</li>
-<li>
-<p>QueryParser now supports +/- operators with any amount of whitespace.</p>
-</li>
-<li>
-<p>DisjunctionMaxScorer now implements visitSubScorers().</p>
-</li>
-<li>
-<p>Changed the visibility of Scorer#visitSubScorers() to
- public, otherwise it's impossible to implement Scorers outside
- the Lucene package. This is a small backwards break, affecting a few
- users who implemented custom Scorers.</p>
-</li>
-<li>
-<p>Various analyzer bugs where fixed: Kuromoji to not produce invalid
- token graph due to UNK with punctuation being decompounded, invalid
- position length in SynonymFilter, loading of Hunspell dictionaries that
- use aliasing, be consistent with closing streams when loading
- Hunspell affix files.</p>
-</li>
-<li>
-<p>Various bugs in FST components were fixed: Offline sorter minimum
- buffer size, integer overflow in sorter, FSTCompletionLookup missed
- to close its sorter.</p>
-</li>
-<li>
-<p>Fixed a synchronization bug in handling taxonomies in facet module.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed: BytesRef/CharsRef copy methods
- with nonzero offsets and subSequence off-by-one, TieredMergePolicy
- returned wrong-scaled floor segment setting.</p>
-</li>
-</ul>
-<p>Solr 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapDirectory was improved, which caused
- a performance regression in comparison to Solr 3.5.0. This affected
- users with 64bit platforms (Linux, Solaris, Windows) or those
- explicitely using MMapDirectoryFactory.</p>
-</li>
-<li>
-<p>ReplicationHandler "maxNumberOfBackups" was fixed to work if backups are
- triggered on commit.</p>
-</li>
-<li>
-<p>Charset problems were fixed with HttpSolrServer, caused by an upgrade to
- a new Commons HttpClient version in 3.6.0.</p>
-</li>
-<li>
-<p>Grouping was fixed to return correct count when not all shards are
- queried in the second pass. Solr no longer throws Exception when using
- result grouping with main=true and using wt=javabin.</p>
-</li>
-<li>
-<p>Config file replication was made less error prone.</p>
-</li>
-<li>
-<p>Data Import Handler threading fixes.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed.</p>
-</li>
-</ul>
-<h2 id="3-july-2012-lucene-core-40-alpha-and-solr-40-alpha-available">3 July 2012 - Lucene Core 4.0-ALPHA and Solr 4.0-ALPHA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-ALPHA and Apache Solr 4.0-ALPHA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p>The index formats for terms, postings lists, stored fields, term vectors, etc
- are pluggable via the Codec api. You can select from the provided
- implementations or customize the index format with your own Codec to meet your needs.</p>
-</li>
-<li>
-<p>Similarity has been decoupled from the vector space model (TF/IDF). Additional models
- such as BM25, Divergence from Randomness, Language Models, and Information-based models
- are provided (see <a href="http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4">http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4</a>).</p>
-</li>
-<li>
-<p>Added support for per-document values (DocValues). DocValues can be used for custom
- scoring factors (accessible via Similarity), for pre-sorted Sort values, and more.</p>
-</li>
-<li>
-<p>When indexing via multiple threads, each IndexWriter thread now flushes its own segment
- to disk concurrently, resulting in substantial performance improvements
- (see <a href="http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html">http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html</a>).</p>
-</li>
-<li>
-<p>Per-document normalization factors ("norms") are no longer limited to a single byte.
- Similarity implementations can use any DocValues type to store norms.</p>
-</li>
-<li>
-<p>Added index statistics such as the number of tokens for a term or field, number of postings
- for a field, and number of documents with a posting for a field: these support additional
- scoring models (see
- <a href="http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html">http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html</a>).</p>
-</li>
-<li>
-<p>Implemented a new default term dictionary/index (BlockTree) that indexes shared prefixes
- instead of every n'th term. This is not only more time- and space- efficient, but can
- also sometimes avoid going to disk at all for terms that do not exist. Alternative term
- dictionary implementions are provided and pluggable via the Codec api.</p>
-</li>
-<li>
-<p>Indexed terms are no longer UTF-16 char sequences, instead terms can be any binary
- value encoded as byte arrays. By default, text terms are now encoded as UTF-8
- bytes. Sort order of terms is now defined by their binary value, which is identical
- to UTF-8 sort order.</p>
-</li>
-<li>
-<p>Substantially faster performance when using a Filter during searching.</p>
-</li>
-<li>
-<p>File-system based directories can rate-limit the IO (MB/sec) of merge
- threads, to reduce IO contention between merging and searching threads.</p>
-</li>
-<li>
-<p>Added a number of alternative Codecs and components for different use-cases: "Appending"
- works with append-only filesystems (such as Hadoop DFS), "Memory" writes the entire
- terms+postings as an FST read into RAM (see
- <a href="http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html">http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html</a>),
- "Pulsing" inlines the postings for low-frequency terms into the term dictionary (see
- <a href="http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html">http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html</a>),
- "SimpleText" writes all files in plain-text for easy debugging/transparency (see
- <a href="http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html">http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html</a>), among others.</p>
-</li>
-<li>
-<p>Term offsets can be optionally encoded into the postings lists and can be retrieved
- per-position.</p>
-</li>
-<li>
-<p>A new AutomatonQuery returns all documents containing any term matching a provided
- finite-state automaton (see <a href="http://www.slideshare.net/otisg/finite-state-queries-in-lucene">http://www.slideshare.net/otisg/finite-state-queries-in-lucene</a>).</p>
-</li>
-<li>
-<p>FuzzyQuery is 100-200 times faster than in past releases (see
- <a href="http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html">http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html</a>).</p>
-</li>
-<li>
-<p>A new spell checker, DirectSpellChecker, finds possible corrections directly against the
- main search index without requiring a separate index.</p>
-</li>
-<li>
-<p>Various in-memory data structures such as the term dictionary and FieldCache are represented
- more efficiently with less object overhead (see <a href="http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html">http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html</a>).</p>
-</li>
-<li>
-<p>All search logic is now required to work per segment, IndexReader was therefore refactored to
- differentiate between atomic and composite readers
- (see <a href="http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html">http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html</a>).</p>
-</li>
-<li>
-<p>Lucene 4.0 provides a modular API, consolidating components such as Analyzers and Queries
- that were previously scattered across Lucene core, contrib, and Solr. These modules also
- include additional functionality such as UIMA analyzer integration and a completely reworked
- spatial search implementation.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<p>The largest set of features goes by the development code-name âSolr Cloudâ and involves bringing easy scalability to Solr. See <a href="http://wiki.apache.org/solr/SolrCloud">http://wiki.apache.org/solr/SolrCloud</a> for more details.</p>
-<ul>
-<li>
-<p>Distributed indexing designed from the ground up for near real-time (NRT) and NoSQL features such as realtime-get, optimistic locking, and durable updates.</p>
-</li>
-<li>
-<p>High availability with no single points of failure.</p>
-</li>
-<li>
-<p>Apache Zookeeper integration for distributed coordination and cluster metadata and configuration storage.</p>
-</li>
-<li>
-<p>Immunity to split-brain issues due to Zookeeper's Paxos distributed consensus protocols.</p>
-</li>
-<li>
-<p>Updates sent to any node in the cluster and are automatically forwarded to the correct shard and replicated to multiple nodes for redundancy.</p>
-</li>
-<li>
-<p>Queries sent to any node automatically perform a full distributed search across the cluster with load balancing and fail-over.</p>
-</li>
-</ul>
-<p>Solr 4.0-alpha includes more NoSQL features for those using Solr as a primary data store:</p>
-<ul>
-<li>
-<p>Update durability â A transaction log ensures that even uncommitted documents are never lost.</p>
-</li>
-<li>
-<p>Real-time Get â The ability to quickly retrieve the latest version of a document, without the need to commit or open a new searcher</p>
-</li>
-<li>
-<p>Versioning and Optimistic Locking â combined with real-time get, this allows read-update-write functionality that ensures no conflicting changes were made concurrently by other clients.</p>
-</li>
-<li>
-<p>Atomic updates - the ability to add, remove, change, and increment fields of an existing document without having to send in the complete document again.</p>
-</li>
-</ul>
-<p>There are many other features coming in Solr 4, such as</p>
-<ul>
-<li>
-<p>Pivot Faceting â Multi-level or hierarchical faceting where the top constraints for one field are found for each top constraint of a different field.</p>
-</li>
-<li>
-<p>Pseudo-fields â The ability to alias fields, or to add metadata along with returned documents, such as function query values and results of spatial distance calculations.</p>
-</li>
-<li>
-<p>A spell checker implementation that can work directly from the main index instead of creating a sidecar index.</p>
-</li>
-<li>
-<p>Pseudo-Join functionality â The ability to select a set of documents based on their relationship to a second set of documents.</p>
-</li>
-<li>
-<p>Function query enhancements including conditional function queries and relevancy functions.</p>
-</li>
-<li>
-<p>New update processors to facilitate modifying documents prior to indexing.</p>
-</li>
-<li>
-<p>A brand new web admin interface, including support for SolrCloud.</p>
-</li>
</ul></div>
Modified: websites/staging/lucene/trunk/content/openrelevance/orpnews.html
==============================================================================
--- websites/staging/lucene/trunk/content/openrelevance/orpnews.html (original)
+++ websites/staging/lucene/trunk/content/openrelevance/orpnews.html Tue Jul 23 16:09:27 2013
@@ -139,7 +139,155 @@
<p>The Apache Lucene PMC has officially voted to add the Open Relevance Project (ORP) as a Lucene subproject. ORP's main goal is to build out collections, judgments and queries in an open environment to make it easier for Lucene developers and users to do relevance testing, much like one would get if using TREC or other evaluation conferences.</p></div>
<div><h1 id="lucenewzxhzdk0tmwzxhzdk1-news">Lucene<span style="vertical-align: super; font-size: xx-small">TM</span> News</h1>
-<h2 id="18-june-2013-apache-lucene-431-and-apache-solrwzxhzdk2tmwzxhzdk3-431-available">18 June 2013 - Apache Lucene 4.3.1 and Apache Solr<span style="vertical-align: super; font-size: xx-small">TM</span> 4.3.1 available</h2>
+<h2 id="23-july-2013-apache-lucene-44-and-apache-solrwzxhzdk2tmwzxhzdk3-44-available">23 July 2013 - Apache Lucene 4.4 and Apache Solr<span style="vertical-align: super; font-size: xx-small">TM</span> 4.4 available</h2>
+<p>The Lucene PMC is pleased to announce the availability
+of Apache Lucene 4.4 and Apache Solr 4.4.</p>
+<p>Lucene can be downloaded from <a href="http://lucene.apache.org/core/mirrors-core-latest-redir.html">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
+and Solr can be downloaded from <a href="http://lucene.apache.org/solr/mirrors-solr-latest-redir.html">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
+<p>See the <a href="/core/4_4_0/changes/Changes.html">Lucene CHANGES.txt</a> and
+<a href="/solr/4_4_0/changes/Changes.html">Solr CHANGES.txt</a> files included
+with the release for a full list of details.</p>
+<h3 id="highlights-of-the-lucene-release-include">Highlights of the Lucene release include:</h3>
+<ul>
+<li>
+<p>New Replicator module: replicate index revisions between server and
+ client. See http://shaierera.blogspot.com/2013/05/the-replicator.html</p>
+</li>
+<li>
+<p>New AnalyzingInfixSuggester: finds suggestions based on matches to any
+ tokens in the suggestion, not just based on pure prefix matching. See
+ http://blog.mikemccandless.com/2013/06/a-new-lucene-suggester-based-on-infix.html</p>
+</li>
+<li>
+<p>New PatternCaptureGroupTokenFilter: emit multiple tokens, one for each
+ capture group in one or more Java regexes.</p>
+</li>
+<li>
+<p>New Lucene Facet module features: </p>
+</li>
+<li>Added dynamic (no taxonomy index used) numeric range faceting (see
+ http://blog.mikemccandless.com/2013/05/dynamic-faceting-with-lucene.html )</li>
+<li>Arbitrary Querys are now allowed for per-dimension drill-down on
+ DrillDownQuery and DrillSideways, to support future dynamic faceting.</li>
+<li>
+<p>New FacetResult.mergeHierarchies: merge multiple FacetResult of the
+ same dimension into a single one with the reconstructed hierarchy.</p>
+</li>
+<li>
+<p>FST's Builder can now handle more than 2.1 billion "tail nodes" while
+ building a minimal FST.</p>
+</li>
+<li>
+<p>FieldCache Ints and Longs now use bit-packing to save memory. String fields
+ have more efficient compression if there are many unique terms.</p>
+</li>
+<li>
+<p>Improved compression for NumericDocValues for dates and fields with very
+ small numbers of unique values.</p>
+</li>
+<li>
+<p>New IndexWriter.hasUncommittedChanges(): returns true if there are changes
+ that have not been committed.</p>
+</li>
+<li>
+<p>multiValuedSeparator in PostingsHighlighter is now configurable, for cases
+ where you want a different logical separator between field values.</p>
+</li>
+<li>
+<p>NorwegianLightStemFilter and NorwegianMinimalStemFilter have been extended
+ to handle "nynorsk".</p>
+</li>
+<li>
+<p>New ScandinavianFoldingFilter and ScandinavianNormalizationFilter.</p>
+</li>
+<li>
+<p>Easier compressed norms: Lucene42NormsFormat now takes an overhead
+ parameter, allowing for values other than PackedInts.FASTEST.</p>
+</li>
+<li>
+<p>Analyzer now has an additional tokenStream(String fieldName, String text)
+ method, so wrapping by StringReader for common use is no longer needed.</p>
+</li>
+<li>
+<p>New SimpleMergedSegmentWarmer: just ensures that data structures
+ (terms, norms, docvalues, etc.) are initialized.</p>
+</li>
+<li>
+<p>IndexWriter flushes segments to the compound file format by default.</p>
+</li>
+</ul>
+<p>Lucene 4.4 also includes numerous optimizations and bugfixes.</p>
+<h3 id="highlights-of-the-solr-release-include">Highlights of the Solr release include:</h3>
+<ul>
+<li>
+<p>Solr indexes and transaction logs may stored in HDFS with full read/write
+ capability.</p>
+</li>
+<li>
+<p>Schemaless mode: Added support for a mode that requires no up-front schema
+ modifications, in which previously unknown fields' types are guessed based
+ on the values in added/updated documents, and are then added to the schema
+ prior to processing the update. Note that the below-described features
+ are also useful independently from schemaless mode operation. </p>
+</li>
+<li>New Parse{Date,Integer,Long,Float,Double,Boolean}UpdateProcessorFactory
+ classes parse/guess the field value class for String-valued and unknown
+ fields.</li>
+<li>New AddSchemaFieldsUpdateProcessor: Automatically add new fields to the
+ schema when adding/updating documents with unknown fields. Custom rules
+ map field value class(es) to schema fieldTypes.</li>
+<li>
+<p>A new schemaless mode example configuration, using the above-described
+ field-value-class-guessing and unknown-field-schema-addition features,
+ is provided at solr/example/example-schemaless/.</p>
+</li>
+<li>
+<p>Core Discovery mode: A new solr.xml format which does not store core
+ information, but instead searches for files named 'core.properties' in
+ the filesystem which tell Solr all the details about that core. The main
+ example and the schemaless example both use this new format.</p>
+</li>
+<li>
+<p>Schema REST API: Add support for creating copy fields.</p>
+</li>
+<li>
+<p>A merged segment warmer may now be plugged into solrconfig.xml. </p>
+</li>
+<li>
+<p>New MaxScoreQParserPlugin: Return max() instead of sum() of terms.</p>
+</li>
+<li>
+<p>Binary files are now supported in ZooKeeper.</p>
+</li>
+<li>
+<p>SolrJ's SolrPing object has new methods for ping, enable, and disable.</p>
+</li>
+<li>
+<p>The Admin UI now supports adding documents to Solr.</p>
+</li>
+<li>
+<p>Added a PUT command to the Solr ZkCli tool.</p>
+</li>
+<li>
+<p>New "deleteshard" collections API that unloads all replicas of a given
+ shard and then removes it from the cluster state. It will remove only
+ those shards which are INACTIVE or have no range.</p>
+</li>
+<li>
+<p>The Overseer can now optionally assign generic node names so that
+ new addresses can host shards without naming confusion.</p>
+</li>
+<li>
+<p>The CSV Update Handler now supports optionally adding the line number/
+ row id to a document.</p>
+</li>
+<li>
+<p>Added a new system wide info admin handler that exposes the system info
+ that could previously only be retrieved using a SolrCore.</p>
+</li>
+</ul>
+<p>Solr 4.4 also includes numerous optimizations and bugfixes.</p>
+<h2 id="18-june-2013-apache-lucene-431-and-apache-solr-431-available">18 June 2013 - Apache Lucene 4.3.1 and Apache Solr 4.3.1 available</h2>
<p>The Lucene PMC is pleased to announce the availability
of Apache Lucene 4.3.1 and Apache Solr 4.3.1.</p>
<p>Lucene can be downloaded from <a href="http://lucene.apache.org/core/mirrors-core-latest-redir.html">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
@@ -147,12 +295,12 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_3_1/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_3_1/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_1">Highlights of the Lucene release include:</h3>
<ul>
<li>Lucene 4.3.1 includes 12 bug fixes and 1 optimization, including fixes
for a serious bug that can cause deadlock.</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_1">Highlights of the Solr release include:</h3>
<ul>
<li>
<p>Solr 4.3.1 includes 24 bug fixes. The list includes a lot of SolrCloud
@@ -170,7 +318,7 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_3_0/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_3_0/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include_1">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_2">Highlights of the Lucene release include:</h3>
<ul>
<li>
<p>Significant performance improvements for minShouldMatch BooleanQuery due to
@@ -217,7 +365,7 @@ with the release for a full list of deta
<p>Various bugfixes and optimizations since the 4.2.1 release.</p>
</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include_1">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_2">Highlights of the Solr release include:</h3>
<ul>
<li>
<p>Tired of maintaining core information in solr.xml? Now you can configure
@@ -265,12 +413,12 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_2_1/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_2_1/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include_2">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_3">Highlights of the Lucene release include:</h3>
<ul>
<li>Lucene 4.2.1 includes 9 bug fixes and 3 optimizations, including a fix
for a serious bug that could result in the loss of an index.</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include_2">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_3">Highlights of the Solr release include:</h3>
<ul>
<li>
<p>Solr 4.2.1 includes 38 bug fixes and 2 optimizations. The list includes
@@ -290,7 +438,7 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_2_0/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_2_0/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include_3">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_4">Highlights of the Lucene release include:</h3>
<ul>
<li>
<p>Lucene 4.2 has a new default codec (Lucene42Codec) with a more
@@ -339,7 +487,7 @@ with the release for a full list of deta
<p>Various bugfixes and optimizations since the 4.1 release.</p>
</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include_3">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_4">Highlights of the Solr release include:</h3>
<ul>
<li>
<p>A read side REST API for the schema. Always wanted to introspect the schema over http?
@@ -386,7 +534,7 @@ and Solr can be downloaded from <a href=
<p>See the <a href="/core/4_1_0/changes/Changes.html">Lucene CHANGES.txt</a> and
<a href="/solr/4_1_0/changes/Changes.html">Solr CHANGES.txt</a> files included
with the release for a full list of details.</p>
-<h3 id="highlights-of-the-lucene-release-include_4">Highlights of the Lucene release include:</h3>
+<h3 id="highlights-of-the-lucene-release-include_5">Highlights of the Lucene release include:</h3>
<ul>
<li>
<p>Lucene 4.1 has a new default codec (Lucene41Codec) based on the
@@ -441,7 +589,7 @@ with the release for a full list of deta
<p>Several bugfixes and optimizations since the 4.0 release.</p>
</li>
</ul>
-<h3 id="highlights-of-the-solr-release-include_4">Highlights of the Solr release include:</h3>
+<h3 id="highlights-of-the-solr-release-include_5">Highlights of the Solr release include:</h3>
<h4 id="solrcloud-enhancements-see-httpwikiapacheorgsolrsolrcloud">SolrCloud enhancements (see <a href="http://wiki.apache.org/solr/SolrCloud">http://wiki.apache.org/solr/SolrCloud</a>):</h4>
<ul>
<li>Simple multi-tenancy through enhanced document routing:<ul>
@@ -602,319 +750,6 @@ details.</p>
<li>Various Admin UI improvements.</li>
<li>SolrCloud related performance optimizations in writing the the transaction log, PeerSync recovery, Leader election, and ClusterState caching.</li>
<li>Numerous bug fixes and optimizations.</li>
-</ul>
-<h2 id="14-august-2012-lucene-core-40-beta-and-solr-40-beta-available">14 August 2012 - Lucene Core 4.0-BETA and Solr 4.0-BETA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-BETA and Apache Solr 4.0-BETA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p><a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/index/IndexWriter.html#tryDeleteDocument%28org.apache.lucene.index.IndexReader,%20int%29">
- IndexWriter.tryDeleteDocument</a> can sometimes delete by document ID,
- for higher performance in some applications.</p>
-</li>
-<li>
-<p>New experimental postings formats: <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/bloom/BloomFilteringPostingsFormat.html">
- BloomFilteringPostingsFormat</a> uses a bloom filter to sometimes avoid
- disk seeks when looking up terms,
- <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/memory/DirectPostingsFormat.html">
- DirectPostingsFormat</a> holds all postings as simple byte[] and int[]
- for very fast performance at the cost of very high RAM consumption.</p>
-</li>
-<li>
-<p>CJK analysis improvements: <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-kuromoji/org/apache/lucene/analysis/ja/JapaneseIterationMarkCharFilter.html">
- JapaneseIterationMarkCharFilter</a> normalizes Japanese iteration marks,
- added unigram+bigram support to <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/cjk/CJKBigramFilter.html">
- CJKBigramFilter</a>.</p>
-</li>
-<li>
-<p>Improvements to Scorer navigation API (<a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/search/Scorer.html#getChildren%28%29">
- Scorer.getChildren</a>) to support all queries, useful for determining
- which portions of the query matched.</p>
-</li>
-<li>
-<p>Analysis improvements: factories for creating <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenizerFactory.html">
- Tokenizer</a>, <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenFilterFactory.html">
- TokenFilter</a>, and <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/CharFilterFactory.html">
- CharFilter</a> have been moved from Solr to Lucene's analysis module,
- less memory overhead for StandardTokenizer and Snowball filters.</p>
-</li>
-<li>
-<p>Improved highlighting for multi-valued fields.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<ul>
-<li>
-<p>Added a Collection management API for <a href="http://wiki.apache.org/solr/SolrCloud/">Solr Cloud</a>.</p>
-</li>
-<li>
-<p>Solr Admin UI now clearly displays failures related to initializing SolrCores</p>
-</li>
-<li>
-<p>Updatable documents can create a document if it doesn't already exist,
- or you can force that the document must already exist.</p>
-</li>
-<li>
-<p>Full delete-by-query support for Solr Cloud.</p>
-</li>
-<li>
-<p>Default to <a href="http://lucene.apache.org/solr/api-4_0_0-BETA/org/apache/solr/core/NRTCachingDirectoryFactory.html">
- NRTCachingDirectory</a> for improved near-realtime performance.</p>
-</li>
-<li>
-<p>Improved <a href="http://wiki.apache.org/solr/Solrj">Solrj</a> client performance
- with Solr Cloud: updates are only sent to leaders by default.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<h2 id="22-july-2012-apache-lucene-361-and-apache-solr-361-available">22 July 2012 - Apache Lucene 3.6.1 and Apache Solr 3.6.1 available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 3.6.1 and Apache Solr 3.6.1.</p>
-<p>This release is a bug fix release for version 3.6.0. It contains numerous
-bug fixes, optimizations, and improvements, some of which are highlighted
-below.</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-3x-redir.html?">http://lucene.apache.org/core/mirrors-core-3x-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-3x-redir.html?">http://lucene.apache.org/solr/mirrors-solr-3x-redir.html</a></p>
-<p>See the CHANGES.txt file included with the release for a full list of
-details.</p>
-<p>Lucene 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapIndexInput.clone() was improved, which caused
- a performance regression in comparison to Lucene 3.5.0.</p>
-</li>
-<li>
-<p>MappingCharFilter was fixed to return correct final token positions.</p>
-</li>
-<li>
-<p>QueryParser now supports +/- operators with any amount of whitespace.</p>
-</li>
-<li>
-<p>DisjunctionMaxScorer now implements visitSubScorers().</p>
-</li>
-<li>
-<p>Changed the visibility of Scorer#visitSubScorers() to
- public, otherwise it's impossible to implement Scorers outside
- the Lucene package. This is a small backwards break, affecting a few
- users who implemented custom Scorers.</p>
-</li>
-<li>
-<p>Various analyzer bugs where fixed: Kuromoji to not produce invalid
- token graph due to UNK with punctuation being decompounded, invalid
- position length in SynonymFilter, loading of Hunspell dictionaries that
- use aliasing, be consistent with closing streams when loading
- Hunspell affix files.</p>
-</li>
-<li>
-<p>Various bugs in FST components were fixed: Offline sorter minimum
- buffer size, integer overflow in sorter, FSTCompletionLookup missed
- to close its sorter.</p>
-</li>
-<li>
-<p>Fixed a synchronization bug in handling taxonomies in facet module.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed: BytesRef/CharsRef copy methods
- with nonzero offsets and subSequence off-by-one, TieredMergePolicy
- returned wrong-scaled floor segment setting.</p>
-</li>
-</ul>
-<p>Solr 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapDirectory was improved, which caused
- a performance regression in comparison to Solr 3.5.0. This affected
- users with 64bit platforms (Linux, Solaris, Windows) or those
- explicitely using MMapDirectoryFactory.</p>
-</li>
-<li>
-<p>ReplicationHandler "maxNumberOfBackups" was fixed to work if backups are
- triggered on commit.</p>
-</li>
-<li>
-<p>Charset problems were fixed with HttpSolrServer, caused by an upgrade to
- a new Commons HttpClient version in 3.6.0.</p>
-</li>
-<li>
-<p>Grouping was fixed to return correct count when not all shards are
- queried in the second pass. Solr no longer throws Exception when using
- result grouping with main=true and using wt=javabin.</p>
-</li>
-<li>
-<p>Config file replication was made less error prone.</p>
-</li>
-<li>
-<p>Data Import Handler threading fixes.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed.</p>
-</li>
-</ul>
-<h2 id="3-july-2012-lucene-core-40-alpha-and-solr-40-alpha-available">3 July 2012 - Lucene Core 4.0-ALPHA and Solr 4.0-ALPHA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-ALPHA and Apache Solr 4.0-ALPHA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p>The index formats for terms, postings lists, stored fields, term vectors, etc
- are pluggable via the Codec api. You can select from the provided
- implementations or customize the index format with your own Codec to meet your needs.</p>
-</li>
-<li>
-<p>Similarity has been decoupled from the vector space model (TF/IDF). Additional models
- such as BM25, Divergence from Randomness, Language Models, and Information-based models
- are provided (see <a href="http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4">http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4</a>).</p>
-</li>
-<li>
-<p>Added support for per-document values (DocValues). DocValues can be used for custom
- scoring factors (accessible via Similarity), for pre-sorted Sort values, and more.</p>
-</li>
-<li>
-<p>When indexing via multiple threads, each IndexWriter thread now flushes its own segment
- to disk concurrently, resulting in substantial performance improvements
- (see <a href="http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html">http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html</a>).</p>
-</li>
-<li>
-<p>Per-document normalization factors ("norms") are no longer limited to a single byte.
- Similarity implementations can use any DocValues type to store norms.</p>
-</li>
-<li>
-<p>Added index statistics such as the number of tokens for a term or field, number of postings
- for a field, and number of documents with a posting for a field: these support additional
- scoring models (see
- <a href="http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html">http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html</a>).</p>
-</li>
-<li>
-<p>Implemented a new default term dictionary/index (BlockTree) that indexes shared prefixes
- instead of every n'th term. This is not only more time- and space- efficient, but can
- also sometimes avoid going to disk at all for terms that do not exist. Alternative term
- dictionary implementions are provided and pluggable via the Codec api.</p>
-</li>
-<li>
-<p>Indexed terms are no longer UTF-16 char sequences, instead terms can be any binary
- value encoded as byte arrays. By default, text terms are now encoded as UTF-8
- bytes. Sort order of terms is now defined by their binary value, which is identical
- to UTF-8 sort order.</p>
-</li>
-<li>
-<p>Substantially faster performance when using a Filter during searching.</p>
-</li>
-<li>
-<p>File-system based directories can rate-limit the IO (MB/sec) of merge
- threads, to reduce IO contention between merging and searching threads.</p>
-</li>
-<li>
-<p>Added a number of alternative Codecs and components for different use-cases: "Appending"
- works with append-only filesystems (such as Hadoop DFS), "Memory" writes the entire
- terms+postings as an FST read into RAM (see
- <a href="http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html">http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html</a>),
- "Pulsing" inlines the postings for low-frequency terms into the term dictionary (see
- <a href="http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html">http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html</a>),
- "SimpleText" writes all files in plain-text for easy debugging/transparency (see
- <a href="http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html">http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html</a>), among others.</p>
-</li>
-<li>
-<p>Term offsets can be optionally encoded into the postings lists and can be retrieved
- per-position.</p>
-</li>
-<li>
-<p>A new AutomatonQuery returns all documents containing any term matching a provided
- finite-state automaton (see <a href="http://www.slideshare.net/otisg/finite-state-queries-in-lucene">http://www.slideshare.net/otisg/finite-state-queries-in-lucene</a>).</p>
-</li>
-<li>
-<p>FuzzyQuery is 100-200 times faster than in past releases (see
- <a href="http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html">http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html</a>).</p>
-</li>
-<li>
-<p>A new spell checker, DirectSpellChecker, finds possible corrections directly against the
- main search index without requiring a separate index.</p>
-</li>
-<li>
-<p>Various in-memory data structures such as the term dictionary and FieldCache are represented
- more efficiently with less object overhead (see <a href="http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html">http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html</a>).</p>
-</li>
-<li>
-<p>All search logic is now required to work per segment, IndexReader was therefore refactored to
- differentiate between atomic and composite readers
- (see <a href="http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html">http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html</a>).</p>
-</li>
-<li>
-<p>Lucene 4.0 provides a modular API, consolidating components such as Analyzers and Queries
- that were previously scattered across Lucene core, contrib, and Solr. These modules also
- include additional functionality such as UIMA analyzer integration and a completely reworked
- spatial search implementation.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<p>The largest set of features goes by the development code-name âSolr Cloudâ and involves bringing easy scalability to Solr. See <a href="http://wiki.apache.org/solr/SolrCloud">http://wiki.apache.org/solr/SolrCloud</a> for more details.</p>
-<ul>
-<li>
-<p>Distributed indexing designed from the ground up for near real-time (NRT) and NoSQL features such as realtime-get, optimistic locking, and durable updates.</p>
-</li>
-<li>
-<p>High availability with no single points of failure.</p>
-</li>
-<li>
-<p>Apache Zookeeper integration for distributed coordination and cluster metadata and configuration storage.</p>
-</li>
-<li>
-<p>Immunity to split-brain issues due to Zookeeper's Paxos distributed consensus protocols.</p>
-</li>
-<li>
-<p>Updates sent to any node in the cluster and are automatically forwarded to the correct shard and replicated to multiple nodes for redundancy.</p>
-</li>
-<li>
-<p>Queries sent to any node automatically perform a full distributed search across the cluster with load balancing and fail-over.</p>
-</li>
-</ul>
-<p>Solr 4.0-alpha includes more NoSQL features for those using Solr as a primary data store:</p>
-<ul>
-<li>
-<p>Update durability â A transaction log ensures that even uncommitted documents are never lost.</p>
-</li>
-<li>
-<p>Real-time Get â The ability to quickly retrieve the latest version of a document, without the need to commit or open a new searcher</p>
-</li>
-<li>
-<p>Versioning and Optimistic Locking â combined with real-time get, this allows read-update-write functionality that ensures no conflicting changes were made concurrently by other clients.</p>
-</li>
-<li>
-<p>Atomic updates - the ability to add, remove, change, and increment fields of an existing document without having to send in the complete document again.</p>
-</li>
-</ul>
-<p>There are many other features coming in Solr 4, such as</p>
-<ul>
-<li>
-<p>Pivot Faceting â Multi-level or hierarchical faceting where the top constraints for one field are found for each top constraint of a different field.</p>
-</li>
-<li>
-<p>Pseudo-fields â The ability to alias fields, or to add metadata along with returned documents, such as function query values and results of spatial distance calculations.</p>
-</li>
-<li>
-<p>A spell checker implementation that can work directly from the main index instead of creating a sidecar index.</p>
-</li>
-<li>
-<p>Pseudo-Join functionality â The ability to select a set of documents based on their relationship to a second set of documents.</p>
-</li>
-<li>
-<p>Function query enhancements including conditional function queries and relevancy functions.</p>
-</li>
-<li>
-<p>New update processors to facilitate modifying documents prior to indexing.</p>
-</li>
-<li>
-<p>A brand new web admin interface, including support for SolrCloud.</p>
-</li>
</ul></div>