You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by bu...@apache.org on 2012/08/22 23:25:29 UTC
svn commit: r829774 - in /websites/staging/lucene/trunk/content: ./
pylucene/features.html pylucene/install.html pylucene/jcc/index.html
pylucene/mailing-lists.html pylucene/pynews.html
Author: buildbot
Date: Wed Aug 22 21:25:28 2012
New Revision: 829774
Log:
Staging update by buildbot for lucene
Modified:
websites/staging/lucene/trunk/content/ (props changed)
websites/staging/lucene/trunk/content/pylucene/features.html
websites/staging/lucene/trunk/content/pylucene/install.html
websites/staging/lucene/trunk/content/pylucene/jcc/index.html
websites/staging/lucene/trunk/content/pylucene/mailing-lists.html
websites/staging/lucene/trunk/content/pylucene/pynews.html
Propchange: websites/staging/lucene/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Wed Aug 22 21:25:28 2012
@@ -1 +1 @@
-1376248
+1376259
Modified: websites/staging/lucene/trunk/content/pylucene/features.html
==============================================================================
--- websites/staging/lucene/trunk/content/pylucene/features.html (original)
+++ websites/staging/lucene/trunk/content/pylucene/features.html Wed Aug 22 21:25:28 2012
@@ -352,321 +352,7 @@ points are to be found in PyLucene's uni
in
Action</em> <a href="http://svn.apache.org/viewcvs.cgi/lucene/pylucene/trunk/samples/LuceneInAction">samples</a>.</p></div>
- <div><h1 id="news">News</h1>
-<h2 id="14-august-2012-lucene-core-40-beta-and-solr-40-beta-available">14 August 2012 - Lucene Core 4.0-BETA and Solr 4.0-BETA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-BETA and Apache Solr 4.0-BETA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p><a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/index/IndexWriter.html#tryDeleteDocument%28org.apache.lucene.index.IndexReader,%20int%29">
- IndexWriter.tryDeleteDocument</a> can sometimes delete by document ID,
- for higher performance in some applications.</p>
-</li>
-<li>
-<p>New experimental postings formats: <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/bloom/BloomFilteringPostingsFormat.html">
- BloomFilteringPostingsFormat</a> uses a bloom filter to sometimes avoid
- disk seeks when looking up terms,
- <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/memory/DirectPostingsFormat.html">
- DirectPostingsFormat</a> holds all postings as simple byte[] and int[]
- for very fast performance at the cost of very high RAM consumption.</p>
-</li>
-<li>
-<p>CJK analysis improvements: <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-kuromoji/org/apache/lucene/analysis/ja/JapaneseIterationMarkCharFilter.html">
- JapaneseIterationMarkCharFilter</a> normalizes Japanese iteration marks,
- added unigram+bigram support to <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/cjk/CJKBigramFilter.html">
- CJKBigramFilter</a>.</p>
-</li>
-<li>
-<p>Improvements to Scorer navigation API (<a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/search/Scorer.html#getChildren%28%29">
- Scorer.getChildren</a>) to support all queries, useful for determining
- which portions of the query matched.</p>
-</li>
-<li>
-<p>Analysis improvements: factories for creating <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenizerFactory.html">
- Tokenizer</a>, <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenFilterFactory.html">
- TokenFilter</a>, and <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/CharFilterFactory.html">
- CharFilter</a> have been moved from Solr to Lucene's analysis module,
- less memory overhead for StandardTokenizer and Snowball filters.</p>
-</li>
-<li>
-<p>Improved highlighting for multi-valued fields.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<ul>
-<li>
-<p>Added a Collection management API for <a href="http://wiki.apache.org/solr/SolrCloud/">Solr Cloud</a>.</p>
-</li>
-<li>
-<p>Solr Admin UI now clearly displays failures related to initializing SolrCores</p>
-</li>
-<li>
-<p>Updatable documents can create a document if it doesn't already exist,
- or you can force that the document must already exist.</p>
-</li>
-<li>
-<p>Full delete-by-query support for Solr Cloud.</p>
-</li>
-<li>
-<p>Default to <a href="http://lucene.apache.org/solr/api-4_0_0-BETA/org/apache/solr/core/NRTCachingDirectoryFactory.html">
- NRTCachingDirectory</a> for improved near-realtime performance.</p>
-</li>
-<li>
-<p>Improved <a href="http://wiki.apache.org/solr/Solrj">Solrj</a> client performance
- with Solr Cloud: updates are only sent to leaders by default.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<h2 id="22-july-2012-apache-lucene-361-and-apache-solr-361-available">22 July 2012 - Apache Lucene 3.6.1 and Apache Solr 3.6.1 available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 3.6.1 and Apache Solr 3.6.1.</p>
-<p>This release is a bug fix release for version 3.6.0. It contains numerous
-bug fixes, optimizations, and improvements, some of which are highlighted
-below.</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-3x-redir.html?">http://lucene.apache.org/core/mirrors-core-3x-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-3x-redir.html?">http://lucene.apache.org/solr/mirrors-solr-3x-redir.html</a></p>
-<p>See the CHANGES.txt file included with the release for a full list of
-details.</p>
-<p>Lucene 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapIndexInput.clone() was improved, which caused
- a performance regression in comparison to Lucene 3.5.0.</p>
-</li>
-<li>
-<p>MappingCharFilter was fixed to return correct final token positions.</p>
-</li>
-<li>
-<p>QueryParser now supports +/- operators with any amount of whitespace.</p>
-</li>
-<li>
-<p>DisjunctionMaxScorer now implements visitSubScorers().</p>
-</li>
-<li>
-<p>Changed the visibility of Scorer#visitSubScorers() to
- public, otherwise it's impossible to implement Scorers outside
- the Lucene package. This is a small backwards break, affecting a few
- users who implemented custom Scorers.</p>
-</li>
-<li>
-<p>Various analyzer bugs where fixed: Kuromoji to not produce invalid
- token graph due to UNK with punctuation being decompounded, invalid
- position length in SynonymFilter, loading of Hunspell dictionaries that
- use aliasing, be consistent with closing streams when loading
- Hunspell affix files.</p>
-</li>
-<li>
-<p>Various bugs in FST components were fixed: Offline sorter minimum
- buffer size, integer overflow in sorter, FSTCompletionLookup missed
- to close its sorter.</p>
-</li>
-<li>
-<p>Fixed a synchronization bug in handling taxonomies in facet module.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed: BytesRef/CharsRef copy methods
- with nonzero offsets and subSequence off-by-one, TieredMergePolicy
- returned wrong-scaled floor segment setting.</p>
-</li>
-</ul>
-<p>Solr 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapDirectory was improved, which caused
- a performance regression in comparison to Solr 3.5.0. This affected
- users with 64bit platforms (Linux, Solaris, Windows) or those
- explicitely using MMapDirectoryFactory.</p>
-</li>
-<li>
-<p>ReplicationHandler "maxNumberOfBackups" was fixed to work if backups are
- triggered on commit.</p>
-</li>
-<li>
-<p>Charset problems were fixed with HttpSolrServer, caused by an upgrade to
- a new Commons HttpClient version in 3.6.0.</p>
-</li>
-<li>
-<p>Grouping was fixed to return correct count when not all shards are
- queried in the second pass. Solr no longer throws Exception when using
- result grouping with main=true and using wt=javabin.</p>
-</li>
-<li>
-<p>Config file replication was made less error prone.</p>
-</li>
-<li>
-<p>Data Import Handler threading fixes.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed.</p>
-</li>
-</ul>
-<h2 id="3-july-2012-lucene-core-40-alpha-and-solr-40-alpha-available">3 July 2012 - Lucene Core 4.0-ALPHA and Solr 4.0-ALPHA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-ALPHA and Apache Solr 4.0-ALPHA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p>The index formats for terms, postings lists, stored fields, term vectors, etc
- are pluggable via the Codec api. You can select from the provided
- implementations or customize the index format with your own Codec to meet your needs.</p>
-</li>
-<li>
-<p>Similarity has been decoupled from the vector space model (TF/IDF). Additional models
- such as BM25, Divergence from Randomness, Language Models, and Information-based models
- are provided (see <a href="http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4">http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4</a>).</p>
-</li>
-<li>
-<p>Added support for per-document values (DocValues). DocValues can be used for custom
- scoring factors (accessible via Similarity), for pre-sorted Sort values, and more.</p>
-</li>
-<li>
-<p>When indexing via multiple threads, each IndexWriter thread now flushes its own segment
- to disk concurrently, resulting in substantial performance improvements
- (see <a href="http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html">http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html</a>).</p>
-</li>
-<li>
-<p>Per-document normalization factors ("norms") are no longer limited to a single byte.
- Similarity implementations can use any DocValues type to store norms.</p>
-</li>
-<li>
-<p>Added index statistics such as the number of tokens for a term or field, number of postings
- for a field, and number of documents with a posting for a field: these support additional
- scoring models (see
- <a href="http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html">http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html</a>).</p>
-</li>
-<li>
-<p>Implemented a new default term dictionary/index (BlockTree) that indexes shared prefixes
- instead of every n'th term. This is not only more time- and space- efficient, but can
- also sometimes avoid going to disk at all for terms that do not exist. Alternative term
- dictionary implementions are provided and pluggable via the Codec api.</p>
-</li>
-<li>
-<p>Indexed terms are no longer UTF-16 char sequences, instead terms can be any binary
- value encoded as byte arrays. By default, text terms are now encoded as UTF-8
- bytes. Sort order of terms is now defined by their binary value, which is identical
- to UTF-8 sort order.</p>
-</li>
-<li>
-<p>Substantially faster performance when using a Filter during searching.</p>
-</li>
-<li>
-<p>File-system based directories can rate-limit the IO (MB/sec) of merge
- threads, to reduce IO contention between merging and searching threads.</p>
-</li>
-<li>
-<p>Added a number of alternative Codecs and components for different use-cases: "Appending"
- works with append-only filesystems (such as Hadoop DFS), "Memory" writes the entire
- terms+postings as an FST read into RAM (see
- <a href="http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html">http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html</a>),
- "Pulsing" inlines the postings for low-frequency terms into the term dictionary (see
- <a href="http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html">http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html</a>),
- "SimpleText" writes all files in plain-text for easy debugging/transparency (see
- <a href="http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html">http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html</a>), among others.</p>
-</li>
-<li>
-<p>Term offsets can be optionally encoded into the postings lists and can be retrieved
- per-position.</p>
-</li>
-<li>
-<p>A new AutomatonQuery returns all documents containing any term matching a provided
- finite-state automaton (see <a href="http://www.slideshare.net/otisg/finite-state-queries-in-lucene">http://www.slideshare.net/otisg/finite-state-queries-in-lucene</a>).</p>
-</li>
-<li>
-<p>FuzzyQuery is 100-200 times faster than in past releases (see
- <a href="http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html">http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html</a>).</p>
-</li>
-<li>
-<p>A new spell checker, DirectSpellChecker, finds possible corrections directly against the
- main search index without requiring a separate index.</p>
-</li>
-<li>
-<p>Various in-memory data structures such as the term dictionary and FieldCache are represented
- more efficiently with less object overhead (see <a href="http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html">http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html</a>).</p>
-</li>
-<li>
-<p>All search logic is now required to work per segment, IndexReader was therefore refactored to
- differentiate between atomic and composite readers
- (see <a href="http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html">http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html</a>).</p>
-</li>
-<li>
-<p>Lucene 4.0 provides a modular API, consolidating components such as Analyzers and Queries
- that were previously scattered across Lucene core, contrib, and Solr. These modules also
- include additional functionality such as UIMA analyzer integration and a completely reworked
- spatial search implementation.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<p>The largest set of features goes by the development code-name âSolr Cloudâ and involves bringing easy scalability to Solr. See <a href="http://wiki.apache.org/solr/SolrCloud">http://wiki.apache.org/solr/SolrCloud</a> for more details.</p>
-<ul>
-<li>
-<p>Distributed indexing designed from the ground up for near real-time (NRT) and NoSQL features such as realtime-get, optimistic locking, and durable updates.</p>
-</li>
-<li>
-<p>High availability with no single points of failure.</p>
-</li>
-<li>
-<p>Apache Zookeeper integration for distributed coordination and cluster metadata and configuration storage.</p>
-</li>
-<li>
-<p>Immunity to split-brain issues due to Zookeeper's Paxos distributed consensus protocols.</p>
-</li>
-<li>
-<p>Updates sent to any node in the cluster and are automatically forwarded to the correct shard and replicated to multiple nodes for redundancy.</p>
-</li>
-<li>
-<p>Queries sent to any node automatically perform a full distributed search across the cluster with load balancing and fail-over.</p>
-</li>
-</ul>
-<p>Solr 4.0-alpha includes more NoSQL features for those using Solr as a primary data store:</p>
-<ul>
-<li>
-<p>Update durability â A transaction log ensures that even uncommitted documents are never lost.</p>
-</li>
-<li>
-<p>Real-time Get â The ability to quickly retrieve the latest version of a document, without the need to commit or open a new searcher</p>
-</li>
-<li>
-<p>Versioning and Optimistic Locking â combined with real-time get, this allows read-update-write functionality that ensures no conflicting changes were made concurrently by other clients.</p>
-</li>
-<li>
-<p>Atomic updates - the ability to add, remove, change, and increment fields of an existing document without having to send in the complete document again.</p>
-</li>
-</ul>
-<p>There are many other features coming in Solr 4, such as</p>
-<ul>
-<li>
-<p>Pivot Faceting â Multi-level or hierarchical faceting where the top constraints for one field are found for each top constraint of a different field.</p>
-</li>
-<li>
-<p>Pseudo-fields â The ability to alias fields, or to add metadata along with returned documents, such as function query values and results of spatial distance calculations.</p>
-</li>
-<li>
-<p>A spell checker implementation that can work directly from the main index instead of creating a sidecar index.</p>
-</li>
-<li>
-<p>Pseudo-Join functionality â The ability to select a set of documents based on their relationship to a second set of documents.</p>
-</li>
-<li>
-<p>Function query enhancements including conditional function queries and relevancy functions.</p>
-</li>
-<li>
-<p>New update processors to facilitate modifying documents prior to indexing.</p>
-</li>
-<li>
-<p>A brand new web admin interface, including support for SolrCloud.</p>
-</li>
-</ul></div>
-
+
<div><h2 id="the-apache-software-foundation">The Apache Software Foundation</h2>
Modified: websites/staging/lucene/trunk/content/pylucene/install.html
==============================================================================
--- websites/staging/lucene/trunk/content/pylucene/install.html (original)
+++ websites/staging/lucene/trunk/content/pylucene/install.html Wed Aug 22 21:25:28 2012
@@ -198,321 +198,7 @@ the C++ compiler is used:<br/>
$ CC=CC gmake
</code></p></div>
- <div><h1 id="news">News</h1>
-<h2 id="14-august-2012-lucene-core-40-beta-and-solr-40-beta-available">14 August 2012 - Lucene Core 4.0-BETA and Solr 4.0-BETA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-BETA and Apache Solr 4.0-BETA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p><a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/index/IndexWriter.html#tryDeleteDocument%28org.apache.lucene.index.IndexReader,%20int%29">
- IndexWriter.tryDeleteDocument</a> can sometimes delete by document ID,
- for higher performance in some applications.</p>
-</li>
-<li>
-<p>New experimental postings formats: <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/bloom/BloomFilteringPostingsFormat.html">
- BloomFilteringPostingsFormat</a> uses a bloom filter to sometimes avoid
- disk seeks when looking up terms,
- <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/memory/DirectPostingsFormat.html">
- DirectPostingsFormat</a> holds all postings as simple byte[] and int[]
- for very fast performance at the cost of very high RAM consumption.</p>
-</li>
-<li>
-<p>CJK analysis improvements: <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-kuromoji/org/apache/lucene/analysis/ja/JapaneseIterationMarkCharFilter.html">
- JapaneseIterationMarkCharFilter</a> normalizes Japanese iteration marks,
- added unigram+bigram support to <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/cjk/CJKBigramFilter.html">
- CJKBigramFilter</a>.</p>
-</li>
-<li>
-<p>Improvements to Scorer navigation API (<a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/search/Scorer.html#getChildren%28%29">
- Scorer.getChildren</a>) to support all queries, useful for determining
- which portions of the query matched.</p>
-</li>
-<li>
-<p>Analysis improvements: factories for creating <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenizerFactory.html">
- Tokenizer</a>, <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenFilterFactory.html">
- TokenFilter</a>, and <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/CharFilterFactory.html">
- CharFilter</a> have been moved from Solr to Lucene's analysis module,
- less memory overhead for StandardTokenizer and Snowball filters.</p>
-</li>
-<li>
-<p>Improved highlighting for multi-valued fields.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<ul>
-<li>
-<p>Added a Collection management API for <a href="http://wiki.apache.org/solr/SolrCloud/">Solr Cloud</a>.</p>
-</li>
-<li>
-<p>Solr Admin UI now clearly displays failures related to initializing SolrCores</p>
-</li>
-<li>
-<p>Updatable documents can create a document if it doesn't already exist,
- or you can force that the document must already exist.</p>
-</li>
-<li>
-<p>Full delete-by-query support for Solr Cloud.</p>
-</li>
-<li>
-<p>Default to <a href="http://lucene.apache.org/solr/api-4_0_0-BETA/org/apache/solr/core/NRTCachingDirectoryFactory.html">
- NRTCachingDirectory</a> for improved near-realtime performance.</p>
-</li>
-<li>
-<p>Improved <a href="http://wiki.apache.org/solr/Solrj">Solrj</a> client performance
- with Solr Cloud: updates are only sent to leaders by default.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<h2 id="22-july-2012-apache-lucene-361-and-apache-solr-361-available">22 July 2012 - Apache Lucene 3.6.1 and Apache Solr 3.6.1 available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 3.6.1 and Apache Solr 3.6.1.</p>
-<p>This release is a bug fix release for version 3.6.0. It contains numerous
-bug fixes, optimizations, and improvements, some of which are highlighted
-below.</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-3x-redir.html?">http://lucene.apache.org/core/mirrors-core-3x-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-3x-redir.html?">http://lucene.apache.org/solr/mirrors-solr-3x-redir.html</a></p>
-<p>See the CHANGES.txt file included with the release for a full list of
-details.</p>
-<p>Lucene 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapIndexInput.clone() was improved, which caused
- a performance regression in comparison to Lucene 3.5.0.</p>
-</li>
-<li>
-<p>MappingCharFilter was fixed to return correct final token positions.</p>
-</li>
-<li>
-<p>QueryParser now supports +/- operators with any amount of whitespace.</p>
-</li>
-<li>
-<p>DisjunctionMaxScorer now implements visitSubScorers().</p>
-</li>
-<li>
-<p>Changed the visibility of Scorer#visitSubScorers() to
- public, otherwise it's impossible to implement Scorers outside
- the Lucene package. This is a small backwards break, affecting a few
- users who implemented custom Scorers.</p>
-</li>
-<li>
-<p>Various analyzer bugs where fixed: Kuromoji to not produce invalid
- token graph due to UNK with punctuation being decompounded, invalid
- position length in SynonymFilter, loading of Hunspell dictionaries that
- use aliasing, be consistent with closing streams when loading
- Hunspell affix files.</p>
-</li>
-<li>
-<p>Various bugs in FST components were fixed: Offline sorter minimum
- buffer size, integer overflow in sorter, FSTCompletionLookup missed
- to close its sorter.</p>
-</li>
-<li>
-<p>Fixed a synchronization bug in handling taxonomies in facet module.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed: BytesRef/CharsRef copy methods
- with nonzero offsets and subSequence off-by-one, TieredMergePolicy
- returned wrong-scaled floor segment setting.</p>
-</li>
-</ul>
-<p>Solr 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapDirectory was improved, which caused
- a performance regression in comparison to Solr 3.5.0. This affected
- users with 64bit platforms (Linux, Solaris, Windows) or those
- explicitely using MMapDirectoryFactory.</p>
-</li>
-<li>
-<p>ReplicationHandler "maxNumberOfBackups" was fixed to work if backups are
- triggered on commit.</p>
-</li>
-<li>
-<p>Charset problems were fixed with HttpSolrServer, caused by an upgrade to
- a new Commons HttpClient version in 3.6.0.</p>
-</li>
-<li>
-<p>Grouping was fixed to return correct count when not all shards are
- queried in the second pass. Solr no longer throws Exception when using
- result grouping with main=true and using wt=javabin.</p>
-</li>
-<li>
-<p>Config file replication was made less error prone.</p>
-</li>
-<li>
-<p>Data Import Handler threading fixes.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed.</p>
-</li>
-</ul>
-<h2 id="3-july-2012-lucene-core-40-alpha-and-solr-40-alpha-available">3 July 2012 - Lucene Core 4.0-ALPHA and Solr 4.0-ALPHA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-ALPHA and Apache Solr 4.0-ALPHA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p>The index formats for terms, postings lists, stored fields, term vectors, etc
- are pluggable via the Codec api. You can select from the provided
- implementations or customize the index format with your own Codec to meet your needs.</p>
-</li>
-<li>
-<p>Similarity has been decoupled from the vector space model (TF/IDF). Additional models
- such as BM25, Divergence from Randomness, Language Models, and Information-based models
- are provided (see <a href="http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4">http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4</a>).</p>
-</li>
-<li>
-<p>Added support for per-document values (DocValues). DocValues can be used for custom
- scoring factors (accessible via Similarity), for pre-sorted Sort values, and more.</p>
-</li>
-<li>
-<p>When indexing via multiple threads, each IndexWriter thread now flushes its own segment
- to disk concurrently, resulting in substantial performance improvements
- (see <a href="http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html">http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html</a>).</p>
-</li>
-<li>
-<p>Per-document normalization factors ("norms") are no longer limited to a single byte.
- Similarity implementations can use any DocValues type to store norms.</p>
-</li>
-<li>
-<p>Added index statistics such as the number of tokens for a term or field, number of postings
- for a field, and number of documents with a posting for a field: these support additional
- scoring models (see
- <a href="http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html">http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html</a>).</p>
-</li>
-<li>
-<p>Implemented a new default term dictionary/index (BlockTree) that indexes shared prefixes
- instead of every n'th term. This is not only more time- and space- efficient, but can
- also sometimes avoid going to disk at all for terms that do not exist. Alternative term
- dictionary implementions are provided and pluggable via the Codec api.</p>
-</li>
-<li>
-<p>Indexed terms are no longer UTF-16 char sequences, instead terms can be any binary
- value encoded as byte arrays. By default, text terms are now encoded as UTF-8
- bytes. Sort order of terms is now defined by their binary value, which is identical
- to UTF-8 sort order.</p>
-</li>
-<li>
-<p>Substantially faster performance when using a Filter during searching.</p>
-</li>
-<li>
-<p>File-system based directories can rate-limit the IO (MB/sec) of merge
- threads, to reduce IO contention between merging and searching threads.</p>
-</li>
-<li>
-<p>Added a number of alternative Codecs and components for different use-cases: "Appending"
- works with append-only filesystems (such as Hadoop DFS), "Memory" writes the entire
- terms+postings as an FST read into RAM (see
- <a href="http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html">http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html</a>),
- "Pulsing" inlines the postings for low-frequency terms into the term dictionary (see
- <a href="http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html">http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html</a>),
- "SimpleText" writes all files in plain-text for easy debugging/transparency (see
- <a href="http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html">http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html</a>), among others.</p>
-</li>
-<li>
-<p>Term offsets can be optionally encoded into the postings lists and can be retrieved
- per-position.</p>
-</li>
-<li>
-<p>A new AutomatonQuery returns all documents containing any term matching a provided
- finite-state automaton (see <a href="http://www.slideshare.net/otisg/finite-state-queries-in-lucene">http://www.slideshare.net/otisg/finite-state-queries-in-lucene</a>).</p>
-</li>
-<li>
-<p>FuzzyQuery is 100-200 times faster than in past releases (see
- <a href="http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html">http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html</a>).</p>
-</li>
-<li>
-<p>A new spell checker, DirectSpellChecker, finds possible corrections directly against the
- main search index without requiring a separate index.</p>
-</li>
-<li>
-<p>Various in-memory data structures such as the term dictionary and FieldCache are represented
- more efficiently with less object overhead (see <a href="http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html">http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html</a>).</p>
-</li>
-<li>
-<p>All search logic is now required to work per segment, IndexReader was therefore refactored to
- differentiate between atomic and composite readers
- (see <a href="http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html">http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html</a>).</p>
-</li>
-<li>
-<p>Lucene 4.0 provides a modular API, consolidating components such as Analyzers and Queries
- that were previously scattered across Lucene core, contrib, and Solr. These modules also
- include additional functionality such as UIMA analyzer integration and a completely reworked
- spatial search implementation.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<p>The largest set of features goes by the development code-name âSolr Cloudâ and involves bringing easy scalability to Solr. See <a href="http://wiki.apache.org/solr/SolrCloud">http://wiki.apache.org/solr/SolrCloud</a> for more details.</p>
-<ul>
-<li>
-<p>Distributed indexing designed from the ground up for near real-time (NRT) and NoSQL features such as realtime-get, optimistic locking, and durable updates.</p>
-</li>
-<li>
-<p>High availability with no single points of failure.</p>
-</li>
-<li>
-<p>Apache Zookeeper integration for distributed coordination and cluster metadata and configuration storage.</p>
-</li>
-<li>
-<p>Immunity to split-brain issues due to Zookeeper's Paxos distributed consensus protocols.</p>
-</li>
-<li>
-<p>Updates sent to any node in the cluster and are automatically forwarded to the correct shard and replicated to multiple nodes for redundancy.</p>
-</li>
-<li>
-<p>Queries sent to any node automatically perform a full distributed search across the cluster with load balancing and fail-over.</p>
-</li>
-</ul>
-<p>Solr 4.0-alpha includes more NoSQL features for those using Solr as a primary data store:</p>
-<ul>
-<li>
-<p>Update durability â A transaction log ensures that even uncommitted documents are never lost.</p>
-</li>
-<li>
-<p>Real-time Get â The ability to quickly retrieve the latest version of a document, without the need to commit or open a new searcher</p>
-</li>
-<li>
-<p>Versioning and Optimistic Locking â combined with real-time get, this allows read-update-write functionality that ensures no conflicting changes were made concurrently by other clients.</p>
-</li>
-<li>
-<p>Atomic updates - the ability to add, remove, change, and increment fields of an existing document without having to send in the complete document again.</p>
-</li>
-</ul>
-<p>There are many other features coming in Solr 4, such as</p>
-<ul>
-<li>
-<p>Pivot Faceting â Multi-level or hierarchical faceting where the top constraints for one field are found for each top constraint of a different field.</p>
-</li>
-<li>
-<p>Pseudo-fields â The ability to alias fields, or to add metadata along with returned documents, such as function query values and results of spatial distance calculations.</p>
-</li>
-<li>
-<p>A spell checker implementation that can work directly from the main index instead of creating a sidecar index.</p>
-</li>
-<li>
-<p>Pseudo-Join functionality â The ability to select a set of documents based on their relationship to a second set of documents.</p>
-</li>
-<li>
-<p>Function query enhancements including conditional function queries and relevancy functions.</p>
-</li>
-<li>
-<p>New update processors to facilitate modifying documents prior to indexing.</p>
-</li>
-<li>
-<p>A brand new web admin interface, including support for SolrCloud.</p>
-</li>
-</ul></div>
-
+
<div><h2 id="the-apache-software-foundation">The Apache Software Foundation</h2>
Modified: websites/staging/lucene/trunk/content/pylucene/jcc/index.html
==============================================================================
--- websites/staging/lucene/trunk/content/pylucene/jcc/index.html (original)
+++ websites/staging/lucene/trunk/content/pylucene/jcc/index.html Wed Aug 22 21:25:28 2012
@@ -141,7 +141,7 @@ interpreter.</p>
<p>When generating Python wrappers, JCC produces a complete Python
extension module via the distutils or
<a href="http://pypi.python.org/pypi/setuptools">setuptools</a> packages. </p>
-<p>See <a href="readme.html">here</a> for more information and documentation about JCC.</p>
+<p>See <a href="features.html">here</a> for more information and documentation about JCC.</p>
<h2 id="requirements">Requirements</h2>
<p>JCC is supported on Mac OS X, Linux, Solaris and Windows.</p>
<p>JCC requires Python version 2.x (x >= 3.5) and Java version 1.x
Modified: websites/staging/lucene/trunk/content/pylucene/mailing-lists.html
==============================================================================
--- websites/staging/lucene/trunk/content/pylucene/mailing-lists.html (original)
+++ websites/staging/lucene/trunk/content/pylucene/mailing-lists.html Wed Aug 22 21:25:28 2012
@@ -168,321 +168,7 @@ system</a> then subscribe to the PyLucen
<li><a href="mailto:pylucene-commits-unsubscribe@lucene.apache.org">Unsubscribe from List</a></li>
</ul></div>
- <div><h1 id="news">News</h1>
-<h2 id="14-august-2012-lucene-core-40-beta-and-solr-40-beta-available">14 August 2012 - Lucene Core 4.0-BETA and Solr 4.0-BETA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-BETA and Apache Solr 4.0-BETA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p><a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/index/IndexWriter.html#tryDeleteDocument%28org.apache.lucene.index.IndexReader,%20int%29">
- IndexWriter.tryDeleteDocument</a> can sometimes delete by document ID,
- for higher performance in some applications.</p>
-</li>
-<li>
-<p>New experimental postings formats: <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/bloom/BloomFilteringPostingsFormat.html">
- BloomFilteringPostingsFormat</a> uses a bloom filter to sometimes avoid
- disk seeks when looking up terms,
- <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/memory/DirectPostingsFormat.html">
- DirectPostingsFormat</a> holds all postings as simple byte[] and int[]
- for very fast performance at the cost of very high RAM consumption.</p>
-</li>
-<li>
-<p>CJK analysis improvements: <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-kuromoji/org/apache/lucene/analysis/ja/JapaneseIterationMarkCharFilter.html">
- JapaneseIterationMarkCharFilter</a> normalizes Japanese iteration marks,
- added unigram+bigram support to <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/cjk/CJKBigramFilter.html">
- CJKBigramFilter</a>.</p>
-</li>
-<li>
-<p>Improvements to Scorer navigation API (<a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/search/Scorer.html#getChildren%28%29">
- Scorer.getChildren</a>) to support all queries, useful for determining
- which portions of the query matched.</p>
-</li>
-<li>
-<p>Analysis improvements: factories for creating <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenizerFactory.html">
- Tokenizer</a>, <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenFilterFactory.html">
- TokenFilter</a>, and <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/CharFilterFactory.html">
- CharFilter</a> have been moved from Solr to Lucene's analysis module,
- less memory overhead for StandardTokenizer and Snowball filters.</p>
-</li>
-<li>
-<p>Improved highlighting for multi-valued fields.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<ul>
-<li>
-<p>Added a Collection management API for <a href="http://wiki.apache.org/solr/SolrCloud/">Solr Cloud</a>.</p>
-</li>
-<li>
-<p>Solr Admin UI now clearly displays failures related to initializing SolrCores</p>
-</li>
-<li>
-<p>Updatable documents can create a document if it doesn't already exist,
- or you can force that the document must already exist.</p>
-</li>
-<li>
-<p>Full delete-by-query support for Solr Cloud.</p>
-</li>
-<li>
-<p>Default to <a href="http://lucene.apache.org/solr/api-4_0_0-BETA/org/apache/solr/core/NRTCachingDirectoryFactory.html">
- NRTCachingDirectory</a> for improved near-realtime performance.</p>
-</li>
-<li>
-<p>Improved <a href="http://wiki.apache.org/solr/Solrj">Solrj</a> client performance
- with Solr Cloud: updates are only sent to leaders by default.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<h2 id="22-july-2012-apache-lucene-361-and-apache-solr-361-available">22 July 2012 - Apache Lucene 3.6.1 and Apache Solr 3.6.1 available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 3.6.1 and Apache Solr 3.6.1.</p>
-<p>This release is a bug fix release for version 3.6.0. It contains numerous
-bug fixes, optimizations, and improvements, some of which are highlighted
-below.</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-3x-redir.html?">http://lucene.apache.org/core/mirrors-core-3x-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-3x-redir.html?">http://lucene.apache.org/solr/mirrors-solr-3x-redir.html</a></p>
-<p>See the CHANGES.txt file included with the release for a full list of
-details.</p>
-<p>Lucene 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapIndexInput.clone() was improved, which caused
- a performance regression in comparison to Lucene 3.5.0.</p>
-</li>
-<li>
-<p>MappingCharFilter was fixed to return correct final token positions.</p>
-</li>
-<li>
-<p>QueryParser now supports +/- operators with any amount of whitespace.</p>
-</li>
-<li>
-<p>DisjunctionMaxScorer now implements visitSubScorers().</p>
-</li>
-<li>
-<p>Changed the visibility of Scorer#visitSubScorers() to
- public, otherwise it's impossible to implement Scorers outside
- the Lucene package. This is a small backwards break, affecting a few
- users who implemented custom Scorers.</p>
-</li>
-<li>
-<p>Various analyzer bugs where fixed: Kuromoji to not produce invalid
- token graph due to UNK with punctuation being decompounded, invalid
- position length in SynonymFilter, loading of Hunspell dictionaries that
- use aliasing, be consistent with closing streams when loading
- Hunspell affix files.</p>
-</li>
-<li>
-<p>Various bugs in FST components were fixed: Offline sorter minimum
- buffer size, integer overflow in sorter, FSTCompletionLookup missed
- to close its sorter.</p>
-</li>
-<li>
-<p>Fixed a synchronization bug in handling taxonomies in facet module.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed: BytesRef/CharsRef copy methods
- with nonzero offsets and subSequence off-by-one, TieredMergePolicy
- returned wrong-scaled floor segment setting.</p>
-</li>
-</ul>
-<p>Solr 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapDirectory was improved, which caused
- a performance regression in comparison to Solr 3.5.0. This affected
- users with 64bit platforms (Linux, Solaris, Windows) or those
- explicitely using MMapDirectoryFactory.</p>
-</li>
-<li>
-<p>ReplicationHandler "maxNumberOfBackups" was fixed to work if backups are
- triggered on commit.</p>
-</li>
-<li>
-<p>Charset problems were fixed with HttpSolrServer, caused by an upgrade to
- a new Commons HttpClient version in 3.6.0.</p>
-</li>
-<li>
-<p>Grouping was fixed to return correct count when not all shards are
- queried in the second pass. Solr no longer throws Exception when using
- result grouping with main=true and using wt=javabin.</p>
-</li>
-<li>
-<p>Config file replication was made less error prone.</p>
-</li>
-<li>
-<p>Data Import Handler threading fixes.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed.</p>
-</li>
-</ul>
-<h2 id="3-july-2012-lucene-core-40-alpha-and-solr-40-alpha-available">3 July 2012 - Lucene Core 4.0-ALPHA and Solr 4.0-ALPHA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-ALPHA and Apache Solr 4.0-ALPHA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p>The index formats for terms, postings lists, stored fields, term vectors, etc
- are pluggable via the Codec api. You can select from the provided
- implementations or customize the index format with your own Codec to meet your needs.</p>
-</li>
-<li>
-<p>Similarity has been decoupled from the vector space model (TF/IDF). Additional models
- such as BM25, Divergence from Randomness, Language Models, and Information-based models
- are provided (see <a href="http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4">http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4</a>).</p>
-</li>
-<li>
-<p>Added support for per-document values (DocValues). DocValues can be used for custom
- scoring factors (accessible via Similarity), for pre-sorted Sort values, and more.</p>
-</li>
-<li>
-<p>When indexing via multiple threads, each IndexWriter thread now flushes its own segment
- to disk concurrently, resulting in substantial performance improvements
- (see <a href="http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html">http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html</a>).</p>
-</li>
-<li>
-<p>Per-document normalization factors ("norms") are no longer limited to a single byte.
- Similarity implementations can use any DocValues type to store norms.</p>
-</li>
-<li>
-<p>Added index statistics such as the number of tokens for a term or field, number of postings
- for a field, and number of documents with a posting for a field: these support additional
- scoring models (see
- <a href="http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html">http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html</a>).</p>
-</li>
-<li>
-<p>Implemented a new default term dictionary/index (BlockTree) that indexes shared prefixes
- instead of every n'th term. This is not only more time- and space- efficient, but can
- also sometimes avoid going to disk at all for terms that do not exist. Alternative term
- dictionary implementions are provided and pluggable via the Codec api.</p>
-</li>
-<li>
-<p>Indexed terms are no longer UTF-16 char sequences, instead terms can be any binary
- value encoded as byte arrays. By default, text terms are now encoded as UTF-8
- bytes. Sort order of terms is now defined by their binary value, which is identical
- to UTF-8 sort order.</p>
-</li>
-<li>
-<p>Substantially faster performance when using a Filter during searching.</p>
-</li>
-<li>
-<p>File-system based directories can rate-limit the IO (MB/sec) of merge
- threads, to reduce IO contention between merging and searching threads.</p>
-</li>
-<li>
-<p>Added a number of alternative Codecs and components for different use-cases: "Appending"
- works with append-only filesystems (such as Hadoop DFS), "Memory" writes the entire
- terms+postings as an FST read into RAM (see
- <a href="http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html">http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html</a>),
- "Pulsing" inlines the postings for low-frequency terms into the term dictionary (see
- <a href="http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html">http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html</a>),
- "SimpleText" writes all files in plain-text for easy debugging/transparency (see
- <a href="http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html">http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html</a>), among others.</p>
-</li>
-<li>
-<p>Term offsets can be optionally encoded into the postings lists and can be retrieved
- per-position.</p>
-</li>
-<li>
-<p>A new AutomatonQuery returns all documents containing any term matching a provided
- finite-state automaton (see <a href="http://www.slideshare.net/otisg/finite-state-queries-in-lucene">http://www.slideshare.net/otisg/finite-state-queries-in-lucene</a>).</p>
-</li>
-<li>
-<p>FuzzyQuery is 100-200 times faster than in past releases (see
- <a href="http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html">http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html</a>).</p>
-</li>
-<li>
-<p>A new spell checker, DirectSpellChecker, finds possible corrections directly against the
- main search index without requiring a separate index.</p>
-</li>
-<li>
-<p>Various in-memory data structures such as the term dictionary and FieldCache are represented
- more efficiently with less object overhead (see <a href="http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html">http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html</a>).</p>
-</li>
-<li>
-<p>All search logic is now required to work per segment, IndexReader was therefore refactored to
- differentiate between atomic and composite readers
- (see <a href="http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html">http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html</a>).</p>
-</li>
-<li>
-<p>Lucene 4.0 provides a modular API, consolidating components such as Analyzers and Queries
- that were previously scattered across Lucene core, contrib, and Solr. These modules also
- include additional functionality such as UIMA analyzer integration and a completely reworked
- spatial search implementation.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<p>The largest set of features goes by the development code-name âSolr Cloudâ and involves bringing easy scalability to Solr. See <a href="http://wiki.apache.org/solr/SolrCloud">http://wiki.apache.org/solr/SolrCloud</a> for more details.</p>
-<ul>
-<li>
-<p>Distributed indexing designed from the ground up for near real-time (NRT) and NoSQL features such as realtime-get, optimistic locking, and durable updates.</p>
-</li>
-<li>
-<p>High availability with no single points of failure.</p>
-</li>
-<li>
-<p>Apache Zookeeper integration for distributed coordination and cluster metadata and configuration storage.</p>
-</li>
-<li>
-<p>Immunity to split-brain issues due to Zookeeper's Paxos distributed consensus protocols.</p>
-</li>
-<li>
-<p>Updates sent to any node in the cluster and are automatically forwarded to the correct shard and replicated to multiple nodes for redundancy.</p>
-</li>
-<li>
-<p>Queries sent to any node automatically perform a full distributed search across the cluster with load balancing and fail-over.</p>
-</li>
-</ul>
-<p>Solr 4.0-alpha includes more NoSQL features for those using Solr as a primary data store:</p>
-<ul>
-<li>
-<p>Update durability â A transaction log ensures that even uncommitted documents are never lost.</p>
-</li>
-<li>
-<p>Real-time Get â The ability to quickly retrieve the latest version of a document, without the need to commit or open a new searcher</p>
-</li>
-<li>
-<p>Versioning and Optimistic Locking â combined with real-time get, this allows read-update-write functionality that ensures no conflicting changes were made concurrently by other clients.</p>
-</li>
-<li>
-<p>Atomic updates - the ability to add, remove, change, and increment fields of an existing document without having to send in the complete document again.</p>
-</li>
-</ul>
-<p>There are many other features coming in Solr 4, such as</p>
-<ul>
-<li>
-<p>Pivot Faceting â Multi-level or hierarchical faceting where the top constraints for one field are found for each top constraint of a different field.</p>
-</li>
-<li>
-<p>Pseudo-fields â The ability to alias fields, or to add metadata along with returned documents, such as function query values and results of spatial distance calculations.</p>
-</li>
-<li>
-<p>A spell checker implementation that can work directly from the main index instead of creating a sidecar index.</p>
-</li>
-<li>
-<p>Pseudo-Join functionality â The ability to select a set of documents based on their relationship to a second set of documents.</p>
-</li>
-<li>
-<p>Function query enhancements including conditional function queries and relevancy functions.</p>
-</li>
-<li>
-<p>New update processors to facilitate modifying documents prior to indexing.</p>
-</li>
-<li>
-<p>A brand new web admin interface, including support for SolrCloud.</p>
-</li>
-</ul></div>
-
+
<div><h2 id="the-apache-software-foundation">The Apache Software Foundation</h2>
Modified: websites/staging/lucene/trunk/content/pylucene/pynews.html
==============================================================================
--- websites/staging/lucene/trunk/content/pylucene/pynews.html (original)
+++ websites/staging/lucene/trunk/content/pylucene/pynews.html Wed Aug 22 21:25:28 2012
@@ -193,321 +193,7 @@ A source distribution is available <a hr
subproject. PyLucene was previously hosted at the Open Source Applications
Foundation since its inception in early 2004.</p></div>
- <div><h1 id="news">News</h1>
-<h2 id="14-august-2012-lucene-core-40-beta-and-solr-40-beta-available">14 August 2012 - Lucene Core 4.0-BETA and Solr 4.0-BETA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-BETA and Apache Solr 4.0-BETA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p><a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/index/IndexWriter.html#tryDeleteDocument%28org.apache.lucene.index.IndexReader,%20int%29">
- IndexWriter.tryDeleteDocument</a> can sometimes delete by document ID,
- for higher performance in some applications.</p>
-</li>
-<li>
-<p>New experimental postings formats: <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/bloom/BloomFilteringPostingsFormat.html">
- BloomFilteringPostingsFormat</a> uses a bloom filter to sometimes avoid
- disk seeks when looking up terms,
- <a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/codecs/memory/DirectPostingsFormat.html">
- DirectPostingsFormat</a> holds all postings as simple byte[] and int[]
- for very fast performance at the cost of very high RAM consumption.</p>
-</li>
-<li>
-<p>CJK analysis improvements: <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-kuromoji/org/apache/lucene/analysis/ja/JapaneseIterationMarkCharFilter.html">
- JapaneseIterationMarkCharFilter</a> normalizes Japanese iteration marks,
- added unigram+bigram support to <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/cjk/CJKBigramFilter.html">
- CJKBigramFilter</a>.</p>
-</li>
-<li>
-<p>Improvements to Scorer navigation API (<a href="http://lucene.apache.org/core/4_0_0-BETA/core/org/apache/lucene/search/Scorer.html#getChildren%28%29">
- Scorer.getChildren</a>) to support all queries, useful for determining
- which portions of the query matched.</p>
-</li>
-<li>
-<p>Analysis improvements: factories for creating <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenizerFactory.html">
- Tokenizer</a>, <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/TokenFilterFactory.html">
- TokenFilter</a>, and <a href="http://lucene.apache.org/core/4_0_0-BETA/analyzers-common/org/apache/lucene/analysis/util/CharFilterFactory.html">
- CharFilter</a> have been moved from Solr to Lucene's analysis module,
- less memory overhead for StandardTokenizer and Snowball filters.</p>
-</li>
-<li>
-<p>Improved highlighting for multi-valued fields.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<ul>
-<li>
-<p>Added a Collection management API for <a href="http://wiki.apache.org/solr/SolrCloud/">Solr Cloud</a>.</p>
-</li>
-<li>
-<p>Solr Admin UI now clearly displays failures related to initializing SolrCores</p>
-</li>
-<li>
-<p>Updatable documents can create a document if it doesn't already exist,
- or you can force that the document must already exist.</p>
-</li>
-<li>
-<p>Full delete-by-query support for Solr Cloud.</p>
-</li>
-<li>
-<p>Default to <a href="http://lucene.apache.org/solr/api-4_0_0-BETA/org/apache/solr/core/NRTCachingDirectoryFactory.html">
- NRTCachingDirectory</a> for improved near-realtime performance.</p>
-</li>
-<li>
-<p>Improved <a href="http://wiki.apache.org/solr/Solrj">Solrj</a> client performance
- with Solr Cloud: updates are only sent to leaders by default.</p>
-</li>
-<li>
-<p>Various other API changes, optimizations and bug fixes.</p>
-</li>
-</ul>
-<h2 id="22-july-2012-apache-lucene-361-and-apache-solr-361-available">22 July 2012 - Apache Lucene 3.6.1 and Apache Solr 3.6.1 available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 3.6.1 and Apache Solr 3.6.1.</p>
-<p>This release is a bug fix release for version 3.6.0. It contains numerous
-bug fixes, optimizations, and improvements, some of which are highlighted
-below.</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-3x-redir.html?">http://lucene.apache.org/core/mirrors-core-3x-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-3x-redir.html?">http://lucene.apache.org/solr/mirrors-solr-3x-redir.html</a></p>
-<p>See the CHANGES.txt file included with the release for a full list of
-details.</p>
-<p>Lucene 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapIndexInput.clone() was improved, which caused
- a performance regression in comparison to Lucene 3.5.0.</p>
-</li>
-<li>
-<p>MappingCharFilter was fixed to return correct final token positions.</p>
-</li>
-<li>
-<p>QueryParser now supports +/- operators with any amount of whitespace.</p>
-</li>
-<li>
-<p>DisjunctionMaxScorer now implements visitSubScorers().</p>
-</li>
-<li>
-<p>Changed the visibility of Scorer#visitSubScorers() to
- public, otherwise it's impossible to implement Scorers outside
- the Lucene package. This is a small backwards break, affecting a few
- users who implemented custom Scorers.</p>
-</li>
-<li>
-<p>Various analyzer bugs where fixed: Kuromoji to not produce invalid
- token graph due to UNK with punctuation being decompounded, invalid
- position length in SynonymFilter, loading of Hunspell dictionaries that
- use aliasing, be consistent with closing streams when loading
- Hunspell affix files.</p>
-</li>
-<li>
-<p>Various bugs in FST components were fixed: Offline sorter minimum
- buffer size, integer overflow in sorter, FSTCompletionLookup missed
- to close its sorter.</p>
-</li>
-<li>
-<p>Fixed a synchronization bug in handling taxonomies in facet module.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed: BytesRef/CharsRef copy methods
- with nonzero offsets and subSequence off-by-one, TieredMergePolicy
- returned wrong-scaled floor segment setting.</p>
-</li>
-</ul>
-<p>Solr 3.6.1 Release Highlights:</p>
-<ul>
-<li>
-<p>The concurrency of MMapDirectory was improved, which caused
- a performance regression in comparison to Solr 3.5.0. This affected
- users with 64bit platforms (Linux, Solaris, Windows) or those
- explicitely using MMapDirectoryFactory.</p>
-</li>
-<li>
-<p>ReplicationHandler "maxNumberOfBackups" was fixed to work if backups are
- triggered on commit.</p>
-</li>
-<li>
-<p>Charset problems were fixed with HttpSolrServer, caused by an upgrade to
- a new Commons HttpClient version in 3.6.0.</p>
-</li>
-<li>
-<p>Grouping was fixed to return correct count when not all shards are
- queried in the second pass. Solr no longer throws Exception when using
- result grouping with main=true and using wt=javabin.</p>
-</li>
-<li>
-<p>Config file replication was made less error prone.</p>
-</li>
-<li>
-<p>Data Import Handler threading fixes.</p>
-</li>
-<li>
-<p>Various minor bugs were fixed.</p>
-</li>
-</ul>
-<h2 id="3-july-2012-lucene-core-40-alpha-and-solr-40-alpha-available">3 July 2012 - Lucene Core 4.0-ALPHA and Solr 4.0-ALPHA Available</h2>
-<p>The Lucene PMC is pleased to announce the availability
-of Apache Lucene 4.0-ALPHA and Apache Solr 4.0-ALPHA</p>
-<p>Lucene can be downloaded from <a href="/core/mirrors-core-latest-redir.html?">http://lucene.apache.org/core/mirrors-core-latest-redir.html</a>
-and Solr can be downloaded from <a href="/solr/mirrors-solr-latest-redir.html?">http://lucene.apache.org/solr/mirrors-solr-latest-redir.html</a></p>
-<p>Highlights of the Lucene release include:</p>
-<ul>
-<li>
-<p>The index formats for terms, postings lists, stored fields, term vectors, etc
- are pluggable via the Codec api. You can select from the provided
- implementations or customize the index format with your own Codec to meet your needs.</p>
-</li>
-<li>
-<p>Similarity has been decoupled from the vector space model (TF/IDF). Additional models
- such as BM25, Divergence from Randomness, Language Models, and Information-based models
- are provided (see <a href="http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4">http://www.lucidimagination.com/blog/2011/09/12/flexible-ranking-in-lucene-4</a>).</p>
-</li>
-<li>
-<p>Added support for per-document values (DocValues). DocValues can be used for custom
- scoring factors (accessible via Similarity), for pre-sorted Sort values, and more.</p>
-</li>
-<li>
-<p>When indexing via multiple threads, each IndexWriter thread now flushes its own segment
- to disk concurrently, resulting in substantial performance improvements
- (see <a href="http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html">http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html</a>).</p>
-</li>
-<li>
-<p>Per-document normalization factors ("norms") are no longer limited to a single byte.
- Similarity implementations can use any DocValues type to store norms.</p>
-</li>
-<li>
-<p>Added index statistics such as the number of tokens for a term or field, number of postings
- for a field, and number of documents with a posting for a field: these support additional
- scoring models (see
- <a href="http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html">http://blog.mikemccandless.com/2012/03/new-index-statistics-in-lucene-40.html</a>).</p>
-</li>
-<li>
-<p>Implemented a new default term dictionary/index (BlockTree) that indexes shared prefixes
- instead of every n'th term. This is not only more time- and space- efficient, but can
- also sometimes avoid going to disk at all for terms that do not exist. Alternative term
- dictionary implementions are provided and pluggable via the Codec api.</p>
-</li>
-<li>
-<p>Indexed terms are no longer UTF-16 char sequences, instead terms can be any binary
- value encoded as byte arrays. By default, text terms are now encoded as UTF-8
- bytes. Sort order of terms is now defined by their binary value, which is identical
- to UTF-8 sort order.</p>
-</li>
-<li>
-<p>Substantially faster performance when using a Filter during searching.</p>
-</li>
-<li>
-<p>File-system based directories can rate-limit the IO (MB/sec) of merge
- threads, to reduce IO contention between merging and searching threads.</p>
-</li>
-<li>
-<p>Added a number of alternative Codecs and components for different use-cases: "Appending"
- works with append-only filesystems (such as Hadoop DFS), "Memory" writes the entire
- terms+postings as an FST read into RAM (see
- <a href="http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html">http://blog.mikemccandless.com/2011/06/primary-key-lookups-are-28x-faster-with.html</a>),
- "Pulsing" inlines the postings for low-frequency terms into the term dictionary (see
- <a href="http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html">http://blog.mikemccandless.com/2010/06/lucenes-pulsingcodec-on-primary-key.html</a>),
- "SimpleText" writes all files in plain-text for easy debugging/transparency (see
- <a href="http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html">http://blog.mikemccandless.com/2010/10/lucenes-simpletext-codec.html</a>), among others.</p>
-</li>
-<li>
-<p>Term offsets can be optionally encoded into the postings lists and can be retrieved
- per-position.</p>
-</li>
-<li>
-<p>A new AutomatonQuery returns all documents containing any term matching a provided
- finite-state automaton (see <a href="http://www.slideshare.net/otisg/finite-state-queries-in-lucene">http://www.slideshare.net/otisg/finite-state-queries-in-lucene</a>).</p>
-</li>
-<li>
-<p>FuzzyQuery is 100-200 times faster than in past releases (see
- <a href="http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html">http://blog.mikemccandless.com/2011/03/lucenes-fuzzyquery-is-100-times-faster.html</a>).</p>
-</li>
-<li>
-<p>A new spell checker, DirectSpellChecker, finds possible corrections directly against the
- main search index without requiring a separate index.</p>
-</li>
-<li>
-<p>Various in-memory data structures such as the term dictionary and FieldCache are represented
- more efficiently with less object overhead (see <a href="http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html">http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html</a>).</p>
-</li>
-<li>
-<p>All search logic is now required to work per segment, IndexReader was therefore refactored to
- differentiate between atomic and composite readers
- (see <a href="http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html">http://blog.thetaphi.de/2012/02/is-your-indexreader-atomic-major.html</a>).</p>
-</li>
-<li>
-<p>Lucene 4.0 provides a modular API, consolidating components such as Analyzers and Queries
- that were previously scattered across Lucene core, contrib, and Solr. These modules also
- include additional functionality such as UIMA analyzer integration and a completely reworked
- spatial search implementation.</p>
-</li>
-</ul>
-<p>Highlights of the Solr release include:</p>
-<p>The largest set of features goes by the development code-name âSolr Cloudâ and involves bringing easy scalability to Solr. See <a href="http://wiki.apache.org/solr/SolrCloud">http://wiki.apache.org/solr/SolrCloud</a> for more details.</p>
-<ul>
-<li>
-<p>Distributed indexing designed from the ground up for near real-time (NRT) and NoSQL features such as realtime-get, optimistic locking, and durable updates.</p>
-</li>
-<li>
-<p>High availability with no single points of failure.</p>
-</li>
-<li>
-<p>Apache Zookeeper integration for distributed coordination and cluster metadata and configuration storage.</p>
-</li>
-<li>
-<p>Immunity to split-brain issues due to Zookeeper's Paxos distributed consensus protocols.</p>
-</li>
-<li>
-<p>Updates sent to any node in the cluster and are automatically forwarded to the correct shard and replicated to multiple nodes for redundancy.</p>
-</li>
-<li>
-<p>Queries sent to any node automatically perform a full distributed search across the cluster with load balancing and fail-over.</p>
-</li>
-</ul>
-<p>Solr 4.0-alpha includes more NoSQL features for those using Solr as a primary data store:</p>
-<ul>
-<li>
-<p>Update durability â A transaction log ensures that even uncommitted documents are never lost.</p>
-</li>
-<li>
-<p>Real-time Get â The ability to quickly retrieve the latest version of a document, without the need to commit or open a new searcher</p>
-</li>
-<li>
-<p>Versioning and Optimistic Locking â combined with real-time get, this allows read-update-write functionality that ensures no conflicting changes were made concurrently by other clients.</p>
-</li>
-<li>
-<p>Atomic updates - the ability to add, remove, change, and increment fields of an existing document without having to send in the complete document again.</p>
-</li>
-</ul>
-<p>There are many other features coming in Solr 4, such as</p>
-<ul>
-<li>
-<p>Pivot Faceting â Multi-level or hierarchical faceting where the top constraints for one field are found for each top constraint of a different field.</p>
-</li>
-<li>
-<p>Pseudo-fields â The ability to alias fields, or to add metadata along with returned documents, such as function query values and results of spatial distance calculations.</p>
-</li>
-<li>
-<p>A spell checker implementation that can work directly from the main index instead of creating a sidecar index.</p>
-</li>
-<li>
-<p>Pseudo-Join functionality â The ability to select a set of documents based on their relationship to a second set of documents.</p>
-</li>
-<li>
-<p>Function query enhancements including conditional function queries and relevancy functions.</p>
-</li>
-<li>
-<p>New update processors to facilitate modifying documents prior to indexing.</p>
-</li>
-<li>
-<p>A brand new web admin interface, including support for SolrCloud.</p>
-</li>
-</ul></div>
-
+
<div><h2 id="the-apache-software-foundation">The Apache Software Foundation</h2>