You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Steven A Rowe <sa...@syr.edu> on 2011/06/16 19:17:50 UTC

RE: svn commit: r1136543 - /lucene/dev/branches/branch_3x/lucene/CHANGES.txt

Thanks Robert, I was the botcher...  TODO: double check CHANGES.txt diff after a merge... - Steve

> -----Original Message-----
> From: rmuir@apache.org [mailto:rmuir@apache.org]
> Sent: Thursday, June 16, 2011 12:57 PM
> To: commits@lucene.apache.org
> Subject: svn commit: r1136543 -
> /lucene/dev/branches/branch_3x/lucene/CHANGES.txt
>
> Author: rmuir
> Date: Thu Jun 16 16:56:39 2011
> New Revision: 1136543
>
> URL: http://svn.apache.org/viewvc?rev=1136543&view=rev
> Log:
> LUCENE-3204: fix botched CHANGES merge
>
> Modified:
>     lucene/dev/branches/branch_3x/lucene/CHANGES.txt
>
> Modified: lucene/dev/branches/branch_3x/lucene/CHANGES.txt
> URL:
> http://svn.apache.org/viewvc/lucene/dev/branches/branch_3x/lucene/CHANGES
> .txt?rev=1136543&r1=1136542&r2=1136543&view=diff
> =========================================================================
> =====
> --- lucene/dev/branches/branch_3x/lucene/CHANGES.txt (original)
> +++ lucene/dev/branches/branch_3x/lucene/CHANGES.txt Thu Jun 16 16:56:39
> 2011
> @@ -3,468 +3,6 @@ Lucene Change Log
>  For more information on past and future Lucene versions, please see:
>  http://s.apache.org/luceneversions
>
> -======================= Trunk (not yet released) =======================
> -
> -Changes in backwards compatibility policy
> -
> -* LUCENE-1458, LUCENE-2111, LUCENE-2354: Changes from flexible indexing:
> -
> -  - On upgrading to 3.1, if you do not fully reindex your documents,
> -    Lucene will emulate the new flex API on top of the old index,
> -    incurring some performance cost (up to ~10% slowdown, typically).
> -    To prevent this slowdown, use oal.index.IndexUpgrader
> -    to upgrade your indexes to latest file format (LUCENE-3082).
> -
> -    Mixed flex/pre-flex indexes are perfectly fine -- the two
> -    emulation layers (flex API on pre-flex index, and pre-flex API on
> -    flex index) will remap the access as required.  So on upgrading to
> -    3.1 you can start indexing new documents into an existing index.
> -    To get optimal performance, use oal.index.IndexUpgrader
> -    to upgrade your indexes to latest file format (LUCENE-3082).
> -
> -  - The postings APIs (TermEnum, TermDocsEnum, TermPositionsEnum)
> -    have been removed in favor of the new flexible
> -    indexing (flex) APIs (Fields, FieldsEnum, Terms, TermsEnum,
> -    DocsEnum, DocsAndPositionsEnum). One big difference is that field
> -    and terms are now enumerated separately: a TermsEnum provides a
> -    BytesRef (wraps a byte[]) per term within a single field, not a
> -    Term.  Another is that when asking for a Docs/AndPositionsEnum, you
> -    now specify the skipDocs explicitly (typically this will be the
> -    deleted docs, but in general you can provide any Bits).
> -
> -  - MultiReader ctor now throws IOException
> -
> -  - Directory.copy/Directory.copyTo now copies all files (not just
> -    index files), since what is and isn't and index file is now
> -    dependent on the codecs used.
> -
> -  - UnicodeUtil now uses BytesRef for UTF-8 output, and some method
> -    signatures have changed to CharSequence.  These are internal APIs
> -    and subject to change suddenly.
> -
> -  - Positional queries (PhraseQuery, *SpanQuery) will now throw an
> -    exception if use them on a field that omits positions during
> -    indexing (previously they silently returned no results).
> -
> -  - FieldCache.{Byte,Short,Int,Long,Float,Double}Parser's API has
> -    changed -- each parse method now takes a BytesRef instead of a
> -    String.  If you have an existing Parser, a simple way to fix it is
> -    invoke BytesRef.utf8ToString, and pass that String to your
> -    existing parser.  This will work, but performance would be better
> -    if you could fix your parser to instead operate directly on the
> -    byte[] in the BytesRef.
> -
> -  - The internal (experimental) API of NumericUtils changed completely
> -    from String to BytesRef. Client code should never use this class,
> -    so the change would normally not affect you. If you used some of
> -    the methods to inspect terms or create TermQueries out of
> -    prefix encoded terms, change to use BytesRef. Please note:
> -    Do not use TermQueries to search for single numeric terms.
> -    The recommended way is to create a corresponding NumericRangeQuery
> -    with upper and lower bound equal and included. TermQueries do not
> -    score correct, so the constant score mode of NRQ is the only
> -    correct way to handle single value queries.
> -
> -  - NumericTokenStream now works directly on byte[] terms. If you
> -    plug a TokenFilter on top of this stream, you will likely get
> -    an IllegalArgumentException, because the NTS does not support
> -    TermAttribute/CharTermAttribute. If you want to further filter
> -    or attach Payloads to NTS, use the new NumericTermAttribute.
> -
> -  (Mike McCandless, Robert Muir, Uwe Schindler, Mark Miller, Michael
> Busch)
> -
> -* LUCENE-2265: FuzzyQuery and WildcardQuery now operate on Unicode
> codepoints,
> -  not unicode code units. For example, a Wildcard "?" represents any
> unicode
> -  character. Furthermore, the rest of the automaton package and
> RegexpQuery use
> -  true Unicode codepoint representation.  (Robert Muir, Mike McCandless)
> -
> -* LUCENE-2380: The String-based FieldCache methods (getStrings,
> -  getStringIndex) have been replaced with BytesRef-based equivalents
> -  (getTerms, getTermsIndex).  Also, the sort values (returned in
> -  FieldDoc.fields) when sorting by SortField.STRING or
> -  SortField.STRING_VAL are now BytesRef instances.  See MIGRATE.txt
> -  for more details. (yonik, Mike McCandless)
> -
> -* LUCENE-2480: Though not a change in backwards compatibility policy,
> pre-3.0
> -  indexes are no longer supported. You should upgrade to 3.x first, then
> run
> -  optimize(), or reindex. (Shai Erera, Earwin Burrfoot)
> -
> -* LUCENE-2484: Removed deprecated TermAttribute. Use CharTermAttribute
> -  and TermToBytesRefAttribute instead.  (Uwe Schindler)
> -
> -* LUCENE-2600: Remove IndexReader.isDeleted in favor of
> -  IndexReader.getDeletedDocs().  (Mike McCandless)
> -
> -* LUCENE-2667: FuzzyQuery's defaults have changed for more performant
> -  behavior: the minimum similarity is 2 edit distances from the word,
> -  and the priority queue size is 50. To support this, FuzzyQuery now
> allows
> -  specifying unscaled edit distances (foobar~2). If your application
> depends
> -  upon the old defaults of 0.5 (scaled) minimum similarity and
> Integer.MAX_VALUE
> -  priority queue size, you can use FuzzyQuery(Term, float, int, int) to
> specify
> -  those explicitly.
> -
> -* LUCENE-2674: MultiTermQuery.TermCollector.collect now accepts the
> -  TermsEnum as well.  (Robert Muir, Mike McCandless)
> -
> -* LUCENE-588: WildcardQuery and QueryParser now allows escaping with
> -  the '\' character. Previously this was impossible (you could not
> escape */?,
> -  for example).  If your code somehow depends on the old behavior, you
> will
> -  need to change it (e.g. using "\\" to escape '\' itself).
> -  (Sunil Kamath, Terry Yang via Robert Muir)
> -
> -* LUCENE-2771: IndexReader.norms() now throws UOE on non-atomic
> IndexReaders. If
> -  you really want a top-level norms, use MultiNorms or
> SlowMultiReaderWrapper.
> -  (Uwe Schindler, Robert Muir)
> -
> -* LUCENE-2837: Collapsed Searcher, Searchable into IndexSearcher;
> -  removed contrib/remote and MultiSearcher (Mike McCandless); absorbed
> -  ParallelMultiSearcher into IndexSearcher as an optional
> -  ExecutorServiced passed to its ctor.  (Mike McCandless)
> -
> -* LUCENE-2908: Removed serialization code from lucene classes. It is
> recommended
> -  that you serialize user search needs at a higher level in your
> application.
> -  (Robert Muir)
> -
> -* LUCENE-2831: Changed Weight#scorer, Weight#explain &
> Filter#getDocIdSet to
> -  operate on a AtomicReaderContext instead of directly on IndexReader to
> enable
> -  searches to be aware of IndexSearcher's context. (Simon Willnauer)
> -
> -* LUCENE-2839: Scorer#score(Collector,int,int) is now public because it
> is
> -  called from other classes and part of public API. (Uwe Schindler)
> -
> -* LUCENE-2865: Weight#scorer(AtomicReaderContext, boolean, boolean) now
> accepts
> -  a ScorerContext struct instead of booleans.(Simon Willnauer)
> -
> -* LUCENE-2882: Cut over SpanQuery#getSpans to AtomicReaderContext to
> enforce
> -  per segment semantics on SpanQuery & Spans. (Simon Willnauer)
> -
> -* LUCENE-2236: Similarity can now be configured on a per-field basis.
> See the
> -  migration notes in MIGRATE.txt for more details.  (Robert Muir, Doron
> Cohen)
> -
> -* LUCENE-2315: AttributeSource's methods for accessing attributes are
> now final,
> -  else its easy to corrupt the internal states.  (Uwe Schindler)
> -
> -* LUCENE-2814: The IndexWriter.flush method no longer takes "boolean
> -  flushDocStores" argument, as we now always flush doc stores (index
> -  files holding stored fields and term vectors) while flushing a
> -  segment.  (Mike McCandless)
> -
> -Changes in Runtime Behavior
> -
> -* LUCENE-2846: omitNorms now behaves like omitTermFrequencyAndPositions,
> if you
> -  omitNorms(true) for field "a" for 1000 documents, but then add a
> document with
> -  omitNorms(false) for field "a", all documents for field "a" will have
> no norms.
> -  Previously, Lucene would fill the first 1000 documents with "fake
> norms" from
> -  Similarity.getDefault(). (Robert Muir, Mike Mccandless)
> -
> -* LUCENE-2846: When some documents contain field "a", and others do not,
> the
> -  documents that don't have the field get a norm byte value of 0.
> Previously, Lucene
> -  would populate "fake norms" with Similarity.getDefault() for these
> documents.
> -  (Robert Muir, Mike Mccandless)
> -
> -* LUCENE-2720: IndexWriter throws IndexFormatTooOldException on open,
> rather
> -  than later when e.g. a merge starts. (Shai Erera, Mike McCandless, Uwe
> Schindler)
> -
> -* LUCENE-2881: FieldInfos is now tracked per segment.  Before it was
> tracked
> -  per IndexWriter session, which resulted in FieldInfos that had the
> FieldInfo
> -  properties from all previous segments combined. Field numbers are now
> tracked
> -  globally across IndexWriter sessions and persisted into a X.fnx file
> on
> -  successful commit. The corresponding file format changes are
> backwards-
> -  compatible. (Michael Busch, Simon Willnauer)
> -
> -* LUCENE-2956, LUCENE-2573, LUCENE-2324, LUCENE-2555: Changes from
> -  DocumentsWriterPerThread:
> -
> -  - IndexWriter now uses a DocumentsWriter per thread when indexing
> documents.
> -    Each DocumentsWriterPerThread indexes documents in its own private
> segment,
> -    and the in memory segments are no longer merged on flush.  Instead,
> each
> -    segment is separately flushed to disk and subsequently merged with
> normal
> -    segment merging.
> -
> -  - DocumentsWriterPerThread (DWPT) is now flushed concurrently based on
> a
> -    FlushPolicy.  When a DWPT is flushed, a fresh DWPT is swapped in so
> that
> -    indexing may continue concurrently with flushing.  The selected
> -    DWPT flushes all its RAM resident documents do disk.  Note: Segment
> flushes
> -    don't flush all RAM resident documents but only the documents
> private to
> -    the DWPT selected for flushing.
> -
> -  - Flushing is now controlled by FlushPolicy that is called for every
> add,
> -    update or delete on IndexWriter. By default DWPTs are flushed either
> on
> -    maxBufferedDocs per DWPT or the global active used memory. Once the
> active
> -    memory exceeds ramBufferSizeMB only the largest DWPT is selected for
> -    flushing and the memory used by this DWPT is substracted from the
> active
> -    memory and added to a flushing memory pool, which can lead to
> temporarily
> -    higher memory usage due to ongoing indexing.
> -
> -  - IndexWriter now can utilize ramBufferSize > 2048 MB. Each DWPT can
> address
> -    up to 2048 MB memory such that the ramBufferSize is now bounded by
> the max
> -    number of DWPT avaliable in the used DocumentsWriterPerThreadPool.
> -    IndexWriters net memory consumption can grow far beyond the 2048 MB
> limit if
> -    the applicatoin can use all available DWPTs. To prevent a DWPT from
> -    exhausting its address space IndexWriter will forcefully flush a
> DWPT if its
> -    hard memory limit is exceeded. The RAMPerThreadHardLimitMB can be
> controlled
> -    via IndexWriterConfig and defaults to 1945 MB.
> -    Since IndexWriter flushes DWPT concurrently not all memory is
> released
> -    immediately. Applications should still use a ramBufferSize
> significantly
> -    lower than the JVMs avaliable heap memory since under high load
> multiple
> -    flushing DWPT can consume substantial transient memory when IO
> performance
> -    is slow relative to indexing rate.
> -
> -  - IndexWriter#commit now doesn't block concurrent indexing while
> flushing all
> -    'currently' RAM resident documents to disk. Yet, flushes that occur
> while a
> -    a full flush is running are queued and will happen after all DWPT
> involved
> -    in the full flush are done flushing. Applications using multiple
> threads
> -    during indexing and trigger a full flush (eg call commmit() or open
> a new
> -    NRT reader) can use significantly more transient memory.
> -
> -  - IndexWriter#addDocument and IndexWriter.updateDocument can block
> indexing
> -    threads if the number of active + number of flushing DWPT exceed a
> -    safety limit. By default this happens if 2 * max number available
> thread
> -    states (DWPTPool) is exceeded. This safety limit prevents
> applications from
> -    exhausting their available memory if flushing can't keep up with
> -    concurrently indexing threads.
> -
> -  - IndexWriter only applies and flushes deletes if the
> maxBufferedDelTerms
> -    limit is reached during indexing. No segment flushes will be
> triggered
> -    due to this setting.
> -
> -  - IndexWriter#flush(boolean, boolean) doesn't synchronized on
> IndexWriter
> -    anymore. A dedicated flushLock has been introduced to prevent
> multiple full-
> -    flushes happening concurrently.
> -
> -  - DocumentsWriter doesn't write shared doc stores anymore.
> -
> -  (Mike McCandless, Michael Busch, Simon Willnauer)
> -
> -* LUCENE-3146: IndexReader.setNorm throws IllegalStateException if the
> field
> -  does not store norms. (Shai Erera, Mike McCandless)
> -
> -API Changes
> -
> -* LUCENE-2302, LUCENE-1458, LUCENE-2111, LUCENE-2514: Terms are no
> longer
> -  required to be character based. Lucene views a term as an arbitrary
> byte[]:
> -  during analysis, character-based terms are converted to UTF8 byte[],
> -  but analyzers are free to directly create terms as byte[]
> -  (NumericField does this, for example).  The term data is buffered as
> -  byte[] during indexing, written as byte[] into the terms dictionary,
> -  and iterated as byte[] (wrapped in a BytesRef) by IndexReader for
> -  searching.
> -
> -* LUCENE-1458, LUCENE-2111: IndexReader now directly exposes its
> -  deleted docs (getDeletedDocs), providing a new Bits interface to
> -  directly query by doc ID.
> -
> -* LUCENE-2691: IndexWriter.getReader() has been made package local and
> is now
> -  exposed via open and reopen methods on IndexReader.  The semantics of
> the
> -  call is the same as it was prior to the API change.
> -  (Grant Ingersoll, Mike McCandless)
> -
> -* LUCENE-2566: QueryParser: Unary operators +,-,! will not be treated as
> -  operators if they are followed by whitespace. (yonik)
> -
> -* LUCENE-2831: Weight#scorer, Weight#explain, Filter#getDocIdSet,
> -  Collector#setNextReader & FieldComparator#setNextReader now expect an
> -  AtomicReaderContext instead of an IndexReader. (Simon Willnauer)
> -
> -* LUCENE-2846: Remove the deprecated IndexReader.setNorm(int, String,
> float).
> -  This method was only syntactic sugar for setNorm(int, String, byte),
> but
> -  using the global Similarity.getDefault().encodeNormValue.  Use the
> byte-based
> -  method instead to ensure that the norm is encoded with your
> Similarity.
> -  Also removed norms(String, byte[], int), which was only used by
> MultiReader
> -  for building top-level norms. If you really need a top-level norms,
> use
> -  MultiNorms or SlowMultiReaderWrapper. (Robert Muir, Mike Mccandless)
> -
> -* LUCENE-2892: Add QueryParser.newFieldQuery (called by getFieldQuery by
> default)
> -  which takes Analyzer as a parameter, for easier customization by
> subclasses.
> -  (Robert Muir)
> -
> -* LUCENE-2953: In addition to changes in 3.x,
> PriorityQueue#initialize(int)
> -  function was moved into the ctor. (Uwe Schindler, Yonik Seeley)
> -
> -New features
> -
> -* LUCENE-2604: Added RegexpQuery support to QueryParser. Regular
> expressions
> -  are directly supported by the standard queryparser.
> -  (Simon Willnauer, Robert Muir)
> -
> -* LUCENE-1606, LUCENE-2089: Adds AutomatonQuery, a MultiTermQuery that
> -  matches terms against a finite-state machine. Implement WildcardQuery
> -  and FuzzyQuery with finite-state methods. Adds RegexpQuery.
> -  (Robert Muir, Mike McCandless, Uwe Schindler, Mark Miller)
> -
> -* LUCENE-1990: Adds internal packed ints implementation, to be used
> -  for more efficient storage of int arrays when the values are
> -  bounded, for example for storing the terms dict index (Toke
> -  Eskildsen via Mike McCandless)
> -
> -* LUCENE-2321: Cutover to a more RAM efficient packed-ints based
> -  representation for the in-memory terms dict index.  (Mike
> -  McCandless)
> -
> -* LUCENE-2126: Add new classes for data (de)serialization: DataInput
> -  and DataOutput.  IndexInput and IndexOutput extend these new classes.
> -  (Michael Busch)
> -
> -* LUCENE-1458, LUCENE-2111: With flexible indexing it is now possible
> -  for an application to create its own postings codec, to alter how
> -  fields, terms, docs and positions are encoded into the index.  The
> -  standard codec is the default codec.  Both IndexWriter and
> -  IndexReader accept a CodecProvider class to obtain codecs for newly
> -  written segments as well as existing segments opened for reading.
> -
> -* LUCENE-1458, LUCENE-2111: Some experimental codecs have been added
> -  for flexible indexing, including pulsing codec (inlines
> -  low-frequency terms directly into the terms dict, avoiding seeking
> -  for some queries), sep codec (stores docs, freqs, positions, skip
> -  data and payloads in 5 separate files instead of the 2 used by
> -  standard codec), and int block (really a "base" for using
> -  block-based compressors like PForDelta for storing postings data).
> -
> -* LUCENE-1458, LUCENE-2111: The in-memory terms index used by standard
> -  codec is more RAM efficient: terms data is stored as block byte
> -  arrays and packed integers.  Net RAM reduction for indexes that have
> -  many unique terms should be substantial, and initial open time for
> -  IndexReaders should be faster.  These gains only apply for newly
> -  written segments after upgrading.
> -
> -* LUCENE-1458, LUCENE-2111: Terms data are now buffered directly as
> -  byte[] during indexing, which uses half the RAM for ascii terms (and
> -  also numeric fields).  This can improve indexing throughput for
> -  applications that have many unique terms, since it reduces how often
> -  a new segment must be flushed given a fixed RAM buffer size.
> -
> -* LUCENE-2489: Added PerFieldCodecWrapper (in oal.index.codecs) which
> -  lets you set the Codec per field (Mike McCandless)
> -
> -* LUCENE-2373: Extend CodecProvider to use SegmentInfosWriter and
> -  SegmentInfosReader to allow customization of SegmentInfos data.
> -  (Andrzej Bialecki)
> -
> -* LUCENE-2504: FieldComparator.setNextReader now returns a
> -  FieldComparator instance.  You can "return this", to just reuse the
> -  same instance, or you can return a comparator optimized to the new
> -  segment.  (yonik, Mike McCandless)
> -
> -* LUCENE-2648: PackedInts.Iterator now supports to advance by more than
> a
> -  single ordinal. (Simon Willnauer)
> -
> -* LUCENE-2649: Objects in the FieldCache can optionally store Bits
> -  that mark which docs have real values in the native[] (ryan)
> -
> -* LUCENE-2664: Add SimpleText codec, which stores all terms/postings
> -  data in a single text file for transparency (at the expense of poor
> -  performance).  (Sahin Buyrukbilen via Mike McCandless)
> -
> -* LUCENE-2589: Add a VariableSizedIntIndexInput, which, when used w/
> -  Sep*, makes it simple to take any variable sized int block coders
> -  (like Simple9/16) and use them in a codec.  (Mike McCandless)
> -
> -* LUCENE-2597: Add oal.index.SlowMultiReaderWrapper, to wrap a
> -  composite reader (eg MultiReader or DirectoryReader), making it
> -  pretend it's an atomic reader.  This is a convenience class (you can
> -  use MultiFields static methods directly, instead) if you need to use
> -  the flex APIs directly on a composite reader.  (Mike McCandless)
> -
> -* LUCENE-2690: MultiTermQuery boolean rewrites per segment.
> -  (Uwe Schindler, Robert Muir, Mike McCandless, Simon Willnauer)
> -
> -* LUCENE-996: The QueryParser now accepts mixed inclusive and exclusive
> -  bounds for range queries.  Example: "{3 TO 5]"
> -  QueryParser subclasses that overrode getRangeQuery will need to be
> changed
> -  to use the new getRangeQuery method.  (Andrew Schurman, Mark Miller,
> yonik)
> -
> -* LUCENE-995: The QueryParser now interprets * as an open end for range
> -  queries.  Literal asterisks may be represented by quoting or escaping
> -  (i.e. \* or "*")  Custom QueryParser subclasses overriding
> getRangeQuery()
> -  will be passed null for any open endpoint. (Adriano Crestani, yonik)
> -
> -* LUCENE-2742: Add native per-field codec support. CodecProvider lets
> you now
> -  register a codec for each field and which is in turn recorded in the
> segment
> -  and field information. Codecs are maintained on a per-segment basis
> and be
> -  resolved without knowing the actual codec used for writing the
> segment.
> -  (Simon Willnauer)
> -
> -* LUCENE-2741: Add support for multiple codecs that use the same file
> -  extensions within the same segment. Codecs now use their per-segment
> codec
> -  ID in the file names. (Simon Willnauer)
> -
> -* LUCENE-2843: Added a new terms index impl,
> -  VariableGapTermsIndexWriter/Reader, that accepts a pluggable
> -  IndexTermSelector for picking which terms should be indexed in the
> -  terms dict.  This impl stores the indexed terms in an FST, which is
> -  much more RAM efficient than FixedGapTermsIndex.  (Mike McCandless)
> -
> -* LUCENE-2862: Added TermsEnum.totalTermFreq() and
> -  Terms.getSumTotalTermFreq().  (Mike McCandless, Robert Muir)
> -
> -* LUCENE-3003: Added new expert class oal.index.DocTermsOrd,
> -  refactored from Solr's UnInvertedField, for accessing term ords for
> -  multi-valued fields, per document.  This is similar to FieldCache in
> -  that it inverts the index to compute the ords, but differs in that
> -  it's able to handle multi-valued fields and does not hold the term
> -  bytes in RAM. (Mike McCandless)
> -
> -* LUCENE-3108, LUCENE-2935, LUCENE-2168, LUCENE-1231: Changes from
> -  IndexDocValues (ColumnStrideFields):
> -
> -  - IndexWriter now supports typesafe dense per-document values stored
> in
> -    a column like storage. IndexDocValues are stored on a per-document
> -    basis where each documents field can hold exactly one value of a
> given
> -    type. IndexDocValues are provided via Fieldable and can be used in
> -    conjunction with stored and indexed values.
> -
> -  - IndexDocValues provides an entirely RAM resident document id to
> value
> -    mapping per field as well as a DocIdSetIterator based disk-resident
> -    sequential access API relying on filesystem-caches.
> -
> -  - Both APIs are exposed via IndexReader and the Codec / Flex API
> allowing
> -    expert users to integrate customized IndexDocValues reader and
> writer
> -    implementations by extending existing Codecs.
> -
> -  - IndexDocValues provides implementations for primitive datatypes like
> int,
> -    long, float, double and arrays of byte. Byte based implementations
> further
> -    provide storage variants like straight or dereferenced stored bytes,
> fixed
> -    and variable length bytes as well as index time sorted based on
> user-provided
> -    comparators.
> -
> -  (Mike McCandless, Simon Willnauer)
> -
> -Optimizations
> -
> -* LUCENE-2588: Don't store unnecessary suffixes when writing the terms
> -  index, saving RAM in IndexReader; change default terms index
> -  interval from 128 to 32, because the terms index now requires much
> -  less RAM.  (Robert Muir, Mike McCandless)
> -
> -* LUCENE-2669: Optimize NumericRangeQuery.NumericRangeTermsEnum to
> -  not seek backwards when a sub-range has no terms. It now only seeks
> -  when the current term is less than the next sub-range's lower end.
> -  (Uwe Schindler, Mike McCandless)
> -
> -* LUCENE-2694: Optimize MultiTermQuery to be single pass for Term
> lookups.
> -  MultiTermQuery now stores TermState per leaf reader during rewrite to
> re-
> -  seek the term dictionary in TermQuery / TermWeight.
> -  (Simon Willnauer, Mike McCandless, Robert Muir)
> -
> -* LUCENE-2897: Apply deleted terms while flushing a segment.  We still
> -  buffer deleted terms to later apply to past segments.  (Mike
> McCandless)
> -
> -* LUCENE-1736: DateTools.java general improvements.
> -  (David Smiley via Steve Rowe)
> -
> -Bug fixes
> -
> -* LUCENE-2633: PackedInts Packed32 and Packed64 did not support internal
> -  structures larger than 256MB (Toke Eskildsen via Mike McCandless)
> -
> -* LUCENE-2803: The FieldCache can miss values if an entry for a reader
> -  with more document deletions is requested before a reader with fewer
> -  deletions, provided they share some segments. (yonik)
> -
> -* LUCENE-2645: Fix false assertion error when same token was added one
> -  after another with 0 posIncr.  (David Smiley, Kurosaka Teruhiko via
> Mike
> -  McCandless)
> -
>  ======================= Lucene 3.x (not yet released) ================
>
>  Changes in backwards compatibility policy
> @@ -515,6 +53,11 @@ Bug fixes
>    ArrayIndexOutOfBoundsException (selckin, Robert Muir, Mike
>    McCandless)
>
> +Optimizations
> +
> +* LUCENE-1736: DateTools.java general improvements.
> +  (David Smiley via Steve Rowe)
> +
>  New Features
>
>  * LUCENE-3140: Added experimental FST implementation to Lucene.
>