You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by si...@apache.org on 2011/02/09 10:36:03 UTC
svn commit: r1068809 [23/36] - in /lucene/dev/branches/docvalues: ./
dev-tools/eclipse/ dev-tools/idea/.idea/ dev-tools/idea/.idea/copyright/
dev-tools/idea/lucene/ dev-tools/idea/lucene/contrib/ant/
dev-tools/idea/lucene/contrib/queryparser/ dev-tools...
Modified: lucene/dev/branches/docvalues/solr/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/CHANGES.txt?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/CHANGES.txt (original)
+++ lucene/dev/branches/docvalues/solr/CHANGES.txt Wed Feb 9 09:35:27 2011
@@ -26,10 +26,201 @@ Versions of Major Components
---------------------
Apache Lucene trunk
Apache Tika 0.8-SNAPSHOT
-Carrot2 3.1.0
+Carrot2 3.4.2
Velocity 1.6.4 and Velocity Tools 2.0
+Upgrading from Solr 3.1-dev
+----------------------
+
+* The Lucene index format has changed and as a result, once you upgrade,
+ previous versions of Solr will no longer be able to read your indices.
+ In a master/slave configuration, all searchers/slaves should be upgraded
+ before the master. If the master were to be updated first, the older
+ searchers would not be able to read the new index format.
+
+* Setting abortOnConfigurationError=false is no longer supported
+ (since it has never worked properly). Solr will now warn you if
+ you attempt to set this configuration option at all. (see SOLR-1846)
+
+* The default logic for the 'mm' param of the 'dismax' QParser has
+ been changed. If no 'mm' param is specified (either in the query,
+ or as a default in solrconfig.xml) then the effective value of the
+ 'q.op' param (either in the query or as a default in solrconfig.xml
+ or from the 'defaultOperator' option in schema.xml) is used to
+ influence the behavior. If q.op is effectively "AND" then mm=100%.
+ If q.op is effectively "OR" then mm=0%. Users who wish to force the
+ legacy behavior should set a default value for the 'mm' param in
+ their solrconfig.xml file.
+
+* In previous releases, sorting on fields that are "multiValued"
+ (either by explicit declaration in schema.xml or by implict behavior
+ because the "version" attribute on the schema was less then 1.2) did
+ not generally work, but it would sometimes silently act as if it
+ succeeded and order the docs arbitrarily. Solr will now fail on any
+ attempt to sort on a multivalued field
+
+
+
+Detailed Change List
+----------------------
+
+New Features
+----------------------
+
+* SOLR-571: The autowarmCount for LRUCaches (LRUCache and FastLRUCache) now
+ supports "percentages" which get evaluated relative the current size of
+ the cache when warming happens.
+ (Tomas Fernandez Lobbe and hossman)
+
+* SOLR-1915: DebugComponent now supports using a NamedList to model
+ Explanation objects in it's responses instead of
+ Explanation.toString (hossman)
+
+* SOLR-1932: New relevancy function queries: termfreq, tf, docfreq, idf
+ norm, maxdoc, numdocs. (yonik)
+
+* SOLR-1682: (SOLR-236, SOLR-237, SOLR-1773, SOLR-1311) Search grouping / Field collapsing.
+ (Martijn van Groningen, Emmanuel Keller, Shalin Shekhar Mangar,
+ Koji Sekiguchi, Iv�n de Prado, Ryan McKinley, Marc Sturlese, Peter Karich,
+ Bojan Smid, Charles Hornberger, Dieter Grad, Dmitry Lihachev, Doug Steigerwald,
+ Karsten Sperling, Michael Gundlach, Oleg Gnatovskiy, Thomas Traeger,
+ Harish Agarwal, yonik)
+
+* SOLR-1665: Add debug component options for timings, results and query info only (gsingers, hossman, yonik)
+
+* SOLR-2113: Add TermQParserPlugin, registered as "term". This is useful
+ when generating filter queries from terms returned from field faceting or
+ the terms component. Example: fq={!term f=weight}1.5 (hossman, yonik)
+
+* SOLR-2001: The query component will substitute an empty query that matches
+ no documents if the query parser returns null. This also prevents an
+ exception from being thrown by the default parser if "q" is missing. (yonik)
+
+* SOLR-2112: Solrj API now supports streaming results. (ryan)
+
+* SOLR-792: Adding PivotFacetComponent for Hierarchical faceting
+ (erik, Jeremy Hinegardner, Thibaut Lassalle, ryan)
+
+* LUCENE-2507: Added DirectSolrSpellChecker, which uses Lucene's DirectSpellChecker
+ to retrieve correction candidates directly from the term dictionary using
+ levenshtein automata. (rmuir)
+
+* SOLR-1873: SolrCloud - added shared/central config and core/shard managment via zookeeper,
+ built-in load balancing, and infrastructure for future SolrCloud work.
+ (yonik, Mark Miller)
+
+* SOLR-1729: Evaluation of NOW for date math is done only once per request for
+ consistency, and is also propagated to shards in distributed search.
+ Adding a parameter NOW=<time_in_ms> to the request will override the
+ current time. (Peter Sturge, yonik)
+
+Optimizations
+----------------------
+
+* SOLR-1875: Per-segment field faceting for single valued string fields.
+ Enable with facet.method=fcs, control the number of threads used with
+ the "threads" local param on the facet.field param. This algorithm will
+ only be faster in the presence of rapid index changes. (yonik)
+
+* SOLR-1904: When facet.enum.cache.minDf > 0 and the base doc set is a
+ SortedIntSet, convert to HashDocSet for better performance. (yonik)
+
+* SOLR-1843: A new "rootName" attribute is now available when
+ configuring <jmx/> in solrconfig.xml. If this attribute is set,
+ Solr will use it as the root name for all MBeans Solr exposes via
+ JMX. The default root name is "solr" followed by the core name.
+ (Constantijn Visinescu, hossman)
+
+* SOLR-2092: Speed up single-valued and multi-valued "fc" faceting. Typical
+ improvement is 5%, but can be much greater (up to 10x faster) when facet.offset
+ is very large (deep paging). (yonik)
+
+Bug Fixes
+----------------------
+
+* SOLR-1908: Fixed SignatureUpdateProcessor to fail to initialize on
+ invalid config. Specificly: a signatureField that does not exist,
+ or overwriteDupes=true with a signatureField that is not indexed.
+ (hossman)
+
+* SOLR-1824: IndexSchema will now fail to initialize if there is a
+ problem initializing one of the fields or field types. (hossman)
+
+* SOLR-1928: TermsComponent didn't correctly break ties for non-text
+ fields sorted by count. (yonik)
+
+* SOLR-2107: MoreLikeThisHandler doesn't work with alternate qparsers. (yonik)
+
+* SOLR-2108: Fixed false positives when using wildcard queries on fields with reversed
+ wildcard support. For example, a query of *zemog* would match documents that contain
+ 'gomez'. (Landon Kuhn via Robert Muir)
+
+* SOLR-1962: SolrCore#initIndex should not use a mix of indexPath and newIndexPath (Mark Miller)
+
+* SOLR-2275: fix DisMax 'mm' parsing to be tolerant of whitespace
+ (Erick Erickson via hossman)
+
+* SOLR-2307: fix bug in PHPSerializedResponseWriter (wt=phps) when
+ dealing with SolrDocumentList objects -- ie: sharded queries.
+ (Antonio Verni via hossman)
+
+* SOLR-2127: Fixed serialization of default core and indentation of solr.xml when serializing.
+ (Ephraim Ofir, Mark Miller)
+
+* SOLR-482: Provide more exception handling in CSVLoader (gsingers)
+
+* SOLR-2320: Fixed ReplicationHandler detail reporting for masters
+ (hossman)
+
+* SOLR-2085: Improve SolrJ behavior when FacetComponent comes before
+ QueryComponent (Tomas Salfischberger via hossman)
+
+* SOLR-1940: Fix SolrDispatchFilter behavior when Content-Type is
+ unknown (Lance Norskog and hossman)
+
+* SOLR-2339: Fix sorting to explicitly generate an error if you
+ attempt to sort on a multiValued field. (hossman)
+
+Other Changes
+----------------------
+
+* SOLR-1846: Eliminate support for the abortOnConfigurationError
+ option. It has never worked very well, and in recent versions of
+ Solr hasn't worked at all. (hossman)
+
+* SOLR-1889: The default logic for the 'mm' param of DismaxQParser and
+ ExtendedDismaxQParser has been changed to be determined based on the
+ effective value of the 'q.op' param (hossman)
+
+* SOLR-1946: Misc improvements to the SystemInfoHandler: /admin/system
+ (hossman)
+
+* SOLR-2289: Tweak spatial coords for example docs so they are a bit
+ more spread out (Erick Erickson via hossman)
+
+* SOLR-2288: Small tweaks to eliminate compiler warnings. primarily
+ using Generics where applicable in method/object declatations, and
+ adding @SuppressWarnings("unchecked") when appropriate (hossman)
+
+* SOLR-2350: Since Solr no longer requires XML files to be in UTF-8
+ (see SOLR-96) SimplePostTool (aka: post.jar) has been improved to
+ work with files of any mime-type or charset. (hossman)
+
+Documentation
+----------------------
+
+* SOLR-2232: Improved README info on solr.solr.home in examples
+ (Eric Pugh and hossman)
+
+================== 3.1.0-dev ==================
+Versions of Major Components
+---------------------
+Apache Lucene trunk
+Apache Tika 0.8-SNAPSHOT
+Carrot2 3.4.2
+
+
Upgrading from Solr 1.4
----------------------
@@ -66,25 +257,10 @@ Upgrading from Solr 1.4
* SOLR-1876: All Analyzers and TokenStreams are now final to enforce
the decorator pattern. (rmuir, uschindler)
-* Setting abortOnConfigurationError=false is no longer supported
- (since it has never worked properly). Solr will now warn you if
- you attempt to set this configuration option at all. (see SOLR-1846)
-
-* The default logic for the 'mm' param of the 'dismax' QParser has
- been changed. If no 'mm' param is specified (either in the query,
- or as a default in solrconfig.xml) then the effective value of the
- 'q.op' param (either in the query or as a default in solrconfig.xml
- or from the 'defaultOperator' option in schema.xml) is used to
- influence the behavior. If q.op is effectively "AND" then mm=100%.
- If q.op is effectively "OR" then mm=0%. Users who wish to force the
- legacy behavior should set a default value for the 'mm' param in
- their solrconfig.xml file.
-
* LUCENE-2608: Added the ability to specify the accuracy on a per request basis.
- Implementations of SolrSpellChecker must change over to the new SolrSpellChecker
- abstract methods using the new SpellingOptions class. While this change is not
- backward compatible, it should be trivial to migrate as the SpellingOptions class
- just encapsulates the parameters that were passed in to the methods before the change. (gsingers)
+ It is recommended that implementations of SolrSpellChecker should change over to the new SolrSpellChecker
+ methods using the new SpellingOptions class, but are not required to. While this change is
+ backward compatible, the trunk version of Solr has already dropped support for all but the SpellingOptions method. (gsingers)
* readercycle script was removed. (SOLR-2046)
@@ -95,10 +271,10 @@ New Features
----------------------
* SOLR-1302: Added several new distance based functions, including
- Great Circle (haversine) for geo search, Manhattan, Euclidean
- and String (using the StringDistance methods in the Lucene spellchecker).
+ Great Circle (haversine), Manhattan, Euclidean and String (using the
+ StringDistance methods in the Lucene spellchecker).
Also added geohash(), deg() and rad() convenience functions.
- See http://wiki.apache.org/solr/FunctionQuery. (yonik, gsingers)
+ See http://wiki.apache.org/solr/FunctionQuery. (gsingers)
* SOLR-1553: New dismax parser implementation (accessible as "edismax")
that supports full lucene syntax, improved reserved char escaping,
@@ -130,12 +306,10 @@ New Features
* SOLR-1653: Add PatternReplaceCharFilter (koji)
-* SOLR-1131: FieldTypes can now output multiple Fields per Type and still be
- searched. This can be handy for hiding the details of a particular
- implementation such as in the spatial case.
- (Chris Mattmann, shalin, noble, gsingers, yonik)
+* SOLR-1131: FieldTypes can now output multiple Fields per Type and still be searched. This can be handy for hiding the details of a particular
+ implementation such as in the spatial case. (Chris Mattmann, shalin, noble, gsingers, yonik)
-* SOLR-1586: Add support for Geohash FieldType (Chris Mattmann, gsingers)
+* SOLR-1586: Add support for Geohash and Spatial Tile FieldType (Chris Mattmann, gsingers)
* SOLR-1697: PluginInfo should load plugins w/o class attribute also (noble)
@@ -167,41 +341,18 @@ New Features
* SOLR-1740: ShingleFilterFactory supports the "minShingleSize" and "tokenSeparator"
parameters for controlling the minimum shingle size produced by the filter, and
the separator string that it uses, respectively. (Steven Rowe via rmuir)
-
+
* SOLR-744: ShingleFilterFactory supports the "outputUnigramsIfNoShingles"
parameter, to output unigrams if the number of input tokens is fewer than
minShingleSize, and no shingles can be generated.
(Chris Harris via Steven Rowe)
-
-* SOLR-571: The autowarmCount for LRUCaches (LRUCache and FastLRUCache) now
- supports "percentages" which get evaluated relative the current size of
- the cache when warming happens.
- (Tomas Fernandez Lobbe and hossman)
-
-* SOLR-397: Date Faceting now supports a "facet.date.include" param
- for specifying when the upper & lower end points of computed date
- ranges should be included in the range. Legal values are: "all",
- "lower", "upper", "edge", and "outer". For backwards compatibility
- the default value is the set: [lower,upper,edge], so that al ranges
- between start and ed are inclusive of their endpoints, but the
- "before" and "after" ranges are not.
-
-* SOLR-945: JSON update handler that accepts add, delete, commit
- commands in JSON format. (Ryan McKinley, yonik)
* SOLR-1923: PhoneticFilterFactory now has support for the
Caverphone algorithm. (rmuir)
-
-* SOLR-1915: DebugComponent now supports using a NamedList to model
- Explanation objects in it's responses instead of
- Explanation.toString (hossman)
-* SOLR-1932: New relevancy function queries: termfreq, tf, docfreq, idf
- norm, maxdoc, numdocs. (yonik)
-
* SOLR-1957: The VelocityResponseWriter contrib moved to core.
- Example search UI now available at http://localhost:8983/solr/browse
- (ehatcher)
+ Example search UI now available at http://localhost:8983/solr/browse
+ (ehatcher)
* SOLR-1974: Add LimitTokenCountFilterFactory. (koji)
@@ -217,11 +368,17 @@ New Features
* SOLR-1984: Add HyphenationCompoundWordTokenFilterFactory. (PB via rmuir)
-* SOLR-1568: Added native filtering support via geofilt for spatial field
- types LatLonType, PointType, GeohashField.
- See http://wiki.apache.org/solr/SpatialSearch for examples.
- (yonik, gsingers)
-
+* SOLR-397: Date Faceting now supports a "facet.date.include" param
+ for specifying when the upper & lower end points of computed date
+ ranges should be included in the range. Legal values are: "all",
+ "lower", "upper", "edge", and "outer". For backwards compatibility
+ the default value is the set: [lower,upper,edge], so that al ranges
+ between start and ed are inclusive of their endpoints, but the
+ "before" and "after" ranges are not.
+
+* SOLR-945: JSON update handler that accepts add, delete, commit
+ commands in JSON format. (Ryan McKinley, yonik)
+
* SOLR-2015: Add a boolean attribute autoGeneratePhraseQueries to TextField.
autoGeneratePhraseQueries="true" (the default) causes the query parser to
generate phrase queries if multiple tokens are generated from a single
@@ -245,77 +402,50 @@ New Features
* SOLR-2053: Add support for custom comparators in Solr spellchecker, per LUCENE-2479 (gsingers)
-* SOLR-1682: (SOLR-236, SOLR-237, SOLR-1773, SOLR-1311) Search grouping / Field collapsing.
- (Martijn van Groningen, Emmanuel Keller, Shalin Shekhar Mangar,
- Koji Sekiguchi, Iv�n de Prado, Ryan McKinley, Marc Sturlese, Peter Karich,
- Bojan Smid, Charles Hornberger, Dieter Grad, Dmitry Lihachev, Doug Steigerwald,
- Karsten Sperling, Michael Gundlach, Oleg Gnatovskiy, Thomas Traeger,
- Harish Agarwal, yonik)
-
-* SOLR-1316: Create autosuggest component.
- (Ankul Garg, Jason Rutherglen, Shalin Shekhar Mangar, gsingers, Robert Muir, ab)
+* SOLR-2049: Add hl.multiValuedSeparatorChar for FastVectorHighlighter, per LUCENE-2603. (koji)
* SOLR-2059: Add "types" attribute to WordDelimiterFilterFactory, which
allows you to customize how WordDelimiterFilter tokenizes text with
a configuration file. (Peter Karich, rmuir)
-* SOLR-1665: Add debug component options for timings, results and query info only (gsingers, hossman, yonik)
-
* SOLR-2099: Add ability to throttle rsync based replication using rsync option --bwlimit.
(Brandon Evans via koji)
-* SOLR-2113: Add TermQParserPlugin, registered as "term". This is useful
- when generating filter queries from terms returned from field faceting or
- the terms component. Example: fq={!term f=weight}1.5 (hossman, yonik)
-
-* SOLR-2001: The query component will substitute an empty query that matches
- no documents if the query parser returns null. This also prevents an
- exception from being thrown by the default parser if "q" is missing. (yonik)
-
-* SOLR-2112: Solrj API now supports streaming results. (ryan)
+* SOLR-1316: Create autosuggest component.
+ (Ankul Garg, Jason Rutherglen, Shalin Shekhar Mangar, Grant Ingersoll, Robert Muir, ab)
-* SOLR-792: Adding PivotFacetComponent for Hierarchical faceting
- (erik, Jeremy Hinegardner, Thibaut Lassalle, ryan)
+* SOLR-1568: Added "native" filtering support for PointType, GeohashField. Added LatLonType with filtering support too. See
+ http://wiki.apache.org/solr/SpatialSearch and the example. Refactored some items in Lucene spatial.
+ Removed SpatialTileField as the underlying CartesianTier is broken beyond repair and is going to be moved. (gsingers)
* SOLR-2128: Full parameter substitution for function queries.
Example: q=add($v1,$v2)&v1=mul(popularity,5)&v2=20.0
(yonik)
-* SOLR-2133: Function query parser can now parse multiple comma separated
+* SOLR-2133: Function query parser can now parse multiple coma separated
value sources. It also now fails if there is extra unexpected text
after parsing the functions, instead of silently ignoring it.
This allows expressions like q=dist(2,vector(1,2),$pt)&pt=3,4 (yonik)
-* LUCENE-2507: Added DirectSolrSpellChecker, which uses Lucene's DirectSpellChecker
- to retrieve correction candidates directly from the term dictionary using
- levenshtein automata. (rmuir)
+* SOLR-2157: Suggester should return alpha-sorted results when onlyMorePopular=false (ab)
* SOLR-2010: Added ability to verify that spell checking collations have
actual results in the index. (James Dyer via gsingers)
-
-* SOLR-1873: SolrCloud - added shared/central config and core/shard managment via zookeeper,
- built-in load balancing, and infrastructure for future SolrCloud work.
- (yonik, Mark Miller)
-* SOLR-2210: Add icu-based tokenizer and filters to contrib/analysis-extras (rmuir)
+* SOLR-2188: Added "maxTokenLength" argument to the factories for ClassicTokenizer,
+ StandardTokenizer, and UAX29URLEmailTokenizer. (Steven Rowe)
-* SOLR-1336: Add SmartChinese (word segmentation for Simplified Chinese)
- tokenizer and filters to contrib/analysis-extras (rmuir)
+* SOLR-2129: Added a Solr module for dynamic metadata extraction/indexing with Apache UIMA.
+ See contrib/uima/README.txt for more information. (Tommaso Teofili via rmuir)
-* SOLR-2211,LUCENE-2763: Added UAX29URLEmailTokenizerFactory, which implements
- UAX#29, a unicode algorithm with good results for most languages, as well as
- URL and E-mail tokenization according to the relevant RFCs.
- (Tom Burton-West via rmuir)
+* SOLR-2325: Allow tagging and exlcusion of main query for faceting. (yonik)
-* SOLR-2237: Added StempelPolishStemFilterFactory to contrib/analysis-extras (rmuir)
+* SOLR-2263: Add ability for RawResponseWriter to stream binary files as well as
+ text files. (Eric Pugh via yonik)
-* SOLR-1729: Evaluation of NOW for date math is done only once per request for
- consistency, and is also propagated to shards in distributed search.
- Adding a parameter NOW=<time_in_ms> to the request will override the
- current time. (Peter Sturge, yonik)
+* SOLR-860: Add debug output for MoreLikeThis. (koji)
-* SOLR-2188: Added "maxTokenLength" argument to the factories for ClassicTokenizer,
- StandardTokenizer, and UAX29URLEmailTokenizer. (Steven Rowe)
+* SOLR-1057: Add PathHierarchyTokenizerFactory. (ryan, koji)
Optimizations
----------------------
@@ -326,39 +456,17 @@ Optimizations
* SOLR-1874: Optimize PatternReplaceFilter for better performance. (rmuir, uschindler)
-* SOLR-1875: Per-segment field faceting for single valued string fields.
- Enable with facet.method=fcs, control the number of threads used with
- the "threads" local param on the facet.field param. This algorithm will
- only be faster in the presence of rapid index changes. (yonik)
-
-* SOLR-1904: When facet.enum.cache.minDf > 0 and the base doc set is a
- SortedIntSet, convert to HashDocSet for better performance. (yonik)
-
-* SOLR-1843: A new "rootName" attribute is now available when
- configuring <jmx/> in solrconfig.xml. If this attribute is set,
- Solr will use it as the root name for all MBeans Solr exposes via
- JMX. The default root name is "solr" followed by the core name.
- (Constantijn Visinescu, hossman)
-
* SOLR-1968: speed up initial filter cache population for facet.method=enum and
also big terms for multi-valued facet.method=fc. The resulting speedup
for the first facet request is anywhere from 30% to 32x, depending on how many
- terms are in the field and how many documents match per term. (yonik)
+ terms are in the field and how many documents match per term. (yonik)
* SOLR-2089: Speed up UnInvertedField faceting (facet.method=fc for
multi-valued fields) when facet.limit is both high, and a high enough
percentage of the number of unique terms in the field. Extreme cases
yield speedups over 3x. (yonik)
-* SOLR-2046: Simplify legacy replication scripts by adding common functions
- to scripts-util. (koji)
-
-* SOLR-2092: Speed up single-valued and multi-valued "fc" faceting. Typical
- improvement is 5%, but can be much greater (up to 10x faster) when facet.offset
- is very large (deep paging). (yonik)
-
-* SOLR-2200: Improve the performance of DataImportHandler for large delta-import
- updates. (Mark Waddle via rmuir)
+* SOLR-2046: add common functions to scripts-util. (koji)
Bug Fixes
----------------------
@@ -436,8 +544,9 @@ Bug Fixes
* SOLR-1711: SolrJ - StreamingUpdateSolrServer had a race condition that
could halt the streaming of documents. The original patch to fix this
(never officially released) introduced another hanging bug due to
- connections not being released. (Attila Babo, Erik Hetzner via yonik)
-
+ connections not being released.
+ (Attila Babo, Erik Hetzner, Johannes Tuchscherer via yonik)
+
* SOLR-1748, SOLR-1747, SOLR-1746, SOLR-1745, SOLR-1744: Streams and Readers
retrieved from ContentStreams are not closed in various places, resulting
in file descriptor leaks.
@@ -470,19 +579,6 @@ Bug Fixes
* SOLR-1706: fixed WordDelimiterFilter for certain combinations of options
where it would output incorrect tokens. (Robert Muir, Chris Male)
-* SOLR-1902: Exposed SolrResourceLoader's class loader for use by Tika
-
-* SOLR-1908: Fixed SignatureUpdateProcessor to fail to initialize on
- invalid config. Specificly: a signatureField that does not exist,
- or overwriteDupes=true with a signatureField that is not indexed.
- (hossman)
-
-* SOLR-1824: IndexSchema will now fail to initialize if there is a
- problem initializing one of the fields or field types. (hossman)
-
-* SOLR-1928: TermsComponent didn't correctly break ties for non-text
- fields sorted by count. (yonik)
-
* SOLR-1936: The JSON response format needed to escape unicode code point
U+2028 - 'LINE SEPARATOR' (Robert Hofstra, yonik)
@@ -520,7 +616,9 @@ Bug Fixes
to be removed before it was finished being copied. This did not affect
normal master/slave replication. (Peter Sturge via yonik)
-* SOLR-2107: MoreLikeThisHandler doesn't work with alternate qparsers. (yonik)
+* SOLR-2114: Fixed parsing error in hsin function. The function signature has changed slightly. (gsingers)
+
+* SOLR-2083: SpellCheckComponent misreports suggestions when distributed (James Dyer via gsingers)
* SOLR-2111: Change exception handling in distributed faceting to work more
like non-distributed faceting, change facet_counts/exception from a String
@@ -532,28 +630,19 @@ Bug Fixes
substitution/dereferencing. Properly encode local params in
distributed faceting. (yonik)
-* SOLR-2083: SpellCheckComponent misreports suggestions when distributed (James Dyer via gsingers)
-
-* SOLR-2108: Fixed false positives when using wildcard queries on fields with reversed
- wildcard support. For example, a query of *zemog* would match documents that contain
- 'gomez'. (Landon Kuhn via Robert Muir)
-
* SOLR-2135: Fix behavior of ConcurrentLRUCache when asking for
getLatestAccessedItems(0) or getOldestAccessedItems(0).
(David Smiley via hossman)
* SOLR-2148: Highlighter doesn't support q.alt. (koji)
-* SOLR-2157 Suggester should return alpha-sorted results when onlyMorePopular=false (ab)
-
* SOLR-1794: Dataimport of CLOB fields fails when getCharacterStream() is
defined in a superclass. (Gunnar Gauslaa Bergem via rmuir)
-* SOLR-2173: Suggester should always rebuild Lookup data if Lookup.load fails. (ab)
-
-* SOLR-2190: change xpath from RSS 0.9 to 1.0 in slashdot sample. (koji)
+* SOLR-2180: It was possible for EmbeddedSolrServer to leave searchers
+ open if a request threw an exception. (yonik)
-* SOLR-1962: SolrCore#initIndex should not use a mix of indexPath and newIndexPath (Mark Miller)
+* SOLR-2173: Suggester should always rebuild Lookup data if Lookup.load fails. (ab)
* SOLR-2057: DataImportHandler never calls UpdateRequestProcessor.finish()
(Drew Farris via koji)
@@ -564,9 +653,10 @@ Bug Fixes
true/on/yes (for TRUE) and false/off/no (for FALSE) can be used for sub-options
(debug, verbose, synchronous, commit, clean, optimize) for full/delta-import commands. (koji)
-* SOLR-2252: When a child entity in nested entities is rootEntity="true", delta-import doesn't work.
- (koji)
-
+* SOLR-2081: BaseResponseWriter.isStreamingDocs causes
+ SingleResponseWriter.end to be called 2x
+ (Chris A. Mattmann via hossman)
+
* SOLR-2219: The init() method of every SolrRequestHandler was being
called twice. (ambikeshwar singh and hossman)
@@ -576,8 +666,18 @@ Bug Fixes
addresses "commitWithin" option on Update requests.
(noble, hossman, and Maxim Valyanskiy)
-* SOLR-2275: fix DisMax 'mm' parsing to be tolerant of whitespace
- (Erick Erickson via hossman)
+* SOLR-2261: fix velocity template layout.vm that referred to an older
+ version of jquery. (Eric Pugh via rmuir)
+
+* SOLR-1983: snappuller fails when modifiedConfFiles is not empty and
+ full copy of index is needed. (Alexander Kanarsky via yonik)
+
+* SOLR-2156: SnapPuller fails to clean Old Index Directories on Full Copy
+ (Jayendra Patil via yonik)
+
+* SOLR-96: Fix XML parsing in XMLUpdateRequestHandler and
+ DocumentAnalysisRequestHandler to respect charset from XML file and only
+ use HTTP header's "Content-Type" as a "hint". (Uwe Schindler)
Other Changes
----------------------
@@ -593,6 +693,8 @@ Other Changes
* SOLR-1592: Refactor XMLWriter startTag to allow arbitrary attributes to be written
(Chris A. Mattmann via noble)
+* SOLR-1561: Added Lucene 2.9.1 spatial contrib jar to lib. (gsingers)
+
* SOLR-1570: Log warnings if uniqueKey is multi-valued or not stored (hossman, shalin)
* SOLR-1558: QueryElevationComponent only works if the uniqueKey field is
@@ -645,29 +747,15 @@ Other Changes
"defaultSearchField" as the default value for the "qf" param instead of failing
with an error when "qf" is not specified. (hossman)
-* SOLR-1821: Fix TimeZone-dependent test failure in TestEvaluatorBag.
- (Chris Male via rmuir)
-
* SOLR-1851: luceneAutoCommit no longer has any effect - it has been remove (Mark Miller)
* SOLR-1865: SolrResourceLoader.getLines ignores Byte Order Markers (BOMs) at the
beginning of input files, these are often created by editors such as Windows
Notepad. (rmuir, hossman)
-* SOLR-1846: Eliminate support for the abortOnConfigurationError
- option. It has never worked very well, and in recent versions of
- Solr hasn't worked at all. (hossman)
-
* SOLR-1938: ElisionFilterFactory will use a default set of French contractions
- if you do not supply a custom articles file. (rmuir)
-
-* SOLR-1889: The default logic for the 'mm' param of DismaxQParser and
- ExtendedDismaxQParser has been changed to be determined based on the
- effective value of the 'q.op' param (hossman)
-
-* SOLR-1946: Misc improvements to the SystemInfoHandler: /admin/system
- (hossman)
-
+ if you do not supply a custom articles file. (rmuir)
+
* SOLR-2003: SolrResourceLoader will report any encoding errors, rather than
silently using replacement characters for invalid inputs (blargy via rmuir)
@@ -683,8 +771,12 @@ Other Changes
* SOLR-2213: Upgrade to jQuery 1.4.3 (Erick Erickson via ryan)
-* SOLR-2289: Tweak spatial coords for example docs so they are a bit
- more spread out (Erick Erickson via hossman)
+* SOLR-1826: Add unit tests for highlighting with termOffsets=true
+ and overlapping tokens. (Stefan Oestreicher via rmuir)
+
+* SOLR-2340: Add version infos to message in JavaBinCodec when throwing
+ exception. (koji)
+
Build
----------------------
@@ -698,6 +790,10 @@ Build
* SOLR-2042: Fixed some Maven deps (Drew Farris via gsingers)
+* LUCENE-2657: Switch from using Maven POM templates to full POMs when
+ generating Maven artifacts (Steven Rowe)
+
+
Documentation
----------------------
@@ -707,9 +803,6 @@ Documentation
* SOLR-1792: Documented peculiar behavior of TestHarness.LocalRequestFactory
(hossman)
-* SOLR-2232: Improved README info on solr.solr.home in examples
- (Eric Pugh and hossman)
-
================== Release 1.4.0 ==================
Release Date: See http://lucene.apache.org/solr for the official release date.
Modified: lucene/dev/branches/docvalues/solr/NOTICE.txt
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/NOTICE.txt?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/NOTICE.txt (original)
+++ lucene/dev/branches/docvalues/solr/NOTICE.txt Wed Feb 9 09:35:27 2011
@@ -156,7 +156,6 @@ This product includes software developed
See http://project.carrot2.org/
-
=========================================================================
== Guava Notice ==
=========================================================================
@@ -168,6 +167,16 @@ This product includes software developed
See http://code.google.com/p/guava-libraries/
=========================================================================
+== Prettify Notice ==
+=========================================================================
+
+Copyright ???? Google, Inc.
+
+This product includes software developed by the Google Prettify project.
+
+See http://code.google.com/p/google-code-prettify/
+
+=========================================================================
== Jackson Notice ==
=========================================================================
Copyright ????
Modified: lucene/dev/branches/docvalues/solr/build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/build.xml?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/build.xml (original)
+++ lucene/dev/branches/docvalues/solr/build.xml Wed Feb 9 09:35:27 2011
@@ -18,6 +18,8 @@
-->
<project name="solr" default="usage" basedir="." xmlns:artifact="antlib:org.apache.maven.artifact.ant">
+
+ <property name="solr-path" value="." />
<import file="common-build.xml"/>
@@ -56,7 +58,7 @@
<echo message="Use 'ant compile' to compile the source code." />
<echo message="Use 'ant dist' to build the project WAR and JAR files." />
<echo message="Use 'ant generate-maven-artifacts' to generate maven artifacts." />
- <echo message="Use 'ant package' to generate zip, tgz, and maven artifacts for distribution." />
+ <echo message="Use 'ant package' to generate zip, tgz for distribution." />
<echo message="Use 'ant luke' to start luke. see: http://luke.googlecode.com" />
<echo message="Use 'ant test' to run unit tests." />
</target>
@@ -216,15 +218,17 @@
<packageset dir="${src}/java" />
<packageset dir="${src}/webapp/src" />
<packageset dir="contrib/dataimporthandler/src/main/java" />
- <!--<packageset dir="contrib/clustering/src/main/java" />-->
+ <packageset dir="contrib/clustering/src/main/java" />
<packageset dir="contrib/extraction/src/main/java" />
+ <packageset dir="contrib/uima/src/main/java" />
<packageset dir="contrib/analysis-extras/src/java" />
<group title="Core" packages="org.apache.*" />
<group title="Common" packages="org.apache.solr.common.*" />
<group title="SolrJ" packages="org.apache.solr.client.solrj*" />
<group title="contrib: DataImportHandler" packages="org.apache.solr.handler.dataimport*" />
- <!--<group title="contrib: Clustering" packages="org.apache.solr.handler.clustering*" />-->
+ <group title="contrib: Clustering" packages="org.apache.solr.handler.clustering*" />
<group title="contrib: Solr Cell" packages="org.apache.solr.handler.extraction*" />
+ <group title="contrib: Solr UIMA" packages="org.apache.solr.uima*" />
</sources>
</invoke-javadoc>
</sequential>
@@ -338,7 +342,7 @@
<path refid="compile.classpath.solrj" />
<pathelement location="${dest}/solr"/>
<pathelement location="${dest}/solrj"/> <!-- include solrj -->
- <pathelement location="${common-solr.dir}/../lucene/build/classes/test" /> <!-- include some lucene test code -->
+ <pathelement location="${common-solr.dir}/../lucene/build/classes/test-framework" /> <!-- include some lucene test code -->
</path>
<path id="test.run.classpath">
@@ -346,7 +350,7 @@
<pathelement location="${dest}/tests"/>
<!-- include the solrj classpath and jetty files included in example -->
<path refid="compile.classpath.solrj" />
- <pathelement location="${common-solr.dir}/../lucene/build/classes/test" /> <!-- include some lucene test code -->
+ <pathelement location="${common-solr.dir}/../lucene/build/classes/test-framework" /> <!-- include some lucene test code -->
<pathelement path="${java.class.path}"/>
</path>
@@ -514,6 +518,7 @@
<fileset dir="contrib/dataimporthandler/src/main/java" />
<fileset dir="contrib/clustering/src/main/java" />
<fileset dir="contrib/extraction/src/main/java" />
+ <fileset dir="contrib/uima/src/main/java" />
<fileset dir="contrib/analysis-extras/src/java" />
</clover-setup>
</target>
@@ -613,10 +618,14 @@
<solr-jar destfile="${dist}/apache-solr-cell-src-${version}.jar"
basedir="contrib/extraction/src" />
- <!--<solr-jar destfile="${dist}/apache-solr-clustering-src-${version}.jar"
- basedir="contrib/clustering/src" />-->
+ <solr-jar destfile="${dist}/apache-solr-clustering-src-${version}.jar"
+ basedir="contrib/clustering/src" />
<solr-jar destfile="${dist}/apache-solr-analysis-extras-src-${version}.jar"
basedir="contrib/analysis-extras/src" />
+ <solr-jar destfile="${dist}/apache-solr-uima-src-${version}.jar"
+ basedir="contrib/uima/src/main/java" >
+ <fileset dir="contrib/uima/src/main/resources" />
+ </solr-jar>
</target>
<target name="dist-javadoc" description="Creates the Solr javadoc distribution files"
@@ -629,12 +638,14 @@
basedir="${build.javadoc}/solrj" />
<solr-jar destfile="${dist}/apache-solr-dataimporthandler-docs-${version}.jar"
basedir="${build.javadoc}/contrib-solr-dataimporthandler" />
- <!--<solr-jar destfile="${dist}/apache-solr-clustering-docs-${version}.jar"
- basedir="${build.javadoc}/contrib-solr-clustering" />-->
+ <solr-jar destfile="${dist}/apache-solr-clustering-docs-${version}.jar"
+ basedir="${build.javadoc}/contrib-solr-clustering" />
<solr-jar destfile="${dist}/apache-solr-cell-docs-${version}.jar"
basedir="${build.javadoc}/contrib-solr-cell" />
<solr-jar destfile="${dist}/apache-solr-analysis-extras-docs-${version}.jar"
basedir="${build.javadoc}/contrib-solr-analysis-extras" />
+ <solr-jar destfile="${dist}/apache-solr-uima-docs-${version}.jar"
+ basedir="${build.javadoc}/contrib-solr-uima" />
</target>
<!-- Creates the solr jar. -->
@@ -714,7 +725,7 @@
</target>
<!-- make a distribution -->
- <target name="package" depends="generate-maven-artifacts"/>
+ <target name="package" depends="create-package"/>
<target name="create-package"
description="Packages the Solr Distribution files and Documentation."
@@ -731,7 +742,7 @@
<tarfileset dir="."
prefix="${fullnamever}"
includes="LICENSE.txt NOTICE.txt *.txt *.xml lucene-libs/** lib/** src/** example/** client/** contrib/"
- excludes="lib/README.committers.txt **/data/ **/logs/* **/classes/ **/*.sh **/bin/ src/scripts/ src/site/build/ **/target/ client/ruby/flare/ client/python contrib/**/build/ **/*.iml **/*.ipr **/*.iws contrib/clustering/example/lib/** contrib/clustering/lib/downloads/** contrib/analysis-extras/lib/**" />
+ excludes="lib/README.committers.txt **/data/ **/logs/* **/classes/ **/*.sh **/bin/ src/scripts/ src/site/build/ **/target/ client/ruby/flare/ client/python contrib/**/build/ **/*.iml **/*.ipr **/*.iws contrib/clustering/example/lib/** contrib/clustering/lib/downloads/** contrib/analysis-extras/lib/** contrib/uima/lib/**" />
<tarfileset dir="."
prefix="${fullnamever}"
includes="src/test-files/solr/lib/classes/empty-file-main-lib.txt" />
@@ -811,29 +822,20 @@
<!-- Maven artifacts -->
<sign-maven-dependency-artifacts artifact.id="solr-commons-csv" gpg.passphrase="${gpg.passphrase}"/>
+ <sign-maven-dependency-artifacts artifact.id="solr-noggit" gpg.passphrase="${gpg.passphrase}"/>
+ <sign-maven-war-artifacts artifact.id="solr" gpg.passphrase="${gpg.passphrase}"/>
+ <sign-maven-artifacts artifact.id="solr-analysis-extras" gpg.passphrase="${gpg.passphrase}"/>
+ <sign-maven-artifacts artifact.id="solr-cell" gpg.passphrase="${gpg.passphrase}"/>
+ <sign-maven-artifacts artifact.id="solr-clustering" gpg.passphrase="${gpg.passphrase}"/>
<sign-maven-artifacts artifact.id="solr-core" gpg.passphrase="${gpg.passphrase}"/>
<sign-maven-artifacts artifact.id="solr-dataimporthandler" gpg.passphrase="${gpg.passphrase}"/>
<sign-maven-artifacts artifact.id="solr-dataimporthandler-extras" gpg.passphrase="${gpg.passphrase}"/>
- <!--<sign-maven-artifacts artifact.id="solr-clustering" gpg.passphrase="${gpg.passphrase}"/>-->
-
- <sign-maven-artifacts artifact.id="solr-cell" gpg.passphrase="${gpg.passphrase}"/>
- <sign-maven-dependency-artifacts artifact.id="solr-lucene-analyzers" gpg.passphrase="${gpg.passphrase}"/>
- <sign-maven-dependency-artifacts artifact.id="solr-lucene-core" gpg.passphrase="${gpg.passphrase}"/>
- <sign-maven-dependency-artifacts artifact.id="solr-lucene-highlighter" gpg.passphrase="${gpg.passphrase}"/>
- <sign-maven-dependency-artifacts artifact.id="solr-lucene-queries" gpg.passphrase="${gpg.passphrase}"/>
- <sign-maven-dependency-artifacts artifact.id="solr-lucene-snowball" gpg.passphrase="${gpg.passphrase}"/>
- <sign-maven-dependency-artifacts artifact.id="solr-lucene-spellchecker" gpg.passphrase="${gpg.passphrase}"/>
<sign-maven-artifacts artifact.id="solr-solrj" gpg.passphrase="${gpg.passphrase}"/>
- <!-- Thes are special since there are no jars, just poms -->
+ <!-- These are special since there are no jars, just poms -->
<sign-artifact input.file="${maven.dist.prefix}/solr-parent/${version}/solr-parent-${version}.pom" gpg.passphrase="${gpg.passphrase}"/>
- <sign-artifact input.file="${maven.dist.prefix}/solr-lucene-contrib/${version}/solr-lucene-contrib-${version}.pom" gpg.passphrase="${gpg.passphrase}"/>
-
</target>
-
-
-
<target name="prepare-release" depends="clean, svn-up, build-site, package, sign-artifacts" description="Prototype helper for Committers. Assumes gpg is in the path">
<tar destfile="${dist}/solr-maven.tar" longfile="gnu">
@@ -846,6 +848,7 @@
<target name="generate-maven-artifacts" depends="maven.ant.tasks-check, create-package">
<sequential>
+ <ant target="get-maven-poms" dir=".."/>
<!--
@@ -860,18 +863,18 @@
<!-- ========== SOLR PARENT POM ========== -->
- <m2-deploy pom.xml="src/maven/solr-parent-pom.xml.template"/>
+ <m2-deploy pom.xml="pom.xml"/>
<!-- ========== SOLR SPECIFIC COMMONS CSV ========== -->
- <m2-deploy pom.xml="lib/solr-commons-csv-pom.xml.template"
- jar.file="lib/commons-csv-1.0-SNAPSHOT-r966014.jar" />
+ <m2-deploy-with-pom-template pom.xml="lib/solr-commons-csv-pom.xml.template"
+ jar.file="lib/commons-csv-1.0-SNAPSHOT-r966014.jar" />
<!-- ========== SOLR ARTIFACTS ========== -->
- <m2-deploy pom.xml="lib/apache-solr-noggit-pom.xml.template"
- jar.file="lib/apache-solr-noggit-r944541.jar" />
+ <m2-deploy-with-pom-template pom.xml="lib/apache-solr-noggit-pom.xml.template"
+ jar.file="lib/apache-solr-noggit-r944541.jar" />
- <m2-deploy pom.xml="contrib/dataimporthandler/solr-dataimporthandler-pom.xml.template"
+ <m2-deploy pom.xml="contrib/dataimporthandler/src/pom.xml"
jar.file="${dist}/apache-solr-dataimporthandler-${version}.jar">
<artifact-attachments>
<attach file="${dist}/apache-solr-dataimporthandler-src-${version}.jar" classifier="sources"/>
@@ -879,63 +882,67 @@
</artifact-attachments>
</m2-deploy>
- <m2-deploy pom.xml="contrib/dataimporthandler/solr-dataimporthandler-extras-pom.xml.template"
+ <m2-deploy pom.xml="contrib/dataimporthandler/src/extras/pom.xml"
jar.file="${dist}/apache-solr-dataimporthandler-extras-${version}.jar">
-
<artifact-attachments>
<attach file="${dist}/apache-solr-dataimporthandler-extras-src-${version}.jar" classifier="sources"/>
<attach file="${dist}/apache-solr-dataimporthandler-docs-${version}.jar" classifier="javadoc"/>
</artifact-attachments>
</m2-deploy>
- <m2-deploy pom.xml="contrib/extraction/solr-cell-pom.xml.template"
+ <m2-deploy pom.xml="contrib/extraction/pom.xml"
jar.file="${dist}/apache-solr-cell-${version}.jar">
<artifact-attachments>
<attach file="${dist}/apache-solr-cell-src-${version}.jar" classifier="sources"/>
<attach file="${dist}/apache-solr-cell-docs-${version}.jar" classifier="javadoc"/>
</artifact-attachments>
</m2-deploy>
-
- <!--<m2-deploy pom.xml="contrib/clustering/solr-clustering-pom.xml.template"
- jar.file="${dist}/apache-solr-clustering-${version}.jar">
-
+
+ <m2-deploy pom.xml="contrib/clustering/pom.xml"
+ jar.file="${dist}/apache-solr-clustering-${version}.jar">
<artifact-attachments>
<attach file="${dist}/apache-solr-clustering-src-${version}.jar" classifier="sources"/>
<attach file="${dist}/apache-solr-clustering-docs-${version}.jar" classifier="javadoc"/>
</artifact-attachments>
- </m2-deploy>-->
- <!-- Clustring specific -->
- <!-- TODO: MORE NEEDED HERE ONCE WE FINALIZE THE LIBS FOR CARROT -->
- <!-- end clustering specific -->
+ </m2-deploy>
+
+ <m2-deploy pom.xml="contrib/analysis-extras/pom.xml"
+ jar.file="${dist}/apache-solr-analysis-extras-${version}.jar">
+ <artifact-attachments>
+ <attach file="${dist}/apache-solr-analysis-extras-src-${version}.jar" classifier="sources"/>
+ <attach file="${dist}/apache-solr-analysis-extras-docs-${version}.jar" classifier="javadoc"/>
+ </artifact-attachments>
+ </m2-deploy>
- <m2-deploy pom.xml="src/maven/solr-core-pom.xml.template"
- jar.file="${dist}/apache-solr-core-${version}.jar">
+ <m2-deploy pom.xml="contrib/uima/pom.xml"
+ jar.file="${dist}/apache-solr-uima-${version}.jar">
+ <artifact-attachments>
+ <attach file="${dist}/apache-solr-uima-src-${version}.jar" classifier="sources"/>
+ <attach file="${dist}/apache-solr-uima-docs-${version}.jar" classifier="javadoc"/>
+ </artifact-attachments>
+ </m2-deploy>
+ <m2-deploy pom.xml="src/pom.xml"
+ jar.file="${dist}/apache-solr-core-${version}.jar">
<artifact-attachments>
<attach file="${dist}/apache-solr-core-src-${version}.jar" classifier="sources"/>
<attach file="${dist}/apache-solr-core-docs-${version}.jar" classifier="javadoc"/>
</artifact-attachments>
-
</m2-deploy>
- <m2-deploy pom.xml="src/maven/solr-solrj-pom.xml.template"
+ <m2-deploy pom.xml="src/solrj/pom.xml"
jar.file="${dist}/apache-solr-solrj-${version}.jar">
-
<artifact-attachments>
<attach file="${dist}/apache-solr-solrj-src-${version}.jar" classifier="sources"/>
<attach file="${dist}/apache-solr-solrj-docs-${version}.jar" classifier="javadoc"/>
</artifact-attachments>
-
</m2-deploy>
+ <m2-deploy pom.xml="src/webapp/pom.xml"
+ jar.file="${dist}/apache-solr-${version}.war"/>
</sequential>
</target>
- <target name="nightly"
- depends="set-fsdir, test, create-package">
- <!-- no description, don't advertise -->
- </target>
-
<target name="set-fsdir">
<property name="use.fsdir" value="true"/>
</target>
@@ -951,6 +958,8 @@
description="runs the tasks over src/java excluding the license directory">
<rat:report xmlns:rat="antlib:org.apache.rat.anttasks">
<fileset dir="src/java"/>
+ <fileset dir="src/test"/>
+ <fileset dir="src/webapp"/>
<fileset dir="src/common"/>
<fileset dir="src/solrj"/>
<fileset dir="client">
@@ -958,12 +967,16 @@
</fileset>
<fileset dir="contrib/dataimporthandler/src/main/java"/>
<fileset dir="contrib/dataimporthandler/src/test/java"/>
+ <fileset dir="contrib/dataimporthandler/src/extras/main/java"/>
+ <fileset dir="contrib/dataimporthandler/src/extras/test/java"/>
<fileset dir="contrib/clustering/src/main/java"/>
<fileset dir="contrib/clustering/src/test/java"/>
<fileset dir="contrib/extraction/src/main/java"/>
<fileset dir="contrib/extraction/src/test/java"/>
<fileset dir="contrib/analysis-extras/src/test"/>
<fileset dir="contrib/analysis-extras/src/test"/>
+ <fileset dir="contrib/uima/src/main/java"/>
+ <fileset dir="contrib/uima/src/test/java"/>
</rat:report>
</target>
Modified: lucene/dev/branches/docvalues/solr/common-build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/common-build.xml?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/common-build.xml (original)
+++ lucene/dev/branches/docvalues/solr/common-build.xml Wed Feb 9 09:35:27 2011
@@ -33,6 +33,9 @@
<property name="name" value="${ant.project.name}"/>
+ <property name="dev-tools.dir" value="${solr-path}/../dev-tools"/>
+ <property name="prettify.dir" value="${dev-tools.dir}/prettify"/>
+
<tstamp>
<format property="year" pattern="yyyy"/>
<format property="DSTAMP" pattern="yyyy-MM-dd"/>
@@ -138,8 +141,8 @@
<property name="junit.output.dir" location="${common-solr.dir}/${dest}/test-results"/>
<property name="junit.reports" location="${common-solr.dir}/${dest}/test-results/reports"/>
<property name="junit.formatter" value="plain"/>
- <condition property="junit.details.formatter"
- value="org.apache.tools.ant.taskdefs.optional.junit.PlainJUnitResultFormatter"
+ <condition property="junit.details.formatter"
+ value="org.apache.tools.ant.taskdefs.optional.junit.BriefJUnitResultFormatter"
else="org.apache.lucene.util.LuceneJUnitResultFormatter">
<isset property="tests.sequential"/>
</condition>
@@ -270,6 +273,7 @@
includeAntRuntime="${javac.includeAntRuntime}"
sourcepath=""
classpathref="@{classpathref}">
+ <compilerarg line="-Xlint -Xlint:-deprecation -Xlint:-serial"/>
<nested />
</javac>
</sequential>
@@ -343,6 +347,9 @@
<attribute name="destdir"/>
<attribute name="title" default="${Name} ${version} API (${specversion})"/>
<sequential>
+ <copy todir="@{destdir}/../prettify" overwrite="false">
+ <fileset dir="${prettify.dir}"/>
+ </copy>
<javadoc
packagenames="org.apache.solr.*"
failonerror="true"
@@ -356,6 +363,7 @@
link="${javadoc.link.java}"
windowtitle="${Name} ${version} API"
doctitle="@{title}"
+ stylesheetfile="@{destdir}/../prettify/stylesheet+prettify.css"
bottom="Copyright &copy; ${year} Apache Software Foundation. All Rights Reserved.">
<tag name="todo" description="To Do:"/>
<tag name="uml.property" description="UML Property:"/>
@@ -367,6 +375,10 @@
<link href="${javadoc.link.java}"/>
<link href="${javadoc.link.junit}"/>
<link href="${javadoc.link.lucene}"/>
+ <header><![CDATA[
+ <script src="{@docRoot}/../prettify/prettify.js" type="text/javascript"></script>
+ <script language="JavaScript">window.onload=function(){windowTitle();prettyPrint();}</script>
+ ]]></header>
<sources />
@@ -377,19 +389,34 @@
<!-- NOTE, the pom.xml MUST be a relative path. An absolute path may break the build on windows -->
<macrodef name="m2-deploy" description="Builds a Maven artifact">
- <element name="artifact-attachments" optional="yes"/>
- <attribute name="pom.xml" default="${pom.xml}"/>
+ <element name="artifact-attachments" optional="yes"/>
+ <attribute name="pom.xml" default="pom.xml"/>
<attribute name="jar.file" default="${jar.file}"/>
<sequential>
+ <artifact:install-provider artifactId="wagon-ssh" version="1.0-beta-7"/>
+ <artifact:pom id="maven.project" file="@{pom.xml}"/>
+ <artifact:deploy file="@{jar.file}">
+ <artifact-attachments/>
+ <remoteRepository url="${m2.repository.url}">
+ <authentication username="${m2.repository.username}" privateKey="${m2.repository.private.key}"/>
+ </remoteRepository>
+ <pom refid="maven.project"/>
+ </artifact:deploy>
+ </sequential>
+ </macrodef>
+
+ <macrodef name="m2-deploy-with-pom-template" description="Builds a Maven artifact given a POM template">
+ <attribute name="pom.xml"/>
+ <attribute name="jar.file"/>
+ <sequential>
<copy file="@{pom.xml}" tofile="${maven.build.dir}/@{pom.xml}">
<filterset begintoken="@" endtoken="@">
<filter token="version" value="${version}"/>
</filterset>
</copy>
- <artifact:install-provider artifactId="wagon-ssh" version="1.0-beta-2"/>
+ <artifact:install-provider artifactId="wagon-ssh" version="1.0-beta-7"/>
<artifact:pom id="maven.project" file="${maven.build.dir}/@{pom.xml}" />
- <artifact:deploy file="@{jar.file}" uniqueVersion="false">
- <artifact-attachments/>
+ <artifact:deploy file="@{jar.file}">
<remoteRepository url="${m2.repository.url}">
<authentication username="${m2.repository.username}" privateKey="${m2.repository.private.key}"/>
</remoteRepository>
@@ -437,6 +464,17 @@
</sequential>
</macrodef>
+ <macrodef name="sign-maven-war-artifacts" description="Signs maven artifacts">
+ <attribute name="artifact.id"/>
+ <attribute name="prefix.dir" default="${maven.dist.prefix}"/>
+ <attribute name="maven.version" default="${version}"/>
+ <attribute name="gpg.passphrase"/>
+ <sequential>
+ <sign-artifact input.file="@{prefix.dir}/@{artifact.id}/@{maven.version}/@{artifact.id}-@{maven.version}.war" gpg.passphrase="@{gpg.passphrase}"/>
+ <sign-artifact input.file="@{prefix.dir}/@{artifact.id}/@{maven.version}/@{artifact.id}-@{maven.version}.pom" gpg.passphrase="@{gpg.passphrase}"/>
+ </sequential>
+ </macrodef>
+
<macrodef name="sign-maven-dependency-artifacts" description="Signs maven artifacts">
<attribute name="artifact.id"/>
<attribute name="prefix.dir" default="${maven.dist.prefix}"/>
Copied: lucene/dev/branches/docvalues/solr/contrib/analysis-extras/CHANGES.txt (from r1068464, lucene/dev/trunk/solr/contrib/analysis-extras/CHANGES.txt)
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/analysis-extras/CHANGES.txt?p2=lucene/dev/branches/docvalues/solr/contrib/analysis-extras/CHANGES.txt&p1=lucene/dev/trunk/solr/contrib/analysis-extras/CHANGES.txt&r1=1068464&r2=1068809&rev=1068809&view=diff
==============================================================================
--- lucene/dev/trunk/solr/contrib/analysis-extras/CHANGES.txt (original)
+++ lucene/dev/branches/docvalues/solr/contrib/analysis-extras/CHANGES.txt Wed Feb 9 09:35:27 2011
@@ -1,30 +1,227 @@
- Apache Solr - Analysis Extras
- Release Notes
+package org.apache.lucene.index.codecs.preflexrw;
-Introduction
-------------
-The analysis-extras plugin provides additional analyzers that rely
-upon large dependencies/dictionaries.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+import java.io.IOException;
+import org.apache.lucene.store.IndexOutput;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.util.UnicodeUtil;
+import org.apache.lucene.index.FieldInfos;
+import org.apache.lucene.util.BytesRef;
+import org.apache.lucene.index.codecs.preflex.TermInfo;
+
+
+/** This stores a monotonically increasing set of <Term, TermInfo> pairs in a
+ Directory. A TermInfos can be written once, in order. */
+
+final class TermInfosWriter {
+ /** The file format version, a negative number. */
+ public static final int FORMAT = -3;
+
+ // Changed strings to true utf8 with length-in-bytes not
+ // length-in-chars
+ public static final int FORMAT_VERSION_UTF8_LENGTH_IN_BYTES = -4;
+
+ // NOTE: always change this if you switch to a new format!
+ public static final int FORMAT_CURRENT = FORMAT_VERSION_UTF8_LENGTH_IN_BYTES;
+
+ private FieldInfos fieldInfos;
+ private IndexOutput output;
+ private TermInfo lastTi = new TermInfo();
+ private long size;
+
+ // TODO: the default values for these two parameters should be settable from
+ // IndexWriter. However, once that's done, folks will start setting them to
+ // ridiculous values and complaining that things don't work well, as with
+ // mergeFactor. So, let's wait until a number of folks find that alternate
+ // values work better. Note that both of these values are stored in the
+ // segment, so that it's safe to change these w/o rebuilding all indexes.
+
+ /** Expert: The fraction of terms in the "dictionary" which should be stored
+ * in RAM. Smaller values use more memory, but make searching slightly
+ * faster, while larger values use less memory and make searching slightly
+ * slower. Searching is typically not dominated by dictionary lookup, so
+ * tweaking this is rarely useful.*/
+ int indexInterval = 128;
+
+ /** Expert: The fraction of {@link TermDocs} entries stored in skip tables,
+ * used to accelerate {@link TermDocs#advance(int)}. Larger values result in
+ * smaller indexes, greater acceleration, but fewer accelerable cases, while
+ * smaller values result in bigger indexes, less acceleration and more
+ * accelerable cases. More detailed experiments would be useful here. */
+ int skipInterval = 16;
+
+ /** Expert: The maximum number of skip levels. Smaller values result in
+ * slightly smaller indexes, but slower skipping in big posting lists.
+ */
+ int maxSkipLevels = 10;
+
+ private long lastIndexPointer;
+ private boolean isIndex;
+ private final BytesRef lastTerm = new BytesRef();
+ private int lastFieldNumber = -1;
+
+ private TermInfosWriter other;
+
+ TermInfosWriter(Directory directory, String segment, FieldInfos fis,
+ int interval)
+ throws IOException {
+ initialize(directory, segment, fis, interval, false);
+ other = new TermInfosWriter(directory, segment, fis, interval, true);
+ other.other = this;
+ }
+
+ private TermInfosWriter(Directory directory, String segment, FieldInfos fis,
+ int interval, boolean isIndex) throws IOException {
+ initialize(directory, segment, fis, interval, isIndex);
+ }
+
+ private void initialize(Directory directory, String segment, FieldInfos fis,
+ int interval, boolean isi) throws IOException {
+ indexInterval = interval;
+ fieldInfos = fis;
+ isIndex = isi;
+ output = directory.createOutput(segment + (isIndex ? ".tii" : ".tis"));
+ output.writeInt(FORMAT_CURRENT); // write format
+ output.writeLong(0); // leave space for size
+ output.writeInt(indexInterval); // write indexInterval
+ output.writeInt(skipInterval); // write skipInterval
+ output.writeInt(maxSkipLevels); // write maxSkipLevels
+ assert initUTF16Results();
+ }
+
+ // Currently used only by assert statements
+ UnicodeUtil.UTF16Result utf16Result1;
+ UnicodeUtil.UTF16Result utf16Result2;
+ private final BytesRef scratchBytes = new BytesRef();
+
+ // Currently used only by assert statements
+ private boolean initUTF16Results() {
+ utf16Result1 = new UnicodeUtil.UTF16Result();
+ utf16Result2 = new UnicodeUtil.UTF16Result();
+ return true;
+ }
+
+ // Currently used only by assert statement
+ private int compareToLastTerm(int fieldNumber, BytesRef term) {
+
+ if (lastFieldNumber != fieldNumber) {
+ final int cmp = fieldInfos.fieldName(lastFieldNumber).compareTo(fieldInfos.fieldName(fieldNumber));
+ // If there is a field named "" (empty string) then we
+ // will get 0 on this comparison, yet, it's "OK". But
+ // it's not OK if two different field numbers map to
+ // the same name.
+ if (cmp != 0 || lastFieldNumber != -1)
+ return cmp;
+ }
+
+ scratchBytes.copy(term);
+ assert lastTerm.offset == 0;
+ UnicodeUtil.UTF8toUTF16(lastTerm.bytes, 0, lastTerm.length, utf16Result1);
+
+ assert scratchBytes.offset == 0;
+ UnicodeUtil.UTF8toUTF16(scratchBytes.bytes, 0, scratchBytes.length, utf16Result2);
+
+ final int len;
+ if (utf16Result1.length < utf16Result2.length)
+ len = utf16Result1.length;
+ else
+ len = utf16Result2.length;
+
+ for(int i=0;i<len;i++) {
+ final char ch1 = utf16Result1.result[i];
+ final char ch2 = utf16Result2.result[i];
+ if (ch1 != ch2)
+ return ch1-ch2;
+ }
+ return utf16Result1.length - utf16Result2.length;
+ }
+
+ /** Adds a new <<fieldNumber, termBytes>, TermInfo> pair to the set.
+ Term must be lexicographically greater than all previous Terms added.
+ TermInfo pointers must be positive and greater than all previous.*/
+ public void add(int fieldNumber, BytesRef term, TermInfo ti)
+ throws IOException {
+
+ assert compareToLastTerm(fieldNumber, term) < 0 ||
+ (isIndex && term.length == 0 && lastTerm.length == 0) :
+ "Terms are out of order: field=" + fieldInfos.fieldName(fieldNumber) + " (number " + fieldNumber + ")" +
+ " lastField=" + fieldInfos.fieldName(lastFieldNumber) + " (number " + lastFieldNumber + ")" +
+ " text=" + term.utf8ToString() + " lastText=" + lastTerm.utf8ToString();
+
+ assert ti.freqPointer >= lastTi.freqPointer: "freqPointer out of order (" + ti.freqPointer + " < " + lastTi.freqPointer + ")";
+ assert ti.proxPointer >= lastTi.proxPointer: "proxPointer out of order (" + ti.proxPointer + " < " + lastTi.proxPointer + ")";
+
+ if (!isIndex && size % indexInterval == 0)
+ other.add(lastFieldNumber, lastTerm, lastTi); // add an index term
+
+ writeTerm(fieldNumber, term); // write term
+
+ output.writeVInt(ti.docFreq); // write doc freq
+ output.writeVLong(ti.freqPointer - lastTi.freqPointer); // write pointers
+ output.writeVLong(ti.proxPointer - lastTi.proxPointer);
+
+ if (ti.docFreq >= skipInterval) {
+ output.writeVInt(ti.skipOffset);
+ }
+
+ if (isIndex) {
+ output.writeVLong(other.output.getFilePointer() - lastIndexPointer);
+ lastIndexPointer = other.output.getFilePointer(); // write pointer
+ }
+
+ lastFieldNumber = fieldNumber;
+ lastTi.set(ti);
+ size++;
+ }
+
+ private void writeTerm(int fieldNumber, BytesRef term)
+ throws IOException {
+
+ //System.out.println(" tiw.write field=" + fieldNumber + " term=" + term.utf8ToString());
+
+ // TODO: UTF16toUTF8 could tell us this prefix
+ // Compute prefix in common with last term:
+ int start = 0;
+ final int limit = term.length < lastTerm.length ? term.length : lastTerm.length;
+ while(start < limit) {
+ if (term.bytes[start+term.offset] != lastTerm.bytes[start+lastTerm.offset])
+ break;
+ start++;
+ }
+
+ final int length = term.length - start;
+ output.writeVInt(start); // write shared prefix length
+ output.writeVInt(length); // write delta length
+ output.writeBytes(term.bytes, start+term.offset, length); // write delta bytes
+ output.writeVInt(fieldNumber); // write field num
+ lastTerm.copy(term);
+ }
+
+ /** Called to complete TermInfos creation. */
+ void close() throws IOException {
+ output.seek(4); // write size after format
+ output.writeLong(size);
+ output.close();
+
+ if (!isIndex)
+ other.close();
+ }
-It includes integration with ICU for multilingual support, and
-analyzers for Chinese and Polish.
-
-
-$Id$
-================== Release 4.0-dev ==================
-
-(No Changes)
-
-================== Release 3.1-dev ==================
-
-* SOLR-2210: Add icu-based tokenizer and filters to contrib/analysis-extras (rmuir)
-
-* SOLR-1336: Add SmartChinese (word segmentation for Simplified Chinese)
- tokenizer and filters to contrib/analysis-extras (rmuir)
-
-* SOLR-2211,LUCENE-2763: Added UAX29URLEmailTokenizerFactory, which implements
- UAX#29, a unicode algorithm with good results for most languages, as well as
- URL and E-mail tokenization according to the relevant RFCs.
- (Tom Burton-West via rmuir)
-
-* SOLR-2237: Added StempelPolishStemFilterFactory to contrib/analysis-extras (rmuir)
+}
Modified: lucene/dev/branches/docvalues/solr/contrib/analysis-extras/build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/analysis-extras/build.xml?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/contrib/analysis-extras/build.xml (original)
+++ lucene/dev/branches/docvalues/solr/contrib/analysis-extras/build.xml Wed Feb 9 09:35:27 2011
@@ -73,7 +73,7 @@
<pathelement path="${dest}/test-classes"/>
<pathelement path="${java.class.path}"/>
<pathelement location="${common-solr.dir}/build/tests"/> <!-- include solr test code -->
- <pathelement location="${common-solr.dir}/../lucene/build/classes/test" /> <!-- include some lucene test code -->
+ <pathelement location="${common-solr.dir}/../lucene/build/classes/test-framework" /> <!-- include some lucene test code -->
<path refid="common.classpath"/>
</path>
Modified: lucene/dev/branches/docvalues/solr/contrib/clustering/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/clustering/CHANGES.txt?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/contrib/clustering/CHANGES.txt (original)
+++ lucene/dev/branches/docvalues/solr/contrib/clustering/CHANGES.txt Wed Feb 9 09:35:27 2011
@@ -7,8 +7,11 @@ See http://wiki.apache.org/solr/Clusteri
CHANGES
$Id$
+================== Release 4.0.0-dev ==================
-================== Release XXXX ==================
+(No Changes)
+
+================== Release 3.1.0-dev ==================
* SOLR-1684: Switch to use the SolrIndexSearcher.doc(int, Set<String>) method b/c it can use the document cache (gsingers)
@@ -18,9 +21,9 @@ $Id$
This release of C2 also does not have a specific Lucene dependency. (Stanislaw Osinski, gsingers)
* SOLR-2282: Add distributed search support for search result clustering.
- (Brad Giaccio, koji)
+ (Brad Giaccio, Dawid Weiss, Stanislaw Osinski, rmuir, koji)
================== Release 1.4.0 ==================
Solr Clustering will be released for the first time in Solr 1.4. See http://wiki.apache.org/solr/ClusteringComponent
- for details on using.
\ No newline at end of file
+ for details on using.
Modified: lucene/dev/branches/docvalues/solr/contrib/clustering/build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/clustering/build.xml?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/contrib/clustering/build.xml (original)
+++ lucene/dev/branches/docvalues/solr/contrib/clustering/build.xml Wed Feb 9 09:35:27 2011
@@ -42,7 +42,7 @@
<pathelement path="${dest}/test-classes"/>
<pathelement path="${java.class.path}"/>
<pathelement location="${common-solr.dir}/build/tests"/> <!-- include solr test code -->
- <pathelement location="${common-solr.dir}/../lucene/build/classes/test" /> <!-- include some lucene test code -->
+ <pathelement location="${common-solr.dir}/../lucene/build/classes/test-framework" /> <!-- include some lucene test code -->
<path refid="common.classpath"/>
<!-- DistributedClusteringComponentTest uses Jetty -->
<fileset dir="${solr-path}/example/lib">
Modified: lucene/dev/branches/docvalues/solr/contrib/clustering/src/main/java/org/apache/solr/handler/clustering/ClusteringComponent.java
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/clustering/src/main/java/org/apache/solr/handler/clustering/ClusteringComponent.java?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/contrib/clustering/src/main/java/org/apache/solr/handler/clustering/ClusteringComponent.java (original)
+++ lucene/dev/branches/docvalues/solr/contrib/clustering/src/main/java/org/apache/solr/handler/clustering/ClusteringComponent.java Wed Feb 9 09:35:27 2011
@@ -46,6 +46,13 @@ import java.util.Set;
* <p/>
* This engine is experimental. Output from this engine is subject to change in future releases.
*
+ * <pre class="prettyprint" >
+ * <searchComponent class="org.apache.solr.handler.clustering.ClusteringComponent" name="clustering">
+ * <lst name="engine">
+ * <str name="name">default</str>
+ * <str name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm</str>
+ * </lst>
+ * </searchComponent></pre>
*/
public class ClusteringComponent extends SearchComponent implements SolrCoreAware {
private transient static Logger log = LoggerFactory.getLogger(ClusteringComponent.class);
Modified: lucene/dev/branches/docvalues/solr/contrib/clustering/src/main/java/org/apache/solr/handler/clustering/carrot2/CarrotClusteringEngine.java
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/clustering/src/main/java/org/apache/solr/handler/clustering/carrot2/CarrotClusteringEngine.java?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/contrib/clustering/src/main/java/org/apache/solr/handler/clustering/carrot2/CarrotClusteringEngine.java (original)
+++ lucene/dev/branches/docvalues/solr/contrib/clustering/src/main/java/org/apache/solr/handler/clustering/carrot2/CarrotClusteringEngine.java Wed Feb 9 09:35:27 2011
@@ -77,6 +77,7 @@ public class CarrotClusteringEngine exte
private String idFieldName;
+ @Override
@Deprecated
public Object cluster(Query query, DocList docList, SolrQueryRequest sreq) {
SolrIndexSearcher searcher = sreq.getSearcher();
@@ -90,6 +91,7 @@ public class CarrotClusteringEngine exte
}
}
+ @Override
public Object cluster(Query query, SolrDocumentList solrDocList,
Map<SolrDocument, Integer> docIds, SolrQueryRequest sreq) {
try {
@@ -124,7 +126,7 @@ public class CarrotClusteringEngine exte
// is included in the code base of Solr, so that it's possible to refactor
// the Lucene APIs the factory relies on if needed.
initAttributes.put("PreprocessingPipeline.languageModelFactory",
- new LuceneLanguageModelFactory());
+ LuceneLanguageModelFactory.class);
this.controller.init(initAttributes);
this.idFieldName = core.getSchema().getUniqueKeyField().getName();
Modified: lucene/dev/branches/docvalues/solr/contrib/clustering/src/main/java/org/apache/solr/handler/clustering/carrot2/LuceneLanguageModelFactory.java
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/clustering/src/main/java/org/apache/solr/handler/clustering/carrot2/LuceneLanguageModelFactory.java?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/contrib/clustering/src/main/java/org/apache/solr/handler/clustering/carrot2/LuceneLanguageModelFactory.java (original)
+++ lucene/dev/branches/docvalues/solr/contrib/clustering/src/main/java/org/apache/solr/handler/clustering/carrot2/LuceneLanguageModelFactory.java Wed Feb 9 09:35:27 2011
@@ -31,7 +31,7 @@ import org.apache.lucene.analysis.tokena
import org.carrot2.core.LanguageCode;
import org.carrot2.text.analysis.ExtendedWhitespaceTokenizer;
import org.carrot2.text.analysis.ITokenizer;
-import org.carrot2.text.linguistic.BaseLanguageModelFactory;
+import org.carrot2.text.linguistic.DefaultLanguageModelFactory;
import org.carrot2.text.linguistic.IStemmer;
import org.carrot2.text.linguistic.IdentityStemmer;
import org.carrot2.text.util.MutableCharArray;
@@ -62,14 +62,15 @@ import org.tartarus.snowball.ext.Turkish
* change, the changes can be made in this class.
*/
@Bindable(prefix = "DefaultLanguageModelFactory")
-public class LuceneLanguageModelFactory extends BaseLanguageModelFactory {
+public class LuceneLanguageModelFactory extends DefaultLanguageModelFactory {
final static Logger logger = org.slf4j.LoggerFactory
.getLogger(LuceneLanguageModelFactory.class);
/**
* Provide an {@link IStemmer} implementation for a given language.
*/
- protected IStemmer createStemmer(LanguageCode language) {
+ @Override
+ protected IStemmer createStemmer(LanguageCode language) {
switch (language) {
case ARABIC:
return ArabicStemmerFactory.createStemmer();
@@ -200,7 +201,7 @@ public class LuceneLanguageModelFactory
logger
.warn(
"Could not instantiate Lucene stemmer for Arabic, clustering quality "
- + "of Chinese content may be degraded. For best quality clusters, "
+ + "of Arabic content may be degraded. For best quality clusters, "
+ "make sure Lucene's Arabic analyzer JAR is in the classpath",
e);
}
Modified: lucene/dev/branches/docvalues/solr/contrib/clustering/src/test/java/org/apache/solr/handler/clustering/DistributedClusteringComponentTest.java
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/clustering/src/test/java/org/apache/solr/handler/clustering/DistributedClusteringComponentTest.java?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/contrib/clustering/src/test/java/org/apache/solr/handler/clustering/DistributedClusteringComponentTest.java (original)
+++ lucene/dev/branches/docvalues/solr/contrib/clustering/src/test/java/org/apache/solr/handler/clustering/DistributedClusteringComponentTest.java Wed Feb 9 09:35:27 2011
@@ -20,13 +20,16 @@ package org.apache.solr.handler.clusteri
import org.apache.solr.BaseDistributedSearchTestCase;
import org.apache.solr.common.params.CommonParams;
-import org.junit.Ignore;
-
-@Ignore("FIXME: test fails on hudson")
public class DistributedClusteringComponentTest extends
BaseDistributedSearchTestCase {
@Override
+ public String getSolrHome() {
+ // TODO: this should work with just "solr-clustering"...
+ return getFile("solr-clustering").getAbsolutePath();
+ }
+
+ @Override
public void doTest() throws Exception {
del("*:*");
int numberOfDocs = 0;
Modified: lucene/dev/branches/docvalues/solr/contrib/clustering/src/test/java/org/apache/solr/handler/clustering/MockDocumentClusteringEngine.java
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/clustering/src/test/java/org/apache/solr/handler/clustering/MockDocumentClusteringEngine.java?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/contrib/clustering/src/test/java/org/apache/solr/handler/clustering/MockDocumentClusteringEngine.java (original)
+++ lucene/dev/branches/docvalues/solr/contrib/clustering/src/test/java/org/apache/solr/handler/clustering/MockDocumentClusteringEngine.java Wed Feb 9 09:35:27 2011
@@ -25,11 +25,13 @@ import org.apache.solr.search.DocSet;
*
**/
public class MockDocumentClusteringEngine extends DocumentClusteringEngine {
+ @Override
public NamedList cluster(DocSet docs, SolrParams solrParams) {
NamedList result = new NamedList();
return result;
}
+ @Override
public NamedList cluster(SolrParams solrParams) {
NamedList result = new NamedList();
return result;
Modified: lucene/dev/branches/docvalues/solr/contrib/clustering/src/test/resources/solr-clustering/conf/solrconfig.xml
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/clustering/src/test/resources/solr-clustering/conf/solrconfig.xml?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/contrib/clustering/src/test/resources/solr-clustering/conf/solrconfig.xml (original)
+++ lucene/dev/branches/docvalues/solr/contrib/clustering/src/test/resources/solr-clustering/conf/solrconfig.xml Wed Feb 9 09:35:27 2011
@@ -428,13 +428,6 @@
-->
<requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
- <!--
- Analysis request handler. Since Solr 1.3. Use to returnhow a document is analyzed. Useful
- for debugging and as a token server for other types of applications
- -->
- <requestHandler name="/analysis" class="solr.AnalysisRequestHandler" />
-
-
<!-- CSV update handler, loaded on demand -->
<requestHandler name="/update/csv" class="solr.CSVRequestHandler" startup="lazy" />
Modified: lucene/dev/branches/docvalues/solr/contrib/dataimporthandler/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/dataimporthandler/CHANGES.txt?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/contrib/dataimporthandler/CHANGES.txt (original)
+++ lucene/dev/branches/docvalues/solr/contrib/dataimporthandler/CHANGES.txt Wed Feb 9 09:35:27 2011
@@ -8,7 +8,11 @@ HTTP data sources quick and easy.
$Id$
-================== 1.5.0-dev ==================
+================== 4.0.0-dev ==================
+
+(No Changes)
+
+================== 3.1.0-dev ==================
Upgrading from Solr 1.4
----------------------
@@ -44,6 +48,9 @@ New Features
Optimizations
----------------------
+* SOLR-2200: Improve the performance of DataImportHandler for large delta-import
+ updates. (Mark Waddle via rmuir)
+
Bug Fixes
----------------------
* SOLR-1638: Fixed NullPointerException during import if uniqueKey is not specified
@@ -65,9 +72,19 @@ Bug Fixes
* SOLR-1811: formatDate should use the current NOW value always (Sean Timm via noble)
+* SOLR-2310: getTimeElapsedSince() returns incorrect hour value when the elapse is over 60 hours
+ (tom liu via koji)
+
+* SOLR-2252: When a child entity in nested entities is rootEntity="true", delta-import doesn't work.
+ (koji)
+
+* SOLR-2330: solrconfig.xml files in example-DIH are broken. (Matt Parker, koji)
+
Other Changes
----------------------
+* SOLR-1821: Fix TimeZone-dependent test failure in TestEvaluatorBag.
+ (Chris Male via rmuir)
Build
----------------------
Copied: lucene/dev/branches/docvalues/solr/contrib/dataimporthandler/README.txt (from r1068464, lucene/dev/trunk/solr/contrib/dataimporthandler/README.txt)
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/dataimporthandler/README.txt?p2=lucene/dev/branches/docvalues/solr/contrib/dataimporthandler/README.txt&p1=lucene/dev/trunk/solr/contrib/dataimporthandler/README.txt&r1=1068464&r2=1068809&rev=1068809&view=diff
==============================================================================
--- lucene/dev/trunk/solr/contrib/dataimporthandler/README.txt (original)
+++ lucene/dev/branches/docvalues/solr/contrib/dataimporthandler/README.txt Wed Feb 9 09:35:27 2011
@@ -1,7 +1,215 @@
-Although Solr strives to be agnostic of the Locale where the server is
-running, some code paths in DataImportHandler are known to depend on the
-System default Locale, Timezone, or Charset. It is recommended that when
-running Solr you set the following system properties:
- -Duser.language=xx -Duser.country=YY -Duser.timezone=ZZZ
+package org.apache.lucene.index.codecs.preflexrw;
-where xx, YY, and ZZZ are consistent with any database server's configuration.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import org.apache.lucene.util.BytesRef;
+import org.apache.lucene.index.codecs.FieldsConsumer;
+import org.apache.lucene.index.codecs.TermsConsumer;
+import org.apache.lucene.index.codecs.PostingsConsumer;
+import org.apache.lucene.index.codecs.standard.DefaultSkipListWriter;
+import org.apache.lucene.index.codecs.preflex.PreFlexCodec;
+import org.apache.lucene.index.CorruptIndexException;
+import org.apache.lucene.index.IndexFileNames;
+import org.apache.lucene.index.SegmentWriteState;
+import org.apache.lucene.index.FieldInfo;
+import org.apache.lucene.index.codecs.preflex.TermInfo;
+import org.apache.lucene.index.codecs.docvalues.DocValuesConsumer;
+import org.apache.lucene.store.IndexOutput;
+
+import java.io.IOException;
+import java.util.Comparator;
+
+class PreFlexFieldsWriter extends FieldsConsumer {
+
+ private final TermInfosWriter termsOut;
+ private final IndexOutput freqOut;
+ private final IndexOutput proxOut;
+ private final DefaultSkipListWriter skipListWriter;
+ private final int totalNumDocs;
+
+ public PreFlexFieldsWriter(SegmentWriteState state) throws IOException {
+ termsOut = new TermInfosWriter(state.directory,
+ state.segmentName,
+ state.fieldInfos,
+ state.termIndexInterval);
+
+ final String freqFile = IndexFileNames.segmentFileName(state.segmentName, "", PreFlexCodec.FREQ_EXTENSION);
+ freqOut = state.directory.createOutput(freqFile);
+ totalNumDocs = state.numDocs;
+
+ if (state.fieldInfos.hasProx()) {
+ final String proxFile = IndexFileNames.segmentFileName(state.segmentName, "", PreFlexCodec.PROX_EXTENSION);
+ proxOut = state.directory.createOutput(proxFile);
+ } else {
+ proxOut = null;
+ }
+
+ skipListWriter = new DefaultSkipListWriter(termsOut.skipInterval,
+ termsOut.maxSkipLevels,
+ totalNumDocs,
+ freqOut,
+ proxOut);
+ //System.out.println("\nw start seg=" + segment);
+ }
+
+ @Override
+ public TermsConsumer addField(FieldInfo field) throws IOException {
+ assert field.number != -1;
+ //System.out.println("w field=" + field.name + " storePayload=" + field.storePayloads + " number=" + field.number);
+ return new PreFlexTermsWriter(field);
+ }
+
+ @Override
+ public void close() throws IOException {
+ termsOut.close();
+ freqOut.close();
+ if (proxOut != null) {
+ proxOut.close();
+ }
+ }
+
+ private class PreFlexTermsWriter extends TermsConsumer {
+ private final FieldInfo fieldInfo;
+ private final boolean omitTF;
+ private final boolean storePayloads;
+
+ private final TermInfo termInfo = new TermInfo();
+ private final PostingsWriter postingsWriter = new PostingsWriter();
+
+ public PreFlexTermsWriter(FieldInfo fieldInfo) {
+ this.fieldInfo = fieldInfo;
+ omitTF = fieldInfo.omitTermFreqAndPositions;
+ storePayloads = fieldInfo.storePayloads;
+ }
+
+ private class PostingsWriter extends PostingsConsumer {
+ private int lastDocID;
+ private int lastPayloadLength = -1;
+ private int lastPosition;
+ private int df;
+
+ public PostingsWriter reset() {
+ df = 0;
+ lastDocID = 0;
+ lastPayloadLength = -1;
+ return this;
+ }
+
+ @Override
+ public void startDoc(int docID, int termDocFreq) throws IOException {
+ //System.out.println(" w doc=" + docID);
+
+ final int delta = docID - lastDocID;
+ if (docID < 0 || (df > 0 && delta <= 0)) {
+ throw new CorruptIndexException("docs out of order (" + docID + " <= " + lastDocID + " )");
+ }
+
+ if ((++df % termsOut.skipInterval) == 0) {
+ skipListWriter.setSkipData(lastDocID, storePayloads, lastPayloadLength);
+ skipListWriter.bufferSkip(df);
+ }
+
+ lastDocID = docID;
+
+ assert docID < totalNumDocs: "docID=" + docID + " totalNumDocs=" + totalNumDocs;
+
+ if (omitTF) {
+ freqOut.writeVInt(delta);
+ } else {
+ final int code = delta << 1;
+ if (termDocFreq == 1) {
+ freqOut.writeVInt(code|1);
+ } else {
+ freqOut.writeVInt(code);
+ freqOut.writeVInt(termDocFreq);
+ }
+ }
+ lastPosition = 0;
+ }
+
+ @Override
+ public void addPosition(int position, BytesRef payload) throws IOException {
+ assert proxOut != null;
+
+ //System.out.println(" w pos=" + position + " payl=" + payload);
+ final int delta = position - lastPosition;
+ lastPosition = position;
+
+ if (storePayloads) {
+ final int payloadLength = payload == null ? 0 : payload.length;
+ if (payloadLength != lastPayloadLength) {
+ //System.out.println(" write payload len=" + payloadLength);
+ lastPayloadLength = payloadLength;
+ proxOut.writeVInt((delta<<1)|1);
+ proxOut.writeVInt(payloadLength);
+ } else {
+ proxOut.writeVInt(delta << 1);
+ }
+ if (payloadLength > 0) {
+ proxOut.writeBytes(payload.bytes, payload.offset, payload.length);
+ }
+ } else {
+ proxOut.writeVInt(delta);
+ }
+ }
+
+ @Override
+ public void finishDoc() throws IOException {
+ }
+ }
+
+ @Override
+ public PostingsConsumer startTerm(BytesRef text) throws IOException {
+ //System.out.println(" w term=" + text.utf8ToString());
+ skipListWriter.resetSkip();
+ termInfo.freqPointer = freqOut.getFilePointer();
+ if (proxOut != null) {
+ termInfo.proxPointer = proxOut.getFilePointer();
+ }
+ return postingsWriter.reset();
+ }
+
+ @Override
+ public void finishTerm(BytesRef text, int numDocs) throws IOException {
+ if (numDocs > 0) {
+ long skipPointer = skipListWriter.writeSkip(freqOut);
+ termInfo.docFreq = numDocs;
+ termInfo.skipOffset = (int) (skipPointer - termInfo.freqPointer);
+ //System.out.println(" w finish term=" + text.utf8ToString() + " fnum=" + fieldInfo.number);
+ termsOut.add(fieldInfo.number,
+ text,
+ termInfo);
+ }
+ }
+
+ @Override
+ public void finish() throws IOException {
+ }
+
+ @Override
+ public Comparator<BytesRef> getComparator() throws IOException {
+ return BytesRef.getUTF8SortedAsUTF16Comparator();
+ }
+ }
+
+ @Override
+ public DocValuesConsumer addValuesField(FieldInfo field) throws IOException {
+ //TODO(simonw): can we fix this easily?
+ throw new UnsupportedOperationException("no implemented");
+ }
+}
\ No newline at end of file
Modified: lucene/dev/branches/docvalues/solr/contrib/dataimporthandler/build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/branches/docvalues/solr/contrib/dataimporthandler/build.xml?rev=1068809&r1=1068808&r2=1068809&view=diff
==============================================================================
--- lucene/dev/branches/docvalues/solr/contrib/dataimporthandler/build.xml (original)
+++ lucene/dev/branches/docvalues/solr/contrib/dataimporthandler/build.xml Wed Feb 9 09:35:27 2011
@@ -56,7 +56,7 @@
<pathelement path="target/classes" />
<pathelement path="target/test-classes" />
<pathelement location="${solr-path}/build/tests"/> <!-- include solr test code -->
- <pathelement location="${solr-path}/../lucene/build/classes/test" /> <!-- include some lucene test code -->
+ <pathelement location="${solr-path}/../lucene/build/classes/test-framework" /> <!-- include some lucene test code -->
<pathelement path="${java.class.path}"/>
</path>
@@ -68,7 +68,7 @@
<pathelement path="target/test-classes" />
<pathelement path="target/extras/test-classes" />
<pathelement location="${solr-path}/build/tests"/> <!-- include solr test code -->
- <pathelement location="${solr-path}/../lucene/build/classes/test" /> <!-- include some lucene test code -->
+ <pathelement location="${solr-path}/../lucene/build/classes/test-framework" /> <!-- include some lucene test code -->
<pathelement path="${java.class.path}"/>
</path>