You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-commits@lucene.apache.org by ma...@apache.org on 2010/02/18 23:25:08 UTC
svn commit: r911619 - in /lucene/solr/branches/cloud: ./ contrib/extraction/
contrib/extraction/lib/ src/java/org/apache/solr/core/
src/java/org/apache/solr/search/
src/java/org/apache/solr/search/function/distance/
src/java/org/apache/solr/update/ src...
Author: markrmiller
Date: Thu Feb 18 22:25:07 2010
New Revision: 911619
URL: http://svn.apache.org/viewvc?rev=911619&view=rev
Log:
merge up to r911595
Added:
lucene/solr/branches/cloud/contrib/extraction/lib/poi-3.6.jar
- copied unchanged from r911595, lucene/solr/trunk/contrib/extraction/lib/poi-3.6.jar
lucene/solr/branches/cloud/contrib/extraction/lib/poi-ooxml-3.6.jar
- copied unchanged from r911595, lucene/solr/trunk/contrib/extraction/lib/poi-ooxml-3.6.jar
lucene/solr/branches/cloud/contrib/extraction/lib/poi-ooxml-schemas-3.6.jar
- copied unchanged from r911595, lucene/solr/trunk/contrib/extraction/lib/poi-ooxml-schemas-3.6.jar
lucene/solr/branches/cloud/contrib/extraction/lib/poi-scratchpad-3.6.jar
- copied unchanged from r911595, lucene/solr/trunk/contrib/extraction/lib/poi-scratchpad-3.6.jar
lucene/solr/branches/cloud/contrib/extraction/lib/tika-core-0.6.jar
- copied unchanged from r911595, lucene/solr/trunk/contrib/extraction/lib/tika-core-0.6.jar
lucene/solr/branches/cloud/contrib/extraction/lib/tika-parsers-0.6.jar
- copied unchanged from r911595, lucene/solr/trunk/contrib/extraction/lib/tika-parsers-0.6.jar
lucene/solr/branches/cloud/contrib/extraction/lib/xercesImpl-2.8.1.jar
- copied unchanged from r911595, lucene/solr/trunk/contrib/extraction/lib/xercesImpl-2.8.1.jar
lucene/solr/branches/cloud/src/test/org/apache/solr/search/TestSort.java
- copied unchanged from r911595, lucene/solr/trunk/src/test/org/apache/solr/search/TestSort.java
Removed:
lucene/solr/branches/cloud/contrib/extraction/lib/ooxml-schemas-1.0.jar
lucene/solr/branches/cloud/contrib/extraction/lib/poi-3.5-FINAL.jar
lucene/solr/branches/cloud/contrib/extraction/lib/poi-ooxml-3.5-FINAL.jar
lucene/solr/branches/cloud/contrib/extraction/lib/poi-scratchpad-3.5-FINAL.jar
lucene/solr/branches/cloud/contrib/extraction/lib/tika-core-0.5.jar
lucene/solr/branches/cloud/contrib/extraction/lib/tika-parsers-0.5.jar
Modified:
lucene/solr/branches/cloud/ (props changed)
lucene/solr/branches/cloud/CHANGES.txt
lucene/solr/branches/cloud/contrib/extraction/CHANGES.txt
lucene/solr/branches/cloud/src/java/org/apache/solr/core/SolrCore.java
lucene/solr/branches/cloud/src/java/org/apache/solr/search/MissingStringLastComparatorSource.java
lucene/solr/branches/cloud/src/java/org/apache/solr/search/function/distance/DistanceUtils.java
lucene/solr/branches/cloud/src/java/org/apache/solr/update/UpdateHandler.java
Propchange: lucene/solr/branches/cloud/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Thu Feb 18 22:25:07 2010
@@ -1 +1 @@
-/lucene/solr/trunk:888480-910306
+/lucene/solr/trunk:888480-911595
Modified: lucene/solr/branches/cloud/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/solr/branches/cloud/CHANGES.txt?rev=911619&r1=911618&r2=911619&view=diff
==============================================================================
--- lucene/solr/branches/cloud/CHANGES.txt (original)
+++ lucene/solr/branches/cloud/CHANGES.txt Thu Feb 18 22:25:07 2010
@@ -34,7 +34,7 @@
Versions of Major Components
---------------------
Apache Lucene 2.9.2-rc1 (r910082 on 2.9 branch)
-Apache Tika 0.4
+Apache Tika 0.6
Carrot2 3.1.0
Detailed Change List
@@ -91,6 +91,10 @@
Optimizations
----------------------
+* SOLR-1679: Don't build up string messages in SolrCore.execute unless they
+ are necessary for the current log level.
+ (Fuad Efendi and hossman)
+
Bug Fixes
----------------------
@@ -181,6 +185,12 @@
* SOLR-1579: Fixes to XML escaping in stats.jsp
(David Bowen and hossman)
+* SOLR-1777: fieldTypes with sortMissingLast=true or sortMissingFirst=true can
+ result in incorrectly sorted results. (yonik)
+
+* SOLR-1779: Fix parsing of points with more than 2 dimensions (gsingers)
+
+
Other Changes
----------------------
@@ -238,6 +248,9 @@
* SOLR-1771: Improved error message when StringIndex cannot be initialized
for a function query (hossman)
+* SOLR-1695: Improved error messages when adding a document that does not
+ contain exactly one value for the uniqueKey field (hossman)
+
Build
----------------------
Modified: lucene/solr/branches/cloud/contrib/extraction/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/solr/branches/cloud/contrib/extraction/CHANGES.txt?rev=911619&r1=911618&r2=911619&view=diff
==============================================================================
--- lucene/solr/branches/cloud/contrib/extraction/CHANGES.txt (original)
+++ lucene/solr/branches/cloud/contrib/extraction/CHANGES.txt Thu Feb 18 22:25:07 2010
@@ -21,11 +21,15 @@
================== Release 1.5-dev ==================
+
* SOLR-1567: Upgrade to Tika 0.5, which upgrades many of the underlying libraries (PDFBox, for example) too (gsingers)
* SOLR-1756: The date.format setting causes ClassCastException when enabled and the config code that
parses this setting does not properly use the same iterator instance. (Christoph Brill, Mark Miller)
+* SOLR-1738: Upgrade to Tika 0.6 (gsingers)
+
+
================== Release 1.4.0 ==================
1. SOLR-284: Added in support for extraction. (Eric Pugh, Chris Harris, gsingers)
Modified: lucene/solr/branches/cloud/src/java/org/apache/solr/core/SolrCore.java
URL: http://svn.apache.org/viewvc/lucene/solr/branches/cloud/src/java/org/apache/solr/core/SolrCore.java?rev=911619&r1=911618&r2=911619&view=diff
==============================================================================
--- lucene/solr/branches/cloud/src/java/org/apache/solr/core/SolrCore.java (original)
+++ lucene/solr/branches/cloud/src/java/org/apache/solr/core/SolrCore.java Thu Feb 18 22:25:07 2010
@@ -1307,29 +1307,38 @@
public void execute(SolrRequestHandler handler, SolrQueryRequest req, SolrQueryResponse rsp) {
if (handler==null) {
- log.warn(logid+"Null Request Handler '" + req.getParams().get(CommonParams.QT) +"' :" + req);
- throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,"Null Request Handler '" + req.getParams().get(CommonParams.QT) + "'", true);
+ String msg = "Null Request Handler '" +
+ req.getParams().get(CommonParams.QT) + "'";
+
+ if (log.isWarnEnabled()) log.warn(logid + msg + ":" + req);
+
+ throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, msg, true);
}
// setup response header and handle request
final NamedList<Object> responseHeader = new SimpleOrderedMap<Object>();
rsp.add("responseHeader", responseHeader);
+
+ // toLog is a local ref to the same NamedList used by the request
NamedList toLog = rsp.getToLog();
- //toLog.add("core", getName());
+ // for back compat, we set these now just in case other code
+ // are expecting them during handleRequest
toLog.add("webapp", req.getContext().get("webapp"));
toLog.add("path", req.getContext().get("path"));
toLog.add("params", "{" + req.getParamString() + "}");
+
handler.handleRequest(req,rsp);
setResponseHeaderValues(handler,req,rsp);
- StringBuilder sb = new StringBuilder();
- for (int i=0; i<toLog.size(); i++) {
- String name = toLog.getName(i);
- Object val = toLog.getVal(i);
- sb.append(name).append("=").append(val).append(" ");
- }
- log.info(logid + sb.toString());
- /*log.info(logid+"" + req.getContext().get("path") + " "
- + req.getParamString()+ " 0 "+
- (int)(rsp.getEndTime() - req.getStartTime()));*/
+
+ if (log.isInfoEnabled()) {
+ StringBuilder sb = new StringBuilder(logid);
+ for (int i=0; i<toLog.size(); i++) {
+ String name = toLog.getName(i);
+ Object val = toLog.getVal(i);
+ sb.append(name).append("=").append(val).append(" ");
+ }
+ log.info(sb.toString());
+ }
+
}
/**
Modified: lucene/solr/branches/cloud/src/java/org/apache/solr/search/MissingStringLastComparatorSource.java
URL: http://svn.apache.org/viewvc/lucene/solr/branches/cloud/src/java/org/apache/solr/search/MissingStringLastComparatorSource.java?rev=911619&r1=911618&r2=911619&view=diff
==============================================================================
--- lucene/solr/branches/cloud/src/java/org/apache/solr/search/MissingStringLastComparatorSource.java (original)
+++ lucene/solr/branches/cloud/src/java/org/apache/solr/search/MissingStringLastComparatorSource.java Thu Feb 18 22:25:07 2010
@@ -43,18 +43,17 @@
}
public FieldComparator newComparator(String fieldname, int numHits, int sortPos, boolean reversed) throws IOException {
- return new MissingLastOrdComparator(numHits, fieldname, sortPos, reversed, true, missingValueProxy);
+ return new MissingLastOrdComparator(numHits, fieldname, sortPos, reversed, missingValueProxy);
}
}
+
// Copied from Lucene and modified since the Lucene version couldn't
// be extended or have it's values accessed.
-
-// NOTE: there were a number of other interesting String
-// comparators explored, but this one seemed to perform
-// best all around. See LUCENE-1483 for details.
-class MissingLastOrdComparator extends FieldComparator {
+ class MissingLastOrdComparator extends FieldComparator {
+ private static final int NULL_ORD = Integer.MAX_VALUE;
+ private final String nullVal;
private final int[] ords;
private final String[] values;
@@ -71,30 +70,19 @@
private final boolean reversed;
private final int sortPos;
- private final int nullCmp;
- private final Comparable nullVal;
-
- public MissingLastOrdComparator(int numHits, String field, int sortPos, boolean reversed, boolean sortMissingLast, Comparable nullVal) {
+ public MissingLastOrdComparator(int numHits, String field, int sortPos, boolean reversed, String nullVal) {
ords = new int[numHits];
values = new String[numHits];
readerGen = new int[numHits];
this.sortPos = sortPos;
this.reversed = reversed;
this.field = field;
- this.nullCmp = sortMissingLast ? 1 : -1;
this.nullVal = nullVal;
}
- public int compare(int slot1, int slot2) {
- int ord1 = ords[slot1];
- int ord2 = ords[slot2];
- int cmp = ord1-ord2;
- if (ord1==0 || ord2==0) {
- if (cmp==0) return 0;
- return ord1==0 ? nullCmp : -nullCmp;
- }
-
+ public int compare(int slot1, int slot2) {
if (readerGen[slot1] == readerGen[slot2]) {
+ int cmp = ords[slot1] - ords[slot2];
if (cmp != 0) {
return cmp;
}
@@ -102,13 +90,14 @@
final String val1 = values[slot1];
final String val2 = values[slot2];
+
if (val1 == null) {
if (val2 == null) {
return 0;
}
- return nullCmp;
+ return 1;
} else if (val2 == null) {
- return -nullCmp;
+ return -1;
}
return val1.compareTo(val2);
}
@@ -116,27 +105,17 @@
public int compareBottom(int doc) {
assert bottomSlot != -1;
int order = this.order[doc];
- final int cmp = bottomOrd - order;
- if (bottomOrd==0 || order==0) {
- if (cmp==0) return 0;
- return bottomOrd==0 ? nullCmp : -nullCmp;
- }
-
+ int ord = (order == 0) ? NULL_ORD : order;
+ final int cmp = bottomOrd - ord;
if (cmp != 0) {
return cmp;
}
final String val2 = lookup[order];
- if (bottomValue == null) {
- if (val2 == null) {
- return 0;
- }
- // bottom wins
- return nullCmp;
- } else if (val2 == null) {
- // doc wins
- return -nullCmp;
- }
+
+ // take care of the case where both vals are null
+ if (bottomValue == val2) return 0;
+
return bottomValue.compareTo(val2);
}
@@ -145,7 +124,8 @@
int index = 0;
String value = values[slot];
if (value == null) {
- ords[slot] = 0;
+ // should already be done
+ // ords[slot] = NULL_ORD;
return;
}
@@ -171,7 +151,7 @@
public void copy(int slot, int doc) {
final int ord = order[doc];
- ords[slot] = ord;
+ ords[slot] = ord == 0 ? NULL_ORD : ord;
assert ord >= 0;
values[slot] = lookup[ord];
readerGen[slot] = currentReaderGen;
@@ -196,14 +176,10 @@
}
bottomOrd = ords[bottom];
assert bottomOrd >= 0;
- assert bottomOrd < lookup.length;
+ // assert bottomOrd < lookup.length;
bottomValue = values[bottom];
}
- public int sortType() {
- return SortField.STRING;
- }
-
public Comparable value(int slot) {
Comparable v = values[slot];
return v==null ? nullVal : v;
@@ -220,4 +196,4 @@
public String getField() {
return field;
}
- }
\ No newline at end of file
+ }
Modified: lucene/solr/branches/cloud/src/java/org/apache/solr/search/function/distance/DistanceUtils.java
URL: http://svn.apache.org/viewvc/lucene/solr/branches/cloud/src/java/org/apache/solr/search/function/distance/DistanceUtils.java?rev=911619&r1=911618&r2=911619&view=diff
==============================================================================
--- lucene/solr/branches/cloud/src/java/org/apache/solr/search/function/distance/DistanceUtils.java (original)
+++ lucene/solr/branches/cloud/src/java/org/apache/solr/search/function/distance/DistanceUtils.java Thu Feb 18 22:25:07 2010
@@ -70,7 +70,7 @@
result = Math.sqrt(squaredEuclideanDistance(vec1, vec2));
} else if (power == Integer.MAX_VALUE || Double.isInfinite(power)) {//infininte norm?
for (int i = 0; i < vec1.length; i++) {
- result = Math.max(vec1[i], vec2[i]);
+ result = Math.max(result, Math.max(vec1[i], vec2[i]));
}
} else {
for (int i = 0; i < vec1.length; i++) {
@@ -139,9 +139,13 @@
for (; i < dimension; i++) {
while (start < end && externalVal.charAt(start) == ' ') start++;
while (end > start && externalVal.charAt(end - 1) == ' ') end--;
+ if (start == end){
+ break;
+ }
out[i] = externalVal.substring(start, end);
start = idx + 1;
end = externalVal.indexOf(',', start);
+ idx = end;
if (end == -1) {
end = externalVal.length();
}
@@ -179,9 +183,13 @@
//TODO: abstract common code with other parsePoint
while (start < end && externalVal.charAt(start) == ' ') start++;
while (end > start && externalVal.charAt(end - 1) == ' ') end--;
+ if (start == end){
+ break;
+ }
out[i] = Double.parseDouble(externalVal.substring(start, end));
start = idx + 1;
end = externalVal.indexOf(',', start);
+ idx = end;
if (end == -1) {
end = externalVal.length();
}
Modified: lucene/solr/branches/cloud/src/java/org/apache/solr/update/UpdateHandler.java
URL: http://svn.apache.org/viewvc/lucene/solr/branches/cloud/src/java/org/apache/solr/update/UpdateHandler.java?rev=911619&r1=911618&r2=911619&view=diff
==============================================================================
--- lucene/solr/branches/cloud/src/java/org/apache/solr/update/UpdateHandler.java (original)
+++ lucene/solr/branches/cloud/src/java/org/apache/solr/update/UpdateHandler.java Thu Feb 18 22:25:07 2010
@@ -111,9 +111,9 @@
// form have that transformation already performed and stored as the field value.
Fieldable[] id = doc.getFieldables( idField.getName() );
if (id == null || id.length < 1)
- throw new SolrException( SolrException.ErrorCode.BAD_REQUEST,"Document is missing uniqueKey field " + idField.getName());
+ throw new SolrException( SolrException.ErrorCode.BAD_REQUEST,"Document is missing mandatory uniqueKey field: " + idField.getName());
if( id.length > 1 )
- throw new SolrException( SolrException.ErrorCode.BAD_REQUEST,"Document specifies multiple unique ids! " + idField.getName());
+ throw new SolrException( SolrException.ErrorCode.BAD_REQUEST,"Document contains multiple values for uniqueKey field: " + idField.getName());
return idFieldType.storedToIndexed( id[0] );
}