You are viewing a plain text version of this content. The canonical link for it is here.
Posted to cvs@cocoon.apache.org by th...@apache.org on 2009/10/13 13:33:26 UTC
svn commit: r824705 - in /cocoon/cocoon3/trunk/cocoon-optional: ./ solr/
solr/conf/
src/main/java/org/apache/cocoon/optional/pipeline/components/sax/solr/
src/test/java/org/apache/cocoon/optional/pipeline/components/sax/solr/
src/test/resources/org/apa...
Author: thorsten
Date: Tue Oct 13 11:33:26 2009
New Revision: 824705
URL: http://svn.apache.org/viewvc?rev=824705&view=rev
Log:
COCOON3-43
Provide a Consumer that interacts with solr
Reporter and Patch-Provder: Bertil Chapuis
Thanks Bertil.
Added:
cocoon/cocoon3/trunk/cocoon-optional/solr/
cocoon/cocoon3/trunk/cocoon-optional/solr/conf/
cocoon/cocoon3/trunk/cocoon-optional/solr/conf/protwords.txt
cocoon/cocoon3/trunk/cocoon-optional/solr/conf/schema.xml
cocoon/cocoon3/trunk/cocoon-optional/solr/conf/solrconfig.xml
cocoon/cocoon3/trunk/cocoon-optional/solr/conf/stopwords.txt
cocoon/cocoon3/trunk/cocoon-optional/solr/conf/synonyms.txt
cocoon/cocoon3/trunk/cocoon-optional/src/main/java/org/apache/cocoon/optional/pipeline/components/sax/solr/
cocoon/cocoon3/trunk/cocoon-optional/src/main/java/org/apache/cocoon/optional/pipeline/components/sax/solr/SolrConsumer.java
cocoon/cocoon3/trunk/cocoon-optional/src/test/java/org/apache/cocoon/optional/pipeline/components/sax/solr/
cocoon/cocoon3/trunk/cocoon-optional/src/test/java/org/apache/cocoon/optional/pipeline/components/sax/solr/SolrConsumerTestCase.java
cocoon/cocoon3/trunk/cocoon-optional/src/test/resources/org/apache/cocoon/optional/pipeline/components/sax/solr/
cocoon/cocoon3/trunk/cocoon-optional/src/test/resources/org/apache/cocoon/optional/pipeline/components/sax/solr/sample.xml
Modified:
cocoon/cocoon3/trunk/cocoon-optional/pom.xml
Modified: cocoon/cocoon3/trunk/cocoon-optional/pom.xml
URL: http://svn.apache.org/viewvc/cocoon/cocoon3/trunk/cocoon-optional/pom.xml?rev=824705&r1=824704&r2=824705&view=diff
==============================================================================
--- cocoon/cocoon3/trunk/cocoon-optional/pom.xml (original)
+++ cocoon/cocoon3/trunk/cocoon-optional/pom.xml Tue Oct 13 11:33:26 2009
@@ -78,6 +78,22 @@
<artifactId>fop</artifactId>
<optional>true</optional>
</dependency>
+ <dependency>
+ <groupId>org.apache.solr</groupId>
+ <artifactId>solr-core</artifactId>
+ <version>1.3.0</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.solr</groupId>
+ <artifactId>solr-solrj</artifactId>
+ <version>1.3.0</version>
+ </dependency>
+ <dependency>
+ <groupId>javax.servlet</groupId>
+ <artifactId>servlet-api</artifactId>
+ <version>2.3</version>
+ <scope>test</scope>
+ </dependency>
<!-- Test libraries -->
<dependency>
<groupId>junit</groupId>
Added: cocoon/cocoon3/trunk/cocoon-optional/solr/conf/protwords.txt
URL: http://svn.apache.org/viewvc/cocoon/cocoon3/trunk/cocoon-optional/solr/conf/protwords.txt?rev=824705&view=auto
==============================================================================
--- cocoon/cocoon3/trunk/cocoon-optional/solr/conf/protwords.txt (added)
+++ cocoon/cocoon3/trunk/cocoon-optional/solr/conf/protwords.txt Tue Oct 13 11:33:26 2009
@@ -0,0 +1,21 @@
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#-----------------------------------------------------------------------
+# Use a protected word file to protect against the stemmer reducing two
+# unrelated words to the same base word.
+
+# Some non-words that normally won't be encountered,
+# just to test that they won't be stemmed.
+dontstems
+zwhacky
+
Added: cocoon/cocoon3/trunk/cocoon-optional/solr/conf/schema.xml
URL: http://svn.apache.org/viewvc/cocoon/cocoon3/trunk/cocoon-optional/solr/conf/schema.xml?rev=824705&view=auto
==============================================================================
--- cocoon/cocoon3/trunk/cocoon-optional/solr/conf/schema.xml (added)
+++ cocoon/cocoon3/trunk/cocoon-optional/solr/conf/schema.xml Tue Oct 13 11:33:26 2009
@@ -0,0 +1,78 @@
+<?xml version="1.0" encoding="UTF-8" ?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<schema name="example" version="1.1">
+ <types>
+
+ <fieldType name="string" class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
+ <fieldType name="boolean" class="solr.BoolField" sortMissingLast="true" omitNorms="true"/>
+ <fieldType name="integer" class="solr.IntField" omitNorms="true"/>
+ <fieldType name="long" class="solr.LongField" omitNorms="true"/>
+ <fieldType name="float" class="solr.FloatField" omitNorms="true"/>
+ <fieldType name="double" class="solr.DoubleField" omitNorms="true"/>
+ <fieldType name="sint" class="solr.SortableIntField" sortMissingLast="true" omitNorms="true"/>
+ <fieldType name="slong" class="solr.SortableLongField" sortMissingLast="true" omitNorms="true"/>
+ <fieldType name="sfloat" class="solr.SortableFloatField" sortMissingLast="true" omitNorms="true"/>
+ <fieldType name="sdouble" class="solr.SortableDoubleField" sortMissingLast="true" omitNorms="true"/>
+ <fieldType name="date" class="solr.DateField" sortMissingLast="true" omitNorms="true"/>
+ <fieldType name="random" class="solr.RandomSortField" indexed="true" />
+
+ <fieldType name="text" class="solr.TextField" positionIncrementGap="100">
+ <analyzer type="index">
+ <tokenizer class="solr.WhitespaceTokenizerFactory"/>
+ <filter class="solr.StopFilterFactory"
+ ignoreCase="true"
+ words="stopwords.txt"
+ enablePositionIncrements="true"
+ />
+ <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
+ <filter class="solr.LowerCaseFilterFactory"/>
+ <filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
+ <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
+ </analyzer>
+ <analyzer type="query">
+ <tokenizer class="solr.WhitespaceTokenizerFactory"/>
+ <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
+ <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
+ <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
+ <filter class="solr.LowerCaseFilterFactory"/>
+ <filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
+ <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
+ </analyzer>
+ </fieldType>
+
+ </types>
+
+ <fields>
+ <field name="id" type="integer" indexed="true" stored="true" required="true" />
+ <field name="title" type="text" indexed="true" stored="true" />
+ <field name="content" type="text" indexed="true" stored="true" />
+ </fields>
+
+ <!-- Field to use to determine and enforce document uniqueness.
+ Unless this field is marked with required="false", it will be a required field
+ -->
+ <uniqueKey>id</uniqueKey>
+
+ <!-- field for the QueryParser to use when an explicit fieldname is absent -->
+ <defaultSearchField>content</defaultSearchField>
+
+ <!-- SolrQueryParser configuration: defaultOperator="AND|OR" -->
+ <solrQueryParser defaultOperator="OR"/>
+
+</schema>
Added: cocoon/cocoon3/trunk/cocoon-optional/solr/conf/solrconfig.xml
URL: http://svn.apache.org/viewvc/cocoon/cocoon3/trunk/cocoon-optional/solr/conf/solrconfig.xml?rev=824705&view=auto
==============================================================================
--- cocoon/cocoon3/trunk/cocoon-optional/solr/conf/solrconfig.xml (added)
+++ cocoon/cocoon3/trunk/cocoon-optional/solr/conf/solrconfig.xml Tue Oct 13 11:33:26 2009
@@ -0,0 +1,190 @@
+<?xml version="1.0" encoding="UTF-8" ?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<config>
+ <abortOnConfigurationError>true</abortOnConfigurationError>
+
+ <!-- must specify a data directory -->
+ <dataDir>${solr.data.dir:data}</dataDir>
+
+
+ <indexDefaults>
+ <!-- Values here affect all index writers and act as a default unless overridden. -->
+ <useCompoundFile>false</useCompoundFile>
+
+ <mergeFactor>10</mergeFactor>
+ <!--
+ If both ramBufferSizeMB and maxBufferedDocs is set, then Lucene will flush based on whichever limit is hit first.
+
+ -->
+ <!--<maxBufferedDocs>1000</maxBufferedDocs>-->
+ <!-- Tell Lucene when to flush documents to disk.
+ Giving Lucene more memory for indexing means faster indexing at the cost of more RAM
+
+ If both ramBufferSizeMB and maxBufferedDocs is set, then Lucene will flush based on whichever limit is hit first.
+
+ -->
+ <ramBufferSizeMB>32</ramBufferSizeMB>
+ <maxMergeDocs>2147483647</maxMergeDocs>
+ <maxFieldLength>10000</maxFieldLength>
+ <writeLockTimeout>1000</writeLockTimeout>
+ <commitLockTimeout>10000</commitLockTimeout>
+ <lockType>single</lockType>
+ </indexDefaults>
+
+ <mainIndex>
+ <!-- options specific to the main on-disk lucene index -->
+ <useCompoundFile>false</useCompoundFile>
+ <ramBufferSizeMB>32</ramBufferSizeMB>
+ <mergeFactor>10</mergeFactor>
+ <!-- Deprecated -->
+ <!--<maxBufferedDocs>1000</maxBufferedDocs>-->
+ <maxMergeDocs>2147483647</maxMergeDocs>
+ <maxFieldLength>10000</maxFieldLength>
+
+ <!-- If true, unlock any held write or commit locks on startup.
+ This defeats the locking mechanism that allows multiple
+ processes to safely access a lucene index, and should be
+ used with care.
+ This is not needed if lock type is 'none' or 'single'
+ -->
+ <unlockOnStartup>true</unlockOnStartup>
+
+ <!--
+ Custom deletion policies can specified here. The class must
+ implement org.apache.lucene.index.IndexDeletionPolicy.
+
+ http://lucene.apache.org/java/2_3_2/api/org/apache/lucene/index/IndexDeletionPolicy.html
+
+ The standard Solr IndexDeletionPolicy implementation supports deleting
+ index commit points on number of commits, age of commit point and
+ optimized status.
+
+ The latest commit point should always be preserved regardless
+ of the criteria.
+ -->
+ <deletionPolicy class="solr.SolrDeletionPolicy">
+ <!-- Keep only optimized commit points -->
+ <str name="keepOptimizedOnly">false</str>
+ <!-- The maximum number of commit points to be kept -->
+ <str name="maxCommitsToKeep">1</str>
+ <!--
+ Delete all commit points once they have reached the given age.
+ Supports DateMathParser syntax e.g.
+
+ <str name="maxCommitAge">30MINUTES</str>
+ <str name="maxCommitAge">1DAY</str>
+ -->
+ </deletionPolicy>
+
+ </mainIndex>
+
+ <jmx />
+
+ <!-- the default high-performance update handler -->
+ <updateHandler class="solr.DirectUpdateHandler2">
+ <!-- Perform a <commit/> automatically under certain conditions:
+ <autoCommit>
+ <maxDocs>10000</maxDocs>
+ <maxTime>1000</maxTime>
+ </autoCommit>
+ -->
+ </updateHandler>
+
+
+ <query>
+ <!-- Maximum number of clauses in a boolean query... can affect
+ range or prefix queries that expand to big boolean
+ queries. An exception is thrown if exceeded. -->
+ <maxBooleanClauses>1024</maxBooleanClauses>
+
+ <filterCache
+ class="solr.LRUCache"
+ size="512"
+ initialSize="512"
+ autowarmCount="128"/>
+
+ <queryResultCache
+ class="solr.LRUCache"
+ size="512"
+ initialSize="512"
+ autowarmCount="32"/>
+
+ <documentCache
+ class="solr.LRUCache"
+ size="512"
+ initialSize="512"
+ autowarmCount="0"/>
+
+ <enableLazyFieldLoading>true</enableLazyFieldLoading>
+
+ <!-- An optimization for use with the queryResultCache. When a search
+ is requested, a superset of the requested number of document ids
+ are collected. For example, if a search for a particular query
+ requests matching documents 10 through 19, and queryWindowSize is 50,
+ then documents 0 through 49 will be collected and cached. Any further
+ requests in that range can be satisfied via the cache. -->
+ <queryResultWindowSize>50</queryResultWindowSize>
+
+ <!-- Maximum number of documents to cache for any entry in the
+ queryResultCache. -->
+ <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
+
+ <!-- This entry enables an int hash representation for filters (DocSets)
+ when the number of items in the set is less than maxSize. For smaller
+ sets, this representation is more memory efficient, more efficient to
+ iterate over, and faster to take intersections. -->
+ <HashDocSet maxSize="3000" loadFactor="0.75"/>
+
+ <!-- If a search request comes in and there is no current registered searcher,
+ then immediately register the still warming searcher and use it. If
+ "false" then all requests will block until the first searcher is done
+ warming. -->
+ <useColdSearcher>false</useColdSearcher>
+
+ <!-- Maximum number of searchers that may be warming in the background
+ concurrently. An error is returned if this limit is exceeded. Recommend
+ 1-2 for read-only slaves, higher for masters w/o cache warming. -->
+ <maxWarmingSearchers>2</maxWarmingSearchers>
+
+ </query>
+
+ <!--
+ Let the dispatch filter handler /select?qt=XXX
+ handleSelect=true will use consistent error handling for /select and /update
+ handleSelect=false will use solr1.1 style error formatting
+ -->
+ <requestDispatcher handleSelect="true" />
+
+ <requestHandler name="standard" class="solr.SearchHandler" default="true">
+ <!-- default values for query parameters -->
+ <lst name="defaults">
+ <str name="echoParams">explicit</str>
+ <!--
+ <int name="rows">10</int>
+ <str name="fl">*</str>
+ <str name="version">2.1</str>
+ -->
+ </lst>
+ </requestHandler>
+
+
+ <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
+ <requestHandler name="/admin/" class="org.apache.solr.handler.admin.AdminHandlers" />
+
+</config>
Added: cocoon/cocoon3/trunk/cocoon-optional/solr/conf/stopwords.txt
URL: http://svn.apache.org/viewvc/cocoon/cocoon3/trunk/cocoon-optional/solr/conf/stopwords.txt?rev=824705&view=auto
==============================================================================
--- cocoon/cocoon3/trunk/cocoon-optional/solr/conf/stopwords.txt (added)
+++ cocoon/cocoon3/trunk/cocoon-optional/solr/conf/stopwords.txt Tue Oct 13 11:33:26 2009
@@ -0,0 +1,57 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#-----------------------------------------------------------------------
+# a couple of test stopwords to test that the words are really being
+# configured from this file:
+stopworda
+stopwordb
+
+#Standard english stop words taken from Lucene's StopAnalyzer
+an
+and
+are
+as
+at
+be
+but
+by
+for
+if
+in
+into
+is
+it
+no
+not
+of
+on
+or
+s
+such
+t
+that
+the
+their
+then
+there
+these
+they
+this
+to
+was
+will
+with
+
Added: cocoon/cocoon3/trunk/cocoon-optional/solr/conf/synonyms.txt
URL: http://svn.apache.org/viewvc/cocoon/cocoon3/trunk/cocoon-optional/solr/conf/synonyms.txt?rev=824705&view=auto
==============================================================================
--- cocoon/cocoon3/trunk/cocoon-optional/solr/conf/synonyms.txt (added)
+++ cocoon/cocoon3/trunk/cocoon-optional/solr/conf/synonyms.txt Tue Oct 13 11:33:26 2009
@@ -0,0 +1,31 @@
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#-----------------------------------------------------------------------
+#some test synonym mappings unlikely to appear in real input text
+aaa => aaaa
+bbb => bbbb1 bbbb2
+ccc => cccc1,cccc2
+a\=>a => b\=>b
+a\,a => b\,b
+fooaaa,baraaa,bazaaa
+
+# Some synonym groups specific to this example
+GB,gib,gigabyte,gigabytes
+MB,mib,megabyte,megabytes
+Television, Televisions, TV, TVs
+#notice we use "gib" instead of "GiB" so any WordDelimiterFilter coming
+#after us won't split it into two words.
+
+# Synonym mappings can be used for spelling correction too
+pixima => pixma
+
Added: cocoon/cocoon3/trunk/cocoon-optional/src/main/java/org/apache/cocoon/optional/pipeline/components/sax/solr/SolrConsumer.java
URL: http://svn.apache.org/viewvc/cocoon/cocoon3/trunk/cocoon-optional/src/main/java/org/apache/cocoon/optional/pipeline/components/sax/solr/SolrConsumer.java?rev=824705&view=auto
==============================================================================
--- cocoon/cocoon3/trunk/cocoon-optional/src/main/java/org/apache/cocoon/optional/pipeline/components/sax/solr/SolrConsumer.java (added)
+++ cocoon/cocoon3/trunk/cocoon-optional/src/main/java/org/apache/cocoon/optional/pipeline/components/sax/solr/SolrConsumer.java Tue Oct 13 11:33:26 2009
@@ -0,0 +1,157 @@
+package org.apache.cocoon.optional.pipeline.components.sax.solr;
+
+import java.net.MalformedURLException;
+import java.util.Arrays;
+
+import org.apache.cocoon.pipeline.ProcessingException;
+import org.apache.cocoon.sax.AbstractSAXProducer;
+import org.apache.cocoon.sax.SAXConsumer;
+import org.apache.solr.client.solrj.SolrServer;
+import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer;
+import org.apache.solr.common.SolrInputDocument;
+import org.apache.solr.common.util.StrUtils;
+import org.xml.sax.Attributes;
+import org.xml.sax.Locator;
+import org.xml.sax.SAXException;
+
+public class SolrConsumer extends AbstractSAXProducer implements SAXConsumer {
+
+ private SolrServer solr = null;
+
+ private boolean isNull = false;
+
+ private SolrInputDocument doc = new SolrInputDocument();
+
+ private float boost = 1.0f;
+
+ private String name = null;
+
+ private String text = "";
+
+ public SolrConsumer(String url) throws MalformedURLException {
+ solr = new CommonsHttpSolrServer(url);
+ }
+
+ public SolrConsumer(SolrServer server) {
+ solr = server;
+ }
+
+ public void characters(char[] ch, int start, int length)
+ throws SAXException {
+ text += new String(Arrays.copyOfRange(ch, start, start + length));
+ getSAXConsumer().characters(ch, start, length);
+ }
+
+ public void startElement(String uri, String localName, String qName,
+ Attributes atts) throws SAXException {
+ if ("field".equals(localName)) {
+ text = "";
+ boost = 1.0f;
+
+ if (atts.getValue("name") != null)
+ name = atts.getValue("name");
+
+ if (atts.getValue("boost") != null)
+ boost = Float.parseFloat(atts.getValue("boost"));
+
+ if (atts.getValue("null") != null)
+ isNull = StrUtils.parseBoolean(atts.getValue("null"));
+
+ }
+ getSAXConsumer().startElement(uri, localName, qName, atts);
+ }
+
+ public void endElement(String uri, String localName, String qName)
+ throws SAXException {
+ if ("docs".equals(localName)) {
+ try {
+ solr.commit();
+ } catch (Exception e) {
+ throw new ProcessingException(
+ "Unable to commit the Solr documents.", e);
+ }
+ } else if ("doc".equals(localName)) {
+ try {
+ solr.add(doc);
+ doc = new SolrInputDocument();
+ } catch (Exception e) {
+ throw new ProcessingException(
+ "Unable to add the Solr document.", e);
+ }
+ } else if ("field".equals(localName)) {
+ if (!isNull && text.length() > 0) {
+ doc.addField(name, text, boost);
+ boost = 1.0f;
+ }
+ }
+ getSAXConsumer().endElement(uri, localName, qName);
+ }
+
+ public void endPrefixMapping(String prefix) throws SAXException {
+ getSAXConsumer().endPrefixMapping(prefix);
+ }
+
+ public void ignorableWhitespace(char[] ch, int start, int length)
+ throws SAXException {
+ getSAXConsumer().ignorableWhitespace(ch, start, length);
+ }
+
+ public void processingInstruction(String target, String data)
+ throws SAXException {
+ getSAXConsumer().processingInstruction(target, data);
+ }
+
+ public void setDocumentLocator(Locator locator) {
+ getSAXConsumer().setDocumentLocator(locator);
+ }
+
+ public void skippedEntity(String name) throws SAXException {
+ getSAXConsumer().skippedEntity(name);
+ }
+
+ public void startDocument() throws SAXException {
+ getSAXConsumer().startDocument();
+ }
+
+ public void startPrefixMapping(String prefix, String uri)
+ throws SAXException {
+ getSAXConsumer().startPrefixMapping(prefix, uri);
+ }
+
+ public void comment(char[] ch, int start, int length) throws SAXException {
+ getSAXConsumer().comment(ch, start, length);
+ }
+
+ public void endCDATA() throws SAXException {
+ getSAXConsumer().endCDATA();
+ }
+
+ public void endDTD() throws SAXException {
+ getSAXConsumer().endDTD();
+ }
+
+ public void endEntity(String name) throws SAXException {
+ getSAXConsumer().endEntity(name);
+ }
+
+ public void startCDATA() throws SAXException {
+ getSAXConsumer().startCDATA();
+ }
+
+ public void startDTD(String name, String publicId, String systemId)
+ throws SAXException {
+ getSAXConsumer().startDTD(name, publicId, systemId);
+ }
+
+ public void startEntity(String name) throws SAXException {
+ getSAXConsumer().startEntity(name);
+ }
+
+ public void endDocument() throws SAXException {
+ getSAXConsumer().endDocument();
+ }
+
+ public void finish() {
+ }
+
+}
Added: cocoon/cocoon3/trunk/cocoon-optional/src/test/java/org/apache/cocoon/optional/pipeline/components/sax/solr/SolrConsumerTestCase.java
URL: http://svn.apache.org/viewvc/cocoon/cocoon3/trunk/cocoon-optional/src/test/java/org/apache/cocoon/optional/pipeline/components/sax/solr/SolrConsumerTestCase.java?rev=824705&view=auto
==============================================================================
--- cocoon/cocoon3/trunk/cocoon-optional/src/test/java/org/apache/cocoon/optional/pipeline/components/sax/solr/SolrConsumerTestCase.java (added)
+++ cocoon/cocoon3/trunk/cocoon-optional/src/test/java/org/apache/cocoon/optional/pipeline/components/sax/solr/SolrConsumerTestCase.java Tue Oct 13 11:33:26 2009
@@ -0,0 +1,66 @@
+package org.apache.cocoon.optional.pipeline.components.sax.solr;
+
+import static org.junit.Assert.*;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.Iterator;
+
+import javax.xml.parsers.ParserConfigurationException;
+
+import org.apache.cocoon.optional.pipeline.components.sax.solr.SolrConsumer;
+import org.apache.cocoon.pipeline.NonCachingPipeline;
+import org.apache.cocoon.pipeline.Pipeline;
+import org.apache.cocoon.sax.SAXPipelineComponent;
+import org.apache.cocoon.sax.component.FileGenerator;
+import org.apache.cocoon.sax.component.XMLSerializer;
+import org.apache.solr.client.solrj.SolrQuery;
+import org.apache.solr.client.solrj.SolrServer;
+import org.apache.solr.client.solrj.embedded.EmbeddedSolrServer;
+import org.apache.solr.common.SolrDocument;
+import org.apache.solr.common.SolrDocumentList;
+import org.apache.solr.core.CoreContainer;
+import org.apache.solr.core.CoreDescriptor;
+import org.apache.solr.core.SolrCore;
+import org.apache.solr.core.SolrResourceLoader;
+import org.junit.Test;
+import org.xml.sax.SAXException;
+
+public final class SolrConsumerTestCase {
+
+ private SolrServer solr;
+
+ public SolrConsumerTestCase() throws ParserConfigurationException, IOException, SAXException {
+ SolrResourceLoader loader = new SolrResourceLoader("solr");
+ CoreContainer container = new CoreContainer(loader);
+ CoreDescriptor descriptor = new CoreDescriptor(container, "cname", "." );
+ SolrCore core = container.create(descriptor);
+ container.register( core.getName(), core, false );
+ solr = new EmbeddedSolrServer(container, core.getName());
+ }
+
+ @Test
+ public void testPipelineWithSolrConsumer() throws Exception {
+ Pipeline<SAXPipelineComponent> pipeline = new NonCachingPipeline<SAXPipelineComponent>();
+
+ pipeline.addComponent(new FileGenerator(this.getClass().getResource("sample.xml")));
+ pipeline.addComponent(new SolrConsumer(solr));
+ pipeline.addComponent(new XMLSerializer());
+
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ pipeline.setup(baos);
+ pipeline.execute();
+
+ SolrQuery query = new SolrQuery();
+
+ query.setQuery("title:title");
+ SolrDocumentList results = solr.query(query).getResults();
+ Iterator<SolrDocument> documents = results.iterator();
+
+ assertEquals(results.size(), 3);
+ assertEquals(documents.next().getFieldValue("title"), "title 2");
+ assertEquals(documents.next().getFieldValue("title"), "title 3");
+ assertEquals(documents.next().getFieldValue("title"), "title 1");
+
+ }
+}
Added: cocoon/cocoon3/trunk/cocoon-optional/src/test/resources/org/apache/cocoon/optional/pipeline/components/sax/solr/sample.xml
URL: http://svn.apache.org/viewvc/cocoon/cocoon3/trunk/cocoon-optional/src/test/resources/org/apache/cocoon/optional/pipeline/components/sax/solr/sample.xml?rev=824705&view=auto
==============================================================================
--- cocoon/cocoon3/trunk/cocoon-optional/src/test/resources/org/apache/cocoon/optional/pipeline/components/sax/solr/sample.xml (added)
+++ cocoon/cocoon3/trunk/cocoon-optional/src/test/resources/org/apache/cocoon/optional/pipeline/components/sax/solr/sample.xml Tue Oct 13 11:33:26 2009
@@ -0,0 +1,34 @@
+<?xml version="1.0"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<docs>
+ <doc>
+ <field name="id">1</field>
+ <field name="title" boost="1">title 1</field>
+ <field name="content">description 1</field>
+ </doc>
+ <doc>
+ <field name="id">2</field>
+ <field name="title" boost="3">title 2</field>
+ <field name="content">description 2</field>
+ </doc>
+ <doc>
+ <field name="id">3</field>
+ <field name="title" boost="2">title 3</field>
+ <field name="content">description 3</field>
+ </doc>
+</docs>
\ No newline at end of file
Re: [c3] Test execution and dealing with optional dependencies
Posted by Thorsten Scherler <th...@juntadeandalucia.es>.
On Thu, 2009-10-15 at 10:06 +0200, Reinhard Pötz wrote:
> Thorsten Scherler wrote:
> >
> > On 13/10/2009, at 23:02, Reinhard Pötz wrote:
> >
> >> thorsten@apache.org wrote:
> >>> Author: thorsten
> >>> Date: Tue Oct 13 11:33:26 2009
> >>> New Revision: 824705
> >>>
> >>> URL: http://svn.apache.org/viewvc?rev=824705&view=rev
> >>> Log:
> >>> COCOON3-43
> >>> Provide a Consumer that interacts with solr
> >>> Reporter and Patch-Provder: Bertil Chapuis
> >>> Thanks Bertil.
> >>>
> >>
> >>> + public void characters(char[] ch, int start, int length)
> >>> + throws SAXException {
> >>> + text += new String(Arrays.copyOfRange(ch, start, start +
> >>> length));
> >>> + getSAXConsumer().characters(ch, start, length);
> >>> + }
> >>
> >> Arrays.copyOfRange() comes with JDK 1.6. Can you please replace it with
> >> an 1.5 compliant alternative? Thanks!
> >>
> >
> > Good catch, should be fixed now.
> >
> > Committed revision 824959.
>
> Thorsten,
>
> thanks for working on this patch. I've just fixed some problems with
> missing license headers.
>
> In order to spot these errors yourself in the future, you can run it.bat
> or it.sh that activate all unit tests, integration tests and license
> header checks (RAT plugin).
>
> I also changed all dependencies of the cocoon-optional module to
> <optional>true</optional> so that you do not add all the Solr stuff to
> your classpath (or have to manually exclude them) just because you want
> to use the FOP serializer for example.
Yeah understood. Thanks very much for the review and I will remember
next time to run it.sh.
salu2
--
Thorsten Scherler <thorsten.at.apache.org>
Open Source Java <consulting, training and solutions>
Sociedad Andaluza para el Desarrollo de la Sociedad
de la Información, S.A.U. (SADESI)
[c3] Test execution and dealing with optional dependencies
Posted by Reinhard Pötz <re...@apache.org>.
Thorsten Scherler wrote:
>
> On 13/10/2009, at 23:02, Reinhard Pötz wrote:
>
>> thorsten@apache.org wrote:
>>> Author: thorsten
>>> Date: Tue Oct 13 11:33:26 2009
>>> New Revision: 824705
>>>
>>> URL: http://svn.apache.org/viewvc?rev=824705&view=rev
>>> Log:
>>> COCOON3-43
>>> Provide a Consumer that interacts with solr
>>> Reporter and Patch-Provder: Bertil Chapuis
>>> Thanks Bertil.
>>>
>>
>>> + public void characters(char[] ch, int start, int length)
>>> + throws SAXException {
>>> + text += new String(Arrays.copyOfRange(ch, start, start +
>>> length));
>>> + getSAXConsumer().characters(ch, start, length);
>>> + }
>>
>> Arrays.copyOfRange() comes with JDK 1.6. Can you please replace it with
>> an 1.5 compliant alternative? Thanks!
>>
>
> Good catch, should be fixed now.
>
> Committed revision 824959.
Thorsten,
thanks for working on this patch. I've just fixed some problems with
missing license headers.
In order to spot these errors yourself in the future, you can run it.bat
or it.sh that activate all unit tests, integration tests and license
header checks (RAT plugin).
I also changed all dependencies of the cocoon-optional module to
<optional>true</optional> so that you do not add all the Solr stuff to
your classpath (or have to manually exclude them) just because you want
to use the FOP serializer for example.
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member reinhard@apache.org
________________________________________________________________________
Re: svn commit: r824705 - in /cocoon/cocoon3/trunk/cocoon-optional: ./ solr/ solr/conf/ src/main/java/org/apache/cocoon/optional/pipeline/components/sax/solr/ src/test/java/org/apache/cocoon/optional/pipeline/components/sax/solr/ src/test/resources/org/apa...
Posted by Thorsten Scherler <th...@apache.org>.
On 13/10/2009, at 23:02, Reinhard Pötz wrote:
> thorsten@apache.org wrote:
>> Author: thorsten
>> Date: Tue Oct 13 11:33:26 2009
>> New Revision: 824705
>>
>> URL: http://svn.apache.org/viewvc?rev=824705&view=rev
>> Log:
>> COCOON3-43
>> Provide a Consumer that interacts with solr
>> Reporter and Patch-Provder: Bertil Chapuis
>> Thanks Bertil.
>>
>
>> + public void characters(char[] ch, int start, int length)
>> + throws SAXException {
>> + text += new String(Arrays.copyOfRange(ch, start, start +
>> length));
>> + getSAXConsumer().characters(ch, start, length);
>> + }
>
> Arrays.copyOfRange() comes with JDK 1.6. Can you please replace it
> with
> an 1.5 compliant alternative? Thanks!
>
Good catch, should be fixed now.
Committed revision 824959.
salu2
> --
> Reinhard Pötz Managing Director, {Indoqa}
> GmbH
> http://www.indoqa.com/en/people/
> reinhard.poetz/
>
> Member of the Apache Software Foundation
> Apache Cocoon Committer, PMC member reinhard@apache.org
> ________________________________________________________________________
Re: svn commit: r824705 - in /cocoon/cocoon3/trunk/cocoon-optional:
./ solr/ solr/conf/ src/main/java/org/apache/cocoon/optional/pipeline/components/sax/solr/
src/test/java/org/apache/cocoon/optional/pipeline/components/sax/solr/ src/test/resources/org/apa...
Posted by Reinhard Pötz <re...@apache.org>.
thorsten@apache.org wrote:
> Author: thorsten
> Date: Tue Oct 13 11:33:26 2009
> New Revision: 824705
>
> URL: http://svn.apache.org/viewvc?rev=824705&view=rev
> Log:
> COCOON3-43
> Provide a Consumer that interacts with solr
> Reporter and Patch-Provder: Bertil Chapuis
> Thanks Bertil.
>
> + public void characters(char[] ch, int start, int length)
> + throws SAXException {
> + text += new String(Arrays.copyOfRange(ch, start, start + length));
> + getSAXConsumer().characters(ch, start, length);
> + }
Arrays.copyOfRange() comes with JDK 1.6. Can you please replace it with
an 1.5 compliant alternative? Thanks!
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member reinhard@apache.org
________________________________________________________________________