You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by rm...@apache.org on 2012/04/22 13:27:53 UTC
svn commit: r1328844 - in /lucene/dev/trunk/lucene: ./ queryparser/src/java/
queryparser/src/java/org/apache/lucene/queryparser/flexible/
queryparser/src/java/org/apache/lucene/queryparser/flexible/core/
queryparser/src/java/org/apache/lucene/querypars...
Author: rmuir
Date: Sun Apr 22 11:27:52 2012
New Revision: 1328844
URL: http://svn.apache.org/viewvc?rev=1328844&view=rev
Log:
expose previously-hidden javadocs in queryparser, fail the build if we have secret javadocs in the future, tidy up package.htmls in queryparser
Removed:
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/surround/package.html
Modified:
lucene/dev/trunk/lucene/common-build.xml
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/builders/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/config/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/messages/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/nodes/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/parser/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/util/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/precedence/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/precedence/processors/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/builders/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/config/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/nodes/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/parser/package.html
lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/processors/package.html
lucene/dev/trunk/lucene/queryparser/src/java/overview.html
Modified: lucene/dev/trunk/lucene/common-build.xml
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/common-build.xml?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/common-build.xml (original)
+++ lucene/dev/trunk/lucene/common-build.xml Sun Apr 22 11:27:52 2012
@@ -1290,6 +1290,7 @@ ${tests-output}/junit4-*.suites - pe
charset="${javadoc.charset}"
docencoding="${javadoc.charset}"
noindex="true"
+ includenosourcepackages="true"
author="true"
version="true"
linksource="@{linksource}"
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/builders/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/builders/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/builders/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/builders/package.html Sun Apr 22 11:27:52 2012
@@ -21,7 +21,7 @@
</head>
<body>
-Contains the necessary classes to implement query builders
+Necessary classes to implement query builders.
<h2>Query Parser Builders</h2>
<p>
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/config/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/config/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/config/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/config/package.html Sun Apr 22 11:27:52 2012
@@ -21,7 +21,7 @@
</head>
<body>
-Contains the base classes used to configure the query processing
+Base classes used to configure the query processing.
<h2>Query Configuration Interfaces</h2>
<p>
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/messages/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/messages/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/messages/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/messages/package.html Sun Apr 22 11:27:52 2012
@@ -21,7 +21,7 @@
</head>
<body>
-Contains messages usually used by query parser implementations
+Messages usually used by query parser implementations.
<h2>Query Parser Messages</h2>
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/nodes/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/nodes/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/nodes/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/nodes/package.html Sun Apr 22 11:27:52 2012
@@ -21,7 +21,7 @@
</head>
<body>
-Contains query nodes that are commonly used by query parser implementations
+Query nodes commonly used by query parser implementations.
<h2>Query Nodes</h2>
<p>
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/package.html Sun Apr 22 11:27:52 2012
@@ -21,7 +21,7 @@
</head>
<body>
-Contains the core classes of the flexible query parser framework
+Core classes of the flexible query parser framework.
<h2>Flexible Query Parser</h2>
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/parser/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/parser/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/parser/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/parser/package.html Sun Apr 22 11:27:52 2012
@@ -21,7 +21,7 @@
</head>
<body>
-Contains the necessary interfaces to implement text parsers
+Necessary interfaces to implement text parsers.
<h2>Parser</h2>
<p>
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/util/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/util/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/util/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/core/util/package.html Sun Apr 22 11:27:52 2012
@@ -20,7 +20,7 @@
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body>
-Utility classes to used with the Query Parser
+Utility classes to used with the Query Parser.
<h2>Utility classes to used with the Query Parser</h2>
<p>
This package contains utility classes used with the query parsers.
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/precedence/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/precedence/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/precedence/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/precedence/package.html Sun Apr 22 11:27:52 2012
@@ -21,7 +21,7 @@
</head>
<body>
-This package contains the Precedence Query Parser Implementation
+Precedence Query Parser Implementation
<h2>Lucene Precedence Query Parser</h2>
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/precedence/processors/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/precedence/processors/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/precedence/processors/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/precedence/processors/package.html Sun Apr 22 11:27:52 2012
@@ -21,7 +21,7 @@
</head>
<body>
-This package contains the processors used by Precedence Query Parser
+Processors used by Precedence Query Parser
<h2>Lucene Precedence Query Parser Processors</h2>
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/builders/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/builders/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/builders/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/builders/package.html Sun Apr 22 11:27:52 2012
@@ -21,6 +21,8 @@
</head>
<body>
+Standard Lucene Query Node Builders.
+
<h2>Standard Lucene Query Node Builders</h2>
<p>
The package org.apache.lucene.queryparser.flexible.standard.builders contains all the builders needed
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/config/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/config/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/config/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/config/package.html Sun Apr 22 11:27:52 2012
@@ -21,6 +21,8 @@
</head>
<body>
+Standard Lucene Query Configuration.
+
<h2>Standard Lucene Query Configuration</h2>
<p>
The package org.apache.lucene.queryparser.flexible.standard.config contains the Lucene
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/nodes/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/nodes/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/nodes/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/nodes/package.html Sun Apr 22 11:27:52 2012
@@ -21,6 +21,8 @@
</head>
<body>
+Standard Lucene Query Nodes.
+
<h2>Standard Lucene Query Nodes</h2>
<p>
The package org.apache.lucene.queryparser.flexible.standard.nodes contains QueryNode classes
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/package.html Sun Apr 22 11:27:52 2012
@@ -21,7 +21,7 @@
</head>
<body>
-Contains the implementation of the {@linkplain org.apache.lucene.queryparser.classic Lucene classic query parser} using the flexible query parser frameworks
+Implementation of the {@linkplain org.apache.lucene.queryparser.classic Lucene classic query parser} using the flexible query parser frameworks
<h2>Lucene Flexible Query Parser Implementation</h2>
<p>
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/parser/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/parser/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/parser/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/parser/package.html Sun Apr 22 11:27:52 2012
@@ -21,6 +21,8 @@
</head>
<body>
+Lucene Query Parser.
+
<h2>Lucene Query Parser</h2>
<p>
The package org.apache.lucene.queryparser.flexible.standard.parser contains the query parser.
Modified: lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/processors/package.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/processors/package.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/processors/package.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/org/apache/lucene/queryparser/flexible/standard/processors/package.html Sun Apr 22 11:27:52 2012
@@ -21,6 +21,8 @@
</head>
<body>
+Lucene Query Node Processors.
+
<h2>Lucene Query Node Processors</h2>
<p>
The package org.apache.lucene.queryparser.flexible.standard.processors contains every processor needed to assembly a pipeline
Modified: lucene/dev/trunk/lucene/queryparser/src/java/overview.html
URL: http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/queryparser/src/java/overview.html?rev=1328844&r1=1328843&r2=1328844&view=diff
==============================================================================
--- lucene/dev/trunk/lucene/queryparser/src/java/overview.html (original)
+++ lucene/dev/trunk/lucene/queryparser/src/java/overview.html Sun Apr 22 11:27:52 2012
@@ -21,6 +21,148 @@
</title>
</head>
<body>
- QueryParsers
+ Apache Lucene QueryParsers.
+ <p>
+ This module provides a number of queryparsers:
+ <ul>
+ <li><a href="#classic">Classic</a>
+ <li><a href="#analyzing">Analyzing</a>
+ <li><a href="#complexphrase">Complex Phrase</a>
+ <li><a href="#extendable">Extendable</a>
+ <li><a href="#flexible">Flexible</a>
+ <li><a href="#surround">Surround</a>
+ <li><a href="#xml">XML</a>
+ </ul>
+ <hr/>
+ <h2><a name="classic">Classic</a></h2>
+ A Simple Lucene QueryParser implemented with JavaCC.
+ <h2><a name="analyzing">Analyzing</a></h2>
+ QueryParser that passes Fuzzy-, Prefix-, Range-, and WildcardQuerys through the given analyzer.
+ <h2><a name="complexphrase">Complex Phrase</a></h2>
+ QueryParser which permits complex phrase query syntax eg "(john jon jonathan~) peters*"
+ <h2><a name="extendable">Extendable</a></h2>
+ Extendable QueryParser provides a simple and flexible extension mechanism by overloading query field names.
+ <h2><a name="flexible">Flexible</a></h2>
+<p>
+This project contains the new Lucene query parser implementation, which matches the syntax of the core QueryParser but offers a more modular architecture to enable customization.
+</p>
+
+<p>
+It's currently divided in 2 main packages:
+<ul>
+<li>{@link org.apache.lucene.queryparser.flexible.core}: it contains the query parser API classes, which should be extended by query parser implementations. </li>
+<li>{@link org.apache.lucene.queryparser.flexible.standard}: it contains the current Lucene query parser implementation using the new query parser API.</li>
+</ul>
+</p>
+
+<h3>Features</h3>
+
+ <ol>
+ <li>Full support for boolean logic (not enabled)</li>
+ <li>QueryNode Trees - support for several syntaxes,
+ that can be converted into similar syntax QueryNode trees.</li>
+ <li>QueryNode Processors - Optimize, validate, rewrite the
+ QueryNode trees</li>
+ <li>Processors Pipelines - Select your favorite Processor
+ and build a processor pipeline, to implement the features you need</li>
+ <li>Config Interfaces - Allow the consumer of the Query Parser to implement
+ a diff Config Handler Objects to suite their needs.</li>
+ <li>Standard Builders - convert QueryNode's into several lucene
+ representations. Supported conversion is using a 2.4 compatible logic</li>
+ <li>QueryNode tree's can be converted to a lucene 2.4 syntax string, using toQueryString</li>
+ </ol>
+
+<h3>Design</h3>
+<p>
+This new query parser was designed to have very generic
+architecture, so that it can be easily used for different
+products with varying query syntaxes. This code is much more
+flexible and extensible than the Lucene query parser in 2.4.X.
+</p>
+<p>
+The new query parser goal is to separate syntax and semantics of a query. E.g. 'a AND
+b', '+a +b', 'AND(a,b)' could be different syntaxes for the same query.
+It distinguishes the semantics of the different query components, e.g.
+whether and how to tokenize/lemmatize/normalize the different terms or
+which Query objects to create for the terms. It allows to
+write a parser with a new syntax, while reusing the underlying
+semantics, as quickly as possible.
+</p>
+<p>
+The query parser has three layers and its core is what we call the
+QueryNode tree. It is a tree that initially represents the syntax of the
+original query, e.g. for 'a AND b':
+</p>
+<pre>
+ AND
+ / \
+ A B
+</pre>
+<p>
+The three layers are:
+</p>
+<dl>
+<dt>QueryParser</dt>
+<dd>
+This layer is the text parsing layer which simply transforms the
+query text string into a {@link org.apache.lucene.queryparser.flexible.core.nodes.QueryNode} tree. Every text parser
+must implement the interface {@link org.apache.lucene.queryparser.flexible.core.parser.SyntaxParser}.
+Lucene default implementations implements it using JavaCC.
+</dd>
+
+<dt>QueryNodeProcessor</dt>
+<dd>The query node processors do most of the work. It is in fact a
+configurable chain of processors. Each processors can walk the tree and
+modify nodes or even the tree's structure. That makes it possible to
+e.g. do query optimization before the query is executed or to tokenize
+terms.
+</dd>
+
+<dt>QueryBuilder</dt>
+<dd>
+The third layer is a configurable map of builders, which map {@link org.apache.lucene.queryparser.flexible.core.nodes.QueryNode} types to its specific
+builder that will transform the QueryNode into Lucene Query object.
+</dd>
+
+</dl>
+
+<p>
+Furthermore, the query parser uses flexible configuration objects. It also uses message classes that
+allow to attach resource bundles. This makes it possible to translate
+messages, which is an important feature of a query parser.
+</p>
+<p>
+This design allows to develop different query syntaxes very quickly.
+</p>
+
+<h3>StandardQueryParser and QueryParserWrapper</h3>
+
+<p>
+The classic Lucene query parser is located under
+{@link org.apache.lucene.queryparser.classic}.
+<p>
+To make it simpler to use the new query parser
+the class {@link org.apache.lucene.queryparser.flexible.standard.StandardQueryParser} may be helpful,
+specially for people that do not want to extend the Query Parser.
+It uses the default Lucene query processors, text parser and builders, so
+you don't need to worry about dealing with those.
+
+{@link org.apache.lucene.queryparser.flexible.standard.StandardQueryParser} usage:
+
+<pre class="prettyprint">
+ StandardQueryParser qpHelper = new StandardQueryParser();
+ StandardQueryConfigHandler config = qpHelper.getQueryConfigHandler();
+ config.setAllowLeadingWildcard(true);
+ config.setAnalyzer(new WhitespaceAnalyzer());
+ Query query = qpHelper.parse("apache AND lucene", "defaultField");
+</pre>
+<h2><a name="surround">Surround</a></h2>
+<p>
+A QueryParser that supports the Span family of queries as well as pre and infix notation.
+</p>
+<h2><a name="xml">XML</a></h2>
+A QueryParser that produces Lucene Query objects from XML streams.
+<p>
+</p>
</body>
</html>