You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucy.apache.org by nw...@apache.org on 2016/02/26 14:44:36 UTC
svn commit: r1732480 [1/6] - in /lucy/site/trunk/content/docs/test: ./ Lucy/
Lucy/Analysis/ Lucy/Docs/ Lucy/Docs/Cookbook/ Lucy/Docs/Tutorial/
Lucy/Document/ Lucy/Highlight/ Lucy/Index/ Lucy/Object/ Lucy/Plan/
Lucy/Search/ Lucy/Search/Collector/ Lucy/S...
Author: nwellnhof
Date: Fri Feb 26 13:44:34 2016
New Revision: 1732480
URL: http://svn.apache.org/viewvc?rev=1732480&view=rev
Log:
Regen mdtext files with strip_verbatim_indent
Modified:
lucy/site/trunk/content/docs/test/Lucy.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/Analyzer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/CaseFolder.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/EasyAnalyzer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/Inversion.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/Normalizer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/PolyAnalyzer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/RegexTokenizer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStemmer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStopFilter.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/StandardTokenizer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/Token.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQueryParser.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/FastUpdates.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/DevGuide.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/DocIDs.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/FileFormat.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/FileLocking.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/AnalysisTutorial.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/BeyondSimpleTutorial.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/FieldTypeTutorial.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/HighlighterTutorial.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/QueryObjectsTutorial.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/SimpleTutorial.mdtext
lucy/site/trunk/content/docs/test/Lucy/Document/Doc.mdtext
lucy/site/trunk/content/docs/test/Lucy/Document/HitDoc.mdtext
lucy/site/trunk/content/docs/test/Lucy/Highlight/Highlighter.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/BackgroundMerger.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/DataReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/DataWriter.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/DeletionsWriter.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/DocReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/IndexManager.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/IndexReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/Indexer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/Lexicon.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/LexiconReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/PolyReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/PostingList.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/PostingListReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/SegReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/SegWriter.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/Segment.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/Similarity.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/Snapshot.mdtext
lucy/site/trunk/content/docs/test/Lucy/Object/BitVector.mdtext
lucy/site/trunk/content/docs/test/Lucy/Plan/Architecture.mdtext
lucy/site/trunk/content/docs/test/Lucy/Plan/BlobType.mdtext
lucy/site/trunk/content/docs/test/Lucy/Plan/FieldType.mdtext
lucy/site/trunk/content/docs/test/Lucy/Plan/FullTextType.mdtext
lucy/site/trunk/content/docs/test/Lucy/Plan/Schema.mdtext
lucy/site/trunk/content/docs/test/Lucy/Plan/StringType.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/ANDQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Collector.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Collector/BitCollector.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Compiler.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Hits.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/IndexSearcher.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/LeafQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/MatchAllQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Matcher.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/NOTQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/NoMatchQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/ORQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/PhraseQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/PolyQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/PolySearcher.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Query.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/QueryParser.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/RangeQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/RequiredOptionalQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Searcher.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/SortRule.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/SortSpec.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Span.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/TermQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Simple.mdtext
lucy/site/trunk/content/docs/test/Lucy/Store/FSFolder.mdtext
lucy/site/trunk/content/docs/test/Lucy/Store/Folder.mdtext
lucy/site/trunk/content/docs/test/Lucy/Store/Lock.mdtext
lucy/site/trunk/content/docs/test/Lucy/Store/LockErr.mdtext
lucy/site/trunk/content/docs/test/Lucy/Store/LockFactory.mdtext
lucy/site/trunk/content/docs/test/Lucy/Store/RAMFolder.mdtext
lucy/site/trunk/content/docs/test/LucyX/Index/ByteBufDocReader.mdtext
lucy/site/trunk/content/docs/test/LucyX/Index/ByteBufDocWriter.mdtext
lucy/site/trunk/content/docs/test/LucyX/Index/LongFieldSim.mdtext
lucy/site/trunk/content/docs/test/LucyX/Remote/ClusterSearcher.mdtext
lucy/site/trunk/content/docs/test/LucyX/Remote/SearchClient.mdtext
lucy/site/trunk/content/docs/test/LucyX/Remote/SearchServer.mdtext
lucy/site/trunk/content/docs/test/LucyX/Search/Filter.mdtext
lucy/site/trunk/content/docs/test/LucyX/Search/ProximityQuery.mdtext
Modified: lucy/site/trunk/content/docs/test/Lucy.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy.mdtext Fri Feb 26 13:44:34 2016
@@ -24,64 +24,67 @@ plan out your index structure,
create the index,
and add documents:</p>
-<pre> # indexer.pl
-
- use Lucy::Index::Indexer;
- use Lucy::Plan::Schema;
- use Lucy::Analysis::EasyAnalyzer;
- use Lucy::Plan::FullTextType;
-
- # Create a Schema which defines index fields.
- my $schema = Lucy::Plan::Schema->new;
- my $easyanalyzer = Lucy::Analysis::EasyAnalyzer->new(
- language => 'en',
- );
- my $type = Lucy::Plan::FullTextType->new(
- analyzer => $easyanalyzer,
- );
- $schema->spec_field( name => 'title', type => $type );
- $schema->spec_field( name => 'content', type => $type );
-
- # Create the index and add documents.
- my $indexer = Lucy::Index::Indexer->new(
- schema => $schema,
- index => '/path/to/index',
- create => 1,
- );
- while ( my ( $title, $content ) = each %source_docs ) {
- $indexer->add_doc({
- title => $title,
- content => $content,
- });
- }
- $indexer->commit;</pre>
-
-<p>Then, search the index:</p>
-
-<pre> # search.pl
-
- use Lucy::Search::IndexSearcher;
-
- my $searcher = Lucy::Search::IndexSearcher->new(
- index => '/path/to/index'
- );
- my $hits = $searcher->hits( query => "foo bar" );
- while ( my $hit = $hits->next ) {
- print "$hit->{title}\n";
- }</pre>
+<pre># indexer.pl
+
+use Lucy::Index::Indexer;
+use Lucy::Plan::Schema;
+use Lucy::Analysis::EasyAnalyzer;
+use Lucy::Plan::FullTextType;
+
+# Create a Schema which defines index fields.
+my $schema = Lucy::Plan::Schema->new;
+my $easyanalyzer = Lucy::Analysis::EasyAnalyzer->new(
+ language => 'en',
+);
+my $type = Lucy::Plan::FullTextType->new(
+ analyzer => $easyanalyzer,
+);
+$schema->spec_field( name => 'title', type => $type );
+$schema->spec_field( name => 'content', type => $type );
+
+# Create the index and add documents.
+my $indexer = Lucy::Index::Indexer->new(
+ schema => $schema,
+ index => '/path/to/index',
+ create => 1,
+);
+while ( my ( $title, $content ) = each %source_docs ) {
+ $indexer->add_doc({
+ title => $title,
+ content => $content,
+ });
+}
+$indexer->commit;</pre>
+
+<p>Then,
+search the index:</p>
+
+<pre># search.pl
+
+use Lucy::Search::IndexSearcher;
+
+my $searcher = Lucy::Search::IndexSearcher->new(
+ index => '/path/to/index'
+);
+my $hits = $searcher->hits( query => "foo bar" );
+while ( my $hit = $hits->next ) {
+ print "$hit->{title}\n";
+}</pre>
<h2><a class='u'
name="DESCRIPTION"
>DESCRIPTION</a></h2>
-<p>The Apache Lucy search engine library delivers high-performance, modular full-text search.</p>
+<p>The Apache Lucy search engine library delivers high-performance,
+modular full-text search.</p>
<h3><a class='u'
name="Features"
>Features</a></h3>
<ul>
-<li>Extremely fast. A single machine can handle millions of documents.</li>
+<li>Extremely fast.
+A single machine can handle millions of documents.</li>
<li>Scalable to multiple machines.</li>
@@ -91,7 +94,9 @@ name="Features"
<li>Unicode support.</li>
-<li>Support for boolean operators AND, OR, and AND NOT; parenthetical groupings; prepended +plus and -minus.</li>
+<li>Support for boolean operators AND,
+OR,
+and AND NOT; parenthetical groupings; prepended +plus and -minus.</li>
<li>Algorithmic selection of relevant excerpts and highlighting of search terms within excerpts.</li>
@@ -142,52 +147,77 @@ name="Delving_Deeper"
<p><a href="./Lucy/Docs/Cookbook.html" class="podlinkpod"
>Lucy::Docs::Cookbook</a> augments the tutorial with more advanced recipes.</p>
-<p>For creating complex queries, see <a href="./Lucy/Search/Query.html" class="podlinkpod"
+<p>For creating complex queries,
+see <a href="./Lucy/Search/Query.html" class="podlinkpod"
>Lucy::Search::Query</a> and its subclasses <a href="./Lucy/Search/TermQuery.html" class="podlinkpod"
->TermQuery</a>, <a href="./Lucy/Search/PhraseQuery.html" class="podlinkpod"
->PhraseQuery</a>, <a href="./Lucy/Search/ANDQuery.html" class="podlinkpod"
->ANDQuery</a>, <a href="./Lucy/Search/ORQuery.html" class="podlinkpod"
->ORQuery</a>, <a href="./Lucy/Search/NOTQuery.html" class="podlinkpod"
->NOTQuery</a>, <a href="./Lucy/Search/RequiredOptionalQuery.html" class="podlinkpod"
->RequiredOptionalQuery</a>, <a href="./Lucy/Search/MatchAllQuery.html" class="podlinkpod"
->MatchAllQuery</a>, and <a href="./Lucy/Search/NoMatchQuery.html" class="podlinkpod"
->NoMatchQuery</a>, plus <a href="./Lucy/Search/QueryParser.html" class="podlinkpod"
+>TermQuery</a>,
+<a href="./Lucy/Search/PhraseQuery.html" class="podlinkpod"
+>PhraseQuery</a>,
+<a href="./Lucy/Search/ANDQuery.html" class="podlinkpod"
+>ANDQuery</a>,
+<a href="./Lucy/Search/ORQuery.html" class="podlinkpod"
+>ORQuery</a>,
+<a href="./Lucy/Search/NOTQuery.html" class="podlinkpod"
+>NOTQuery</a>,
+<a href="./Lucy/Search/RequiredOptionalQuery.html" class="podlinkpod"
+>RequiredOptionalQuery</a>,
+<a href="./Lucy/Search/MatchAllQuery.html" class="podlinkpod"
+>MatchAllQuery</a>,
+and <a href="./Lucy/Search/NoMatchQuery.html" class="podlinkpod"
+>NoMatchQuery</a>,
+plus <a href="./Lucy/Search/QueryParser.html" class="podlinkpod"
>Lucy::Search::QueryParser</a>.</p>
-<p>For distributed searching, see <a href="./LucyX/Remote/SearchServer.html" class="podlinkpod"
->LucyX::Remote::SearchServer</a>, <a href="./LucyX/Remote/SearchClient.html" class="podlinkpod"
->LucyX::Remote::SearchClient</a>, and <a href="./LucyX/Remote/ClusterSearcher.html" class="podlinkpod"
+<p>For distributed searching,
+see <a href="./LucyX/Remote/SearchServer.html" class="podlinkpod"
+>LucyX::Remote::SearchServer</a>,
+<a href="./LucyX/Remote/SearchClient.html" class="podlinkpod"
+>LucyX::Remote::SearchClient</a>,
+and <a href="./LucyX/Remote/ClusterSearcher.html" class="podlinkpod"
>LucyX::Remote::ClusterSearcher</a>.</p>
<h3><a class='u'
name="Backwards_Compatibility_Policy"
>Backwards Compatibility Policy</a></h3>
-<p>Lucy will spin off stable forks into new namespaces periodically. The first will be named "Lucy1". Users who require strong backwards compatibility should use a stable fork.</p>
-
-<p>The main namespace, "Lucy", is an API-unstable development branch (as hinted at by its 0.x.x version number). Superficial interface changes happen frequently. Hard file format compatibility breaks which require reindexing are rare, as we generally try to provide continuity across multiple releases, but we reserve the right to make such changes.</p>
+<p>Lucy will spin off stable forks into new namespaces periodically.
+The first will be named "Lucy1".
+Users who require strong backwards compatibility should use a stable fork.</p>
+
+<p>The main namespace,
+"Lucy",
+is an API-unstable development branch (as hinted at by its 0.x.x version number).
+Superficial interface changes happen frequently.
+Hard file format compatibility breaks which require reindexing are rare,
+as we generally try to provide continuity across multiple releases,
+but we reserve the right to make such changes.</p>
<h2><a class='u'
name="CLASS_METHODS"
>CLASS METHODS</a></h2>
-<p>The Lucy module itself does not have a large interface, providing only a single public class method.</p>
+<p>The Lucy module itself does not have a large interface,
+providing only a single public class method.</p>
<h3><a class='u'
name="error"
>error</a></h3>
-<pre> my $instream = $folder->open_in( file => 'foo' ) or die Clownfish->error;</pre>
+<pre>my $instream = $folder->open_in( file => 'foo' ) or die Clownfish->error;</pre>
-<p>Access a shared variable which is set by some routines on failure. It will always be either a <a href="./Clownfish/Err.html" class="podlinkpod"
+<p>Access a shared variable which is set by some routines on failure.
+It will always be either a <a href="./Clownfish/Err.html" class="podlinkpod"
>Clownfish::Err</a> object or undef.</p>
<h2><a class='u'
name="SUPPORT"
>SUPPORT</a></h2>
-<p>The Apache Lucy homepage, where you'll find links to our mailing lists and so on, is <a href="http://lucy.apache.org" class="podlinkurl"
->http://lucy.apache.org</a>. Please direct support questions to the Lucy users mailing list.</p>
+<p>The Apache Lucy homepage,
+where you'll find links to our mailing lists and so on,
+is <a href="http://lucy.apache.org" class="podlinkurl"
+>http://lucy.apache.org</a>.
+Please direct support questions to the Lucy users mailing list.</p>
<h2><a class='u'
name="BUGS"
@@ -197,13 +227,17 @@ name="BUGS"
<p>Some exceptions leak memory.</p>
-<p>If you find a bug, please inquire on the Lucy users mailing list about it, then report it on the Lucy issue tracker once it has been confirmed: <a href="https://issues.apache.org/jira/browse/LUCY" class="podlinkurl"
+<p>If you find a bug,
+please inquire on the Lucy users mailing list about it,
+then report it on the Lucy issue tracker once it has been confirmed: <a href="https://issues.apache.org/jira/browse/LUCY" class="podlinkurl"
>https://issues.apache.org/jira/browse/LUCY</a>.</p>
<h2><a class='u'
name="COPYRIGHT"
>COPYRIGHT</a></h2>
-<p>Apache Lucy is distributed under the Apache License, Version 2.0, as described in the file <code>LICENSE</code> included with the distribution.</p>
+<p>Apache Lucy is distributed under the Apache License,
+Version 2.0,
+as described in the file <code>LICENSE</code> included with the distribution.</p>
</div>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/Analyzer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/Analyzer.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/Analyzer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/Analyzer.mdtext Fri Feb 26 13:44:34 2016
@@ -13,14 +13,18 @@ name="NAME"
name="SYNOPSIS"
>SYNOPSIS</a></h2>
-<pre> # Abstract base class.</pre>
+<pre># Abstract base class.</pre>
<h2><a class='u'
name="DESCRIPTION"
>DESCRIPTION</a></h2>
-<p>An Analyzer is a filter which processes text, transforming it from one form into another. For instance, an analyzer might break up a long text into smaller pieces (<a href="../../Lucy/Analysis/RegexTokenizer.html" class="podlinkpod"
->RegexTokenizer</a>), or it might perform case folding to facilitate case-insensitive search (<a href="../../Lucy/Analysis/Normalizer.html" class="podlinkpod"
+<p>An Analyzer is a filter which processes text,
+transforming it from one form into another.
+For instance,
+an analyzer might break up a long text into smaller pieces (<a href="../../Lucy/Analysis/RegexTokenizer.html" class="podlinkpod"
+>RegexTokenizer</a>),
+or it might perform case folding to facilitate case-insensitive search (<a href="../../Lucy/Analysis/Normalizer.html" class="podlinkpod"
>Normalizer</a>).</p>
<h2><a class='u'
@@ -31,17 +35,18 @@ name="CONSTRUCTORS"
name="new"
>new</a></h3>
-<pre> package MyAnalyzer;
- use base qw( Lucy::Analysis::Analyzer );
- our %foo;
- sub new {
- my $self = shift->SUPER::new;
- my %args = @_;
- $foo{$$self} = $args{foo};
- return $self;
- }</pre>
+<pre>package MyAnalyzer;
+use base qw( Lucy::Analysis::Analyzer );
+our %foo;
+sub new {
+ my $self = shift->SUPER::new;
+ my %args = @_;
+ $foo{$$self} = $args{foo};
+ return $self;
+}</pre>
-<p>Abstract constructor. Takes no arguments.</p>
+<p>Abstract constructor.
+Takes no arguments.</p>
<h2><a class='u'
name="ABSTRACT_METHODS"
@@ -51,10 +56,12 @@ name="ABSTRACT_METHODS"
name="transform"
>transform</a></h3>
-<pre> my $retval = $analyzer->transform($inversion);</pre>
+<pre>my $retval = $analyzer->transform($inversion);</pre>
<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
->Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+>Inversion</a> as input and returns an Inversion,
+either the same one (presumably transformed in some way),
+or a new one.</p>
<ul>
<li><b>inversion</b> - An inversion.</li>
@@ -68,10 +75,14 @@ name="METHODS"
name="transform_text"
>transform_text</a></h3>
-<pre> my $retval = $analyzer->transform_text($text);</pre>
+<pre>my $retval = $analyzer->transform_text($text);</pre>
-<p>Kick off an analysis chain, creating an Inversion from string input. The default implementation simply creates an initial Inversion with a single Token, then calls <a href="#transform" class="podlinkpod"
->transform()</a>, but occasionally subclasses will provide an optimized implementation which minimizes string copies.</p>
+<p>Kick off an analysis chain,
+creating an Inversion from string input.
+The default implementation simply creates an initial Inversion with a single Token,
+then calls <a href="#transform" class="podlinkpod"
+>transform()</a>,
+but occasionally subclasses will provide an optimized implementation which minimizes string copies.</p>
<ul>
<li><b>text</b> - A string.</li>
@@ -81,7 +92,7 @@ name="transform_text"
name="split"
>split</a></h3>
-<pre> my $retval = $analyzer->split($text);</pre>
+<pre>my $retval = $analyzer->split($text);</pre>
<p>Analyze text and return an array of token texts.</p>
@@ -93,12 +104,13 @@ name="split"
name="dump"
>dump</a></h3>
-<pre> my $retval = $analyzer->dump();</pre>
+<pre>my $retval = $analyzer->dump();</pre>
<p>Dump the analyzer as hash.</p>
<p>Subclasses should call <a href="#dump" class="podlinkpod"
->dump()</a> on the superclass. The returned object is a hash which should be populated with parameters of the analyzer.</p>
+>dump()</a> on the superclass.
+The returned object is a hash which should be populated with parameters of the analyzer.</p>
<p>Returns: A hash containing a description of the analyzer.</p>
@@ -106,12 +118,13 @@ name="dump"
name="load"
>load</a></h3>
-<pre> my $retval = $analyzer->load($dump);</pre>
+<pre>my $retval = $analyzer->load($dump);</pre>
<p>Reconstruct an analyzer from a dump.</p>
<p>Subclasses should first call <a href="#load" class="podlinkpod"
->load()</a> on the superclass. The returned object is an analyzer which should be reconstructed by setting the dumped parameters from the hash contained in <code>dump</code>.</p>
+>load()</a> on the superclass.
+The returned object is an analyzer which should be reconstructed by setting the dumped parameters from the hash contained in <code>dump</code>.</p>
<p>Note that the invocant analyzer is unused.</p>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/CaseFolder.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/CaseFolder.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/CaseFolder.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/CaseFolder.mdtext Fri Feb 26 13:44:34 2016
@@ -14,20 +14,22 @@ facilitating case-insensitive search.</p
name="SYNOPSIS"
>SYNOPSIS</a></h2>
-<pre> my $case_folder = Lucy::Analysis::CaseFolder->new;
+<pre>my $case_folder = Lucy::Analysis::CaseFolder->new;
- my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
- analyzers => [ $tokenizer, $case_folder, $stemmer ],
- );</pre>
+my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
+ analyzers => [ $tokenizer, $case_folder, $stemmer ],
+);</pre>
<h2><a class='u'
name="DESCRIPTION"
>DESCRIPTION</a></h2>
-<p>CaseFolder is DEPRECATED. Use <a href="../../Lucy/Analysis/Normalizer.html" class="podlinkpod"
+<p>CaseFolder is DEPRECATED.
+Use <a href="../../Lucy/Analysis/Normalizer.html" class="podlinkpod"
>Normalizer</a> instead.</p>
-<p>CaseFolder normalizes text according to Unicode case-folding rules, so that searches will be case-insensitive.</p>
+<p>CaseFolder normalizes text according to Unicode case-folding rules,
+so that searches will be case-insensitive.</p>
<h2><a class='u'
name="CONSTRUCTORS"
@@ -37,9 +39,10 @@ name="CONSTRUCTORS"
name="new"
>new</a></h3>
-<pre> my $case_folder = Lucy::Analysis::CaseFolder->new;</pre>
+<pre>my $case_folder = Lucy::Analysis::CaseFolder->new;</pre>
-<p>Constructor. Takes no arguments.</p>
+<p>Constructor.
+Takes no arguments.</p>
<h2><a class='u'
name="METHODS"
@@ -49,10 +52,12 @@ name="METHODS"
name="transform"
>transform</a></h3>
-<pre> my $retval = $case_folder->transform($inversion);</pre>
+<pre>my $retval = $case_folder->transform($inversion);</pre>
<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
->Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+>Inversion</a> as input and returns an Inversion,
+either the same one (presumably transformed in some way),
+or a new one.</p>
<ul>
<li><b>inversion</b> - An inversion.</li>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/EasyAnalyzer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/EasyAnalyzer.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/EasyAnalyzer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/EasyAnalyzer.mdtext Fri Feb 26 13:44:34 2016
@@ -13,42 +13,44 @@ name="NAME"
name="SYNOPSIS"
>SYNOPSIS</a></h2>
-<pre> my $schema = Lucy::Plan::Schema->new;
- my $analyzer = Lucy::Analysis::EasyAnalyzer->new(
- language => 'en',
- );
- my $type = Lucy::Plan::FullTextType->new(
- analyzer => $analyzer,
- );
- $schema->spec_field( name => 'title', type => $type );
- $schema->spec_field( name => 'content', type => $type );</pre>
+<pre>my $schema = Lucy::Plan::Schema->new;
+my $analyzer = Lucy::Analysis::EasyAnalyzer->new(
+ language => 'en',
+);
+my $type = Lucy::Plan::FullTextType->new(
+ analyzer => $analyzer,
+);
+$schema->spec_field( name => 'title', type => $type );
+$schema->spec_field( name => 'content', type => $type );</pre>
<h2><a class='u'
name="DESCRIPTION"
>DESCRIPTION</a></h2>
<p>EasyAnalyzer is an analyzer chain consisting of a <a href="../../Lucy/Analysis/StandardTokenizer.html" class="podlinkpod"
->StandardTokenizer</a>, a <a href="../../Lucy/Analysis/Normalizer.html" class="podlinkpod"
->Normalizer</a>, and a <a href="../../Lucy/Analysis/SnowballStemmer.html" class="podlinkpod"
+>StandardTokenizer</a>,
+a <a href="../../Lucy/Analysis/Normalizer.html" class="podlinkpod"
+>Normalizer</a>,
+and a <a href="../../Lucy/Analysis/SnowballStemmer.html" class="podlinkpod"
>SnowballStemmer</a>.</p>
<p>Supported languages:</p>
-<pre> en => English,
- da => Danish,
- de => German,
- es => Spanish,
- fi => Finnish,
- fr => French,
- hu => Hungarian,
- it => Italian,
- nl => Dutch,
- no => Norwegian,
- pt => Portuguese,
- ro => Romanian,
- ru => Russian,
- sv => Swedish,
- tr => Turkish,</pre>
+<pre>en => English,
+da => Danish,
+de => German,
+es => Spanish,
+fi => Finnish,
+fr => French,
+hu => Hungarian,
+it => Italian,
+nl => Dutch,
+no => Norwegian,
+pt => Portuguese,
+ro => Romanian,
+ru => Russian,
+sv => Swedish,
+tr => Turkish,</pre>
<h2><a class='u'
name="CONSTRUCTORS"
@@ -58,9 +60,9 @@ name="CONSTRUCTORS"
name="new"
>new</a></h3>
-<pre> my $analyzer = Lucy::Analysis::EasyAnalyzer->new(
- language => 'es',
- );</pre>
+<pre>my $analyzer = Lucy::Analysis::EasyAnalyzer->new(
+ language => 'es',
+);</pre>
<p>Create a new EasyAnalyzer.</p>
@@ -76,10 +78,12 @@ name="METHODS"
name="transform"
>transform</a></h3>
-<pre> my $retval = $easy_analyzer->transform($inversion);</pre>
+<pre>my $retval = $easy_analyzer->transform($inversion);</pre>
<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
->Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+>Inversion</a> as input and returns an Inversion,
+either the same one (presumably transformed in some way),
+or a new one.</p>
<ul>
<li><b>inversion</b> - An inversion.</li>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/Inversion.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/Inversion.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/Inversion.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/Inversion.mdtext Fri Feb 26 13:44:34 2016
@@ -13,17 +13,18 @@ name="NAME"
name="SYNOPSIS"
>SYNOPSIS</a></h2>
-<pre> my $result = Lucy::Analysis::Inversion->new;
+<pre>my $result = Lucy::Analysis::Inversion->new;
- while (my $token = $inversion->next) {
- $result->append($token);
- }</pre>
+while (my $token = $inversion->next) {
+ $result->append($token);
+}</pre>
<h2><a class='u'
name="DESCRIPTION"
>DESCRIPTION</a></h2>
-<p>An Inversion is a collection of Token objects which you can add to, then iterate over.</p>
+<p>An Inversion is a collection of Token objects which you can add to,
+then iterate over.</p>
<h2><a class='u'
name="CONSTRUCTORS"
@@ -33,14 +34,15 @@ name="CONSTRUCTORS"
name="new"
>new</a></h3>
-<pre> my $inversion = Lucy::Analysis::Inversion->new(
- $seed, # optional
- );</pre>
+<pre>my $inversion = Lucy::Analysis::Inversion->new(
+ $seed, # optional
+);</pre>
<p>Create a new Inversion.</p>
<ul>
-<li><b>seed</b> - An initial Token to start things off, which may be undef.</li>
+<li><b>seed</b> - An initial Token to start things off,
+which may be undef.</li>
</ul>
<h2><a class='u'
@@ -51,7 +53,7 @@ name="METHODS"
name="append"
>append</a></h3>
-<pre> $inversion->append($token);</pre>
+<pre>$inversion->append($token);</pre>
<p>Tack a token onto the end of the Inversion.</p>
@@ -63,7 +65,7 @@ name="append"
name="next"
>next</a></h3>
-<pre> my $retval = $inversion->next();</pre>
+<pre>my $retval = $inversion->next();</pre>
<p>Return the next token in the Inversion until out of tokens.</p>
@@ -71,9 +73,10 @@ name="next"
name="reset"
>reset</a></h3>
-<pre> $inversion->reset();</pre>
+<pre>$inversion->reset();</pre>
-<p>Reset the Inversion’s iterator, so that the next call to next() returns the first Token in the inversion.</p>
+<p>Reset the Inversion’s iterator,
+so that the next call to next() returns the first Token in the inversion.</p>
<h2><a class='u'
name="INHERITANCE"
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/Normalizer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/Normalizer.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/Normalizer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/Normalizer.mdtext Fri Feb 26 13:44:34 2016
@@ -14,20 +14,23 @@ case folding and accent stripping.</p>
name="SYNOPSIS"
>SYNOPSIS</a></h2>
-<pre> my $normalizer = Lucy::Analysis::Normalizer->new;
-
- my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
- analyzers => [ $tokenizer, $normalizer, $stemmer ],
- );</pre>
+<pre>my $normalizer = Lucy::Analysis::Normalizer->new;
+
+my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
+ analyzers => [ $tokenizer, $normalizer, $stemmer ],
+);</pre>
<h2><a class='u'
name="DESCRIPTION"
>DESCRIPTION</a></h2>
<p>Normalizer is an <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
->Analyzer</a> which normalizes tokens to one of the Unicode normalization forms. Optionally, it performs Unicode case folding and converts accented characters to their base character.</p>
+>Analyzer</a> which normalizes tokens to one of the Unicode normalization forms.
+Optionally,
+it performs Unicode case folding and converts accented characters to their base character.</p>
-<p>If you use highlighting, Normalizer should be run after tokenization because it might add or remove characters.</p>
+<p>If you use highlighting,
+Normalizer should be run after tokenization because it might add or remove characters.</p>
<h2><a class='u'
name="CONSTRUCTORS"
@@ -37,20 +40,27 @@ name="CONSTRUCTORS"
name="new"
>new</a></h3>
-<pre> my $normalizer = Lucy::Analysis::Normalizer->new(
- normalization_form => 'NFKC',
- case_fold => 1,
- strip_accents => 0,
- );</pre>
+<pre>my $normalizer = Lucy::Analysis::Normalizer->new(
+ normalization_form => 'NFKC',
+ case_fold => 1,
+ strip_accents => 0,
+);</pre>
<p>Create a new Normalizer.</p>
<ul>
-<li><b>normalization_form</b> - Unicode normalization form, can be one of ‘NFC’, ‘NFKC’, ‘NFD’, ‘NFKD’. Defaults to ‘NFKC’.</li>
+<li><b>normalization_form</b> - Unicode normalization form,
+can be one of ‘NFC’,
+‘NFKC’,
+‘NFD’,
+‘NFKD’.
+Defaults to ‘NFKC’.</li>
-<li><b>case_fold</b> - Perform case folding, default is true.</li>
+<li><b>case_fold</b> - Perform case folding,
+default is true.</li>
-<li><b>strip_accents</b> - Strip accents, default is false.</li>
+<li><b>strip_accents</b> - Strip accents,
+default is false.</li>
</ul>
<h2><a class='u'
@@ -61,10 +71,12 @@ name="METHODS"
name="transform"
>transform</a></h3>
-<pre> my $retval = $normalizer->transform($inversion);</pre>
+<pre>my $retval = $normalizer->transform($inversion);</pre>
<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
->Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+>Inversion</a> as input and returns an Inversion,
+either the same one (presumably transformed in some way),
+or a new one.</p>
<ul>
<li><b>inversion</b> - An inversion.</li>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/PolyAnalyzer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/PolyAnalyzer.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/PolyAnalyzer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/PolyAnalyzer.mdtext Fri Feb 26 13:44:34 2016
@@ -13,46 +13,53 @@ name="NAME"
name="SYNOPSIS"
>SYNOPSIS</a></h2>
-<pre> my $schema = Lucy::Plan::Schema->new;
- my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
- analyzers => \@analyzers,
- );
- my $type = Lucy::Plan::FullTextType->new(
- analyzer => $polyanalyzer,
- );
- $schema->spec_field( name => 'title', type => $type );
- $schema->spec_field( name => 'content', type => $type );</pre>
+<pre>my $schema = Lucy::Plan::Schema->new;
+my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
+ analyzers => \@analyzers,
+);
+my $type = Lucy::Plan::FullTextType->new(
+ analyzer => $polyanalyzer,
+);
+$schema->spec_field( name => 'title', type => $type );
+$schema->spec_field( name => 'content', type => $type );</pre>
<h2><a class='u'
name="DESCRIPTION"
>DESCRIPTION</a></h2>
<p>A PolyAnalyzer is a series of <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
->Analyzers</a>, each of which will be called upon to “analyze” text in turn. You can either provide the Analyzers yourself, or you can specify a supported language, in which case a PolyAnalyzer consisting of a <a href="../../Lucy/Analysis/CaseFolder.html" class="podlinkpod"
->CaseFolder</a>, a <a href="../../Lucy/Analysis/RegexTokenizer.html" class="podlinkpod"
->RegexTokenizer</a>, and a <a href="../../Lucy/Analysis/SnowballStemmer.html" class="podlinkpod"
+>Analyzers</a>,
+each of which will be called upon to “analyze” text in turn.
+You can either provide the Analyzers yourself,
+or you can specify a supported language,
+in which case a PolyAnalyzer consisting of a <a href="../../Lucy/Analysis/CaseFolder.html" class="podlinkpod"
+>CaseFolder</a>,
+a <a href="../../Lucy/Analysis/RegexTokenizer.html" class="podlinkpod"
+>RegexTokenizer</a>,
+and a <a href="../../Lucy/Analysis/SnowballStemmer.html" class="podlinkpod"
>SnowballStemmer</a> will be generated for you.</p>
-<p>The language parameter is DEPRECATED. Use <a href="../../Lucy/Analysis/EasyAnalyzer.html" class="podlinkpod"
+<p>The language parameter is DEPRECATED.
+Use <a href="../../Lucy/Analysis/EasyAnalyzer.html" class="podlinkpod"
>EasyAnalyzer</a> instead.</p>
<p>Supported languages:</p>
-<pre> en => English,
- da => Danish,
- de => German,
- es => Spanish,
- fi => Finnish,
- fr => French,
- hu => Hungarian,
- it => Italian,
- nl => Dutch,
- no => Norwegian,
- pt => Portuguese,
- ro => Romanian,
- ru => Russian,
- sv => Swedish,
- tr => Turkish,</pre>
+<pre>en => English,
+da => Danish,
+de => German,
+es => Spanish,
+fi => Finnish,
+fr => French,
+hu => Hungarian,
+it => Italian,
+nl => Dutch,
+no => Norwegian,
+pt => Portuguese,
+ro => Romanian,
+ru => Russian,
+sv => Swedish,
+tr => Turkish,</pre>
<h2><a class='u'
name="CONSTRUCTORS"
@@ -62,19 +69,32 @@ name="CONSTRUCTORS"
name="new"
>new</a></h3>
-<pre> my $tokenizer = Lucy::Analysis::StandardTokenizer->new;
- my $normalizer = Lucy::Analysis::Normalizer->new;
- my $stemmer = Lucy::Analysis::SnowballStemmer->new( language => 'en' );
- my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
- analyzers => [ $tokenizer, $normalizer, $stemmer, ], );</pre>
+<pre>my $tokenizer = Lucy::Analysis::StandardTokenizer->new;
+my $normalizer = Lucy::Analysis::Normalizer->new;
+my $stemmer = Lucy::Analysis::SnowballStemmer->new( language => 'en' );
+my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
+ analyzers => [ $tokenizer, $normalizer, $stemmer, ], );</pre>
<p>Create a new PolyAnalyzer.</p>
<ul>
-<li><b>language</b> - An ISO code from the list of supported languages. DEPRECATED, use <a href="../../Lucy/Analysis/EasyAnalyzer.html" class="podlinkpod"
+<li><b>language</b> - An ISO code from the list of supported languages.
+DEPRECATED,
+use <a href="../../Lucy/Analysis/EasyAnalyzer.html" class="podlinkpod"
>EasyAnalyzer</a> instead.</li>
-<li><b>analyzers</b> - An array of Analyzers. The order of the analyzers matters. Don’t put a SnowballStemmer before a RegexTokenizer (can’t stem whole documents or paragraphs – just individual words), or a SnowballStopFilter after a SnowballStemmer (stemmed words, e.g. “themselv”, will not appear in a stoplist). In general, the sequence should be: tokenize, normalize, stopalize, stem.</li>
+<li><b>analyzers</b> - An array of Analyzers.
+The order of the analyzers matters.
+Don’t put a SnowballStemmer before a RegexTokenizer (can’t stem whole documents or paragraphs – just individual words),
+or a SnowballStopFilter after a SnowballStemmer (stemmed words,
+e.g.
+“themselv”,
+will not appear in a stoplist).
+In general,
+the sequence should be: tokenize,
+normalize,
+stopalize,
+stem.</li>
</ul>
<h2><a class='u'
@@ -85,7 +105,7 @@ name="METHODS"
name="get_analyzers"
>get_analyzers</a></h3>
-<pre> my $retval = $poly_analyzer->get_analyzers();</pre>
+<pre>my $retval = $poly_analyzer->get_analyzers();</pre>
<p>Getter for “analyzers” member.</p>
@@ -93,10 +113,12 @@ name="get_analyzers"
name="transform"
>transform</a></h3>
-<pre> my $retval = $poly_analyzer->transform($inversion);</pre>
+<pre>my $retval = $poly_analyzer->transform($inversion);</pre>
<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
->Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+>Inversion</a> as input and returns an Inversion,
+either the same one (presumably transformed in some way),
+or a new one.</p>
<ul>
<li><b>inversion</b> - An inversion.</li>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/RegexTokenizer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/RegexTokenizer.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/RegexTokenizer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/RegexTokenizer.mdtext Fri Feb 26 13:44:34 2016
@@ -13,43 +13,49 @@ name="NAME"
name="SYNOPSIS"
>SYNOPSIS</a></h2>
-<pre> my $whitespace_tokenizer
- = Lucy::Analysis::RegexTokenizer->new( pattern => '\S+' );
+<pre>my $whitespace_tokenizer
+ = Lucy::Analysis::RegexTokenizer->new( pattern => '\S+' );
- # or...
- my $word_char_tokenizer
- = Lucy::Analysis::RegexTokenizer->new( pattern => '\w+' );
-
- # or...
- my $apostrophising_tokenizer = Lucy::Analysis::RegexTokenizer->new;
-
- # Then... once you have a tokenizer, put it into a PolyAnalyzer:
- my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
- analyzers => [ $word_char_tokenizer, $normalizer, $stemmer ], );</pre>
+# or...
+my $word_char_tokenizer
+ = Lucy::Analysis::RegexTokenizer->new( pattern => '\w+' );
+
+# or...
+my $apostrophising_tokenizer = Lucy::Analysis::RegexTokenizer->new;
+
+# Then... once you have a tokenizer, put it into a PolyAnalyzer:
+my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
+ analyzers => [ $word_char_tokenizer, $normalizer, $stemmer ], );</pre>
<h2><a class='u'
name="DESCRIPTION"
>DESCRIPTION</a></h2>
-<p>Generically, “tokenizing” is a process of breaking up a string into an array of “tokens”. For instance, the string “three blind mice” might be tokenized into “three”, “blind”, “mice”.</p>
+<p>Generically,
+“tokenizing” is a process of breaking up a string into an array of “tokens”.
+For instance,
+the string “three blind mice” might be tokenized into “three”,
+“blind”,
+“mice”.</p>
-<p>Lucy::Analysis::RegexTokenizer decides where it should break up the text based on a regular expression compiled from a supplied <code>pattern</code> matching one token. If our source string is…</p>
+<p>Lucy::Analysis::RegexTokenizer decides where it should break up the text based on a regular expression compiled from a supplied <code>pattern</code> matching one token.
+If our source string is…</p>
-<pre> "Eats, Shoots and Leaves."</pre>
+<pre>"Eats, Shoots and Leaves."</pre>
<p>… then a “whitespace tokenizer” with a <code>pattern</code> of <code>"\\S+"</code> produces…</p>
-<pre> Eats,
- Shoots
- and
- Leaves.</pre>
+<pre>Eats,
+Shoots
+and
+Leaves.</pre>
<p>… while a “word character tokenizer” with a <code>pattern</code> of <code>"\\w+"</code> produces…</p>
-<pre> Eats
- Shoots
- and
- Leaves</pre>
+<pre>Eats
+Shoots
+and
+Leaves</pre>
<p>… the difference being that the word character tokenizer skips over punctuation as well as whitespace when determining token boundaries.</p>
@@ -61,14 +67,16 @@ name="CONSTRUCTORS"
name="new"
>new</a></h3>
-<pre> my $word_char_tokenizer = Lucy::Analysis::RegexTokenizer->new(
- pattern => '\w+', # required
- );</pre>
+<pre>my $word_char_tokenizer = Lucy::Analysis::RegexTokenizer->new(
+ pattern => '\w+', # required
+);</pre>
<p>Create a new RegexTokenizer.</p>
<ul>
-<li><b>pattern</b> - A string specifying a Perl-syntax regular expression which should match one token. The default value is <code>\w+(?:[\x{2019}']\w+)*</code>, which matches “it’s” as well as “it” and “O’Henry’s” as well as “Henry”.</li>
+<li><b>pattern</b> - A string specifying a Perl-syntax regular expression which should match one token.
+The default value is <code>\w+(?:[\x{2019}']\w+)*</code>,
+which matches “it’s” as well as “it” and “O’Henry’s” as well as “Henry”.</li>
</ul>
<h2><a class='u'
@@ -79,10 +87,12 @@ name="METHODS"
name="transform"
>transform</a></h3>
-<pre> my $retval = $regex_tokenizer->transform($inversion);</pre>
+<pre>my $retval = $regex_tokenizer->transform($inversion);</pre>
<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
->Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+>Inversion</a> as input and returns an Inversion,
+either the same one (presumably transformed in some way),
+or a new one.</p>
<ul>
<li><b>inversion</b> - An inversion.</li>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStemmer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStemmer.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStemmer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStemmer.mdtext Fri Feb 26 13:44:34 2016
@@ -13,20 +13,25 @@ name="NAME"
name="SYNOPSIS"
>SYNOPSIS</a></h2>
-<pre> my $stemmer = Lucy::Analysis::SnowballStemmer->new( language => 'es' );
-
- my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
- analyzers => [ $tokenizer, $normalizer, $stemmer ],
- );</pre>
+<pre>my $stemmer = Lucy::Analysis::SnowballStemmer->new( language => 'es' );
-<p>This class is a wrapper around the Snowball stemming library, so it supports the same languages.</p>
+my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
+ analyzers => [ $tokenizer, $normalizer, $stemmer ],
+);</pre>
+
+<p>This class is a wrapper around the Snowball stemming library,
+so it supports the same languages.</p>
<h2><a class='u'
name="DESCRIPTION"
>DESCRIPTION</a></h2>
<p>SnowballStemmer is an <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
->Analyzer</a> which reduces related words to a root form (using the “Snowball” stemming library). For instance, “horse”, “horses”, and “horsing” all become “hors” – so that a search for ‘horse’ will also match documents containing ‘horses’ and ‘horsing’.</p>
+>Analyzer</a> which reduces related words to a root form (using the “Snowball” stemming library).
+For instance,
+“horse”,
+“horses”,
+and “horsing” all become “hors” – so that a search for ‘horse’ will also match documents containing ‘horses’ and ‘horsing’.</p>
<h2><a class='u'
name="CONSTRUCTORS"
@@ -36,7 +41,7 @@ name="CONSTRUCTORS"
name="new"
>new</a></h3>
-<pre> my $stemmer = Lucy::Analysis::SnowballStemmer->new( language => 'es' );</pre>
+<pre>my $stemmer = Lucy::Analysis::SnowballStemmer->new( language => 'es' );</pre>
<p>Create a new SnowballStemmer.</p>
@@ -52,10 +57,12 @@ name="METHODS"
name="transform"
>transform</a></h3>
-<pre> my $retval = $snowball_stemmer->transform($inversion);</pre>
+<pre>my $retval = $snowball_stemmer->transform($inversion);</pre>
<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
->Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+>Inversion</a> as input and returns an Inversion,
+either the same one (presumably transformed in some way),
+or a new one.</p>
<ul>
<li><b>inversion</b> - An inversion.</li>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStopFilter.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStopFilter.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStopFilter.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStopFilter.mdtext Fri Feb 26 13:44:34 2016
@@ -13,47 +13,53 @@ name="NAME"
name="SYNOPSIS"
>SYNOPSIS</a></h2>
-<pre> my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
- language => 'fr',
- );
- my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
- analyzers => [ $tokenizer, $normalizer, $stopfilter, $stemmer ],
- );</pre>
+<pre>my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
+ language => 'fr',
+);
+my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
+ analyzers => [ $tokenizer, $normalizer, $stopfilter, $stemmer ],
+);</pre>
<h2><a class='u'
name="DESCRIPTION"
>DESCRIPTION</a></h2>
-<p>A “stoplist” is collection of “stopwords”: words which are common enough to be of little value when determining search results. For example, so many documents in English contain “the”, “if”, and “maybe” that it may improve both performance and relevance to block them.</p>
+<p>A “stoplist” is collection of “stopwords”: words which are common enough to be of little value when determining search results.
+For example,
+so many documents in English contain “the”,
+“if”,
+and “maybe” that it may improve both performance and relevance to block them.</p>
<p>Before filtering stopwords:</p>
-<pre> ("i", "am", "the", "walrus")</pre>
+<pre>("i", "am", "the", "walrus")</pre>
<p>After filtering stopwords:</p>
-<pre> ("walrus")</pre>
+<pre>("walrus")</pre>
-<p>SnowballStopFilter provides default stoplists for several languages, courtesy of the <a href="http://snowball.tartarus.org" class="podlinkurl"
->Snowball project</a>, or you may supply your own.</p>
-
-<pre> |-----------------------|
- | ISO CODE | LANGUAGE |
- |-----------------------|
- | da | Danish |
- | de | German |
- | en | English |
- | es | Spanish |
- | fi | Finnish |
- | fr | French |
- | hu | Hungarian |
- | it | Italian |
- | nl | Dutch |
- | no | Norwegian |
- | pt | Portuguese |
- | sv | Swedish |
- | ru | Russian |
- |-----------------------|</pre>
+<p>SnowballStopFilter provides default stoplists for several languages,
+courtesy of the <a href="http://snowball.tartarus.org" class="podlinkurl"
+>Snowball project</a>,
+or you may supply your own.</p>
+
+<pre>|-----------------------|
+| ISO CODE | LANGUAGE |
+|-----------------------|
+| da | Danish |
+| de | German |
+| en | English |
+| es | Spanish |
+| fi | Finnish |
+| fr | French |
+| hu | Hungarian |
+| it | Italian |
+| nl | Dutch |
+| no | Norwegian |
+| pt | Portuguese |
+| sv | Swedish |
+| ru | Russian |
+|-----------------------|</pre>
<h2><a class='u'
name="CONSTRUCTORS"
@@ -63,14 +69,14 @@ name="CONSTRUCTORS"
name="new"
>new</a></h3>
-<pre> my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
- language => 'de',
- );
-
- # or...
- my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
- stoplist => \%stoplist,
- );</pre>
+<pre>my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
+ language => 'de',
+);
+
+# or...
+my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
+ stoplist => \%stoplist,
+);</pre>
<p>Create a new SnowballStopFilter.</p>
@@ -88,10 +94,12 @@ name="METHODS"
name="transform"
>transform</a></h3>
-<pre> my $retval = $snowball_stop_filter->transform($inversion);</pre>
+<pre>my $retval = $snowball_stop_filter->transform($inversion);</pre>
<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
->Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+>Inversion</a> as input and returns an Inversion,
+either the same one (presumably transformed in some way),
+or a new one.</p>
<ul>
<li><b>inversion</b> - An inversion.</li>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/StandardTokenizer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/StandardTokenizer.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/StandardTokenizer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/StandardTokenizer.mdtext Fri Feb 26 13:44:34 2016
@@ -13,19 +13,25 @@ name="NAME"
name="SYNOPSIS"
>SYNOPSIS</a></h2>
-<pre> my $tokenizer = Lucy::Analysis::StandardTokenizer->new;
+<pre>my $tokenizer = Lucy::Analysis::StandardTokenizer->new;
- # Then... once you have a tokenizer, put it into a PolyAnalyzer:
- my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
- analyzers => [ $tokenizer, $normalizer, $stemmer ], );</pre>
+# Then... once you have a tokenizer, put it into a PolyAnalyzer:
+my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
+ analyzers => [ $tokenizer, $normalizer, $stemmer ], );</pre>
<h2><a class='u'
name="DESCRIPTION"
>DESCRIPTION</a></h2>
-<p>Generically, “tokenizing” is a process of breaking up a string into an array of “tokens”. For instance, the string “three blind mice” might be tokenized into “three”, “blind”, “mice”.</p>
+<p>Generically,
+“tokenizing” is a process of breaking up a string into an array of “tokens”.
+For instance,
+the string “three blind mice” might be tokenized into “three”,
+“blind”,
+“mice”.</p>
-<p>Lucy::Analysis::StandardTokenizer breaks up the text at the word boundaries defined in Unicode Standard Annex #29. It then returns those words that contain alphabetic or numeric characters.</p>
+<p>Lucy::Analysis::StandardTokenizer breaks up the text at the word boundaries defined in Unicode Standard Annex #29.
+It then returns those words that contain alphabetic or numeric characters.</p>
<h2><a class='u'
name="CONSTRUCTORS"
@@ -35,9 +41,10 @@ name="CONSTRUCTORS"
name="new"
>new</a></h3>
-<pre> my $tokenizer = Lucy::Analysis::StandardTokenizer->new;</pre>
+<pre>my $tokenizer = Lucy::Analysis::StandardTokenizer->new;</pre>
-<p>Constructor. Takes no arguments.</p>
+<p>Constructor.
+Takes no arguments.</p>
<h2><a class='u'
name="METHODS"
@@ -47,10 +54,12 @@ name="METHODS"
name="transform"
>transform</a></h3>
-<pre> my $retval = $standard_tokenizer->transform($inversion);</pre>
+<pre>my $retval = $standard_tokenizer->transform($inversion);</pre>
<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
->Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+>Inversion</a> as input and returns an Inversion,
+either the same one (presumably transformed in some way),
+or a new one.</p>
<ul>
<li><b>inversion</b> - An inversion.</li>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/Token.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/Token.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/Token.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/Token.mdtext Fri Feb 26 13:44:34 2016
@@ -13,27 +13,59 @@ name="NAME"
name="SYNOPSIS"
>SYNOPSIS</a></h2>
-<pre> my $token = Lucy::Analysis::Token->new(
- text => 'blind',
- start_offset => 8,
- end_offset => 13,
- );
+<pre> my $token = Lucy::Analysis::Token->new(
+ text => 'blind',
+ start_offset => 8,
+ end_offset => 13,
+ );
- $token->set_text('mice');</pre>
+ $token->set_text('mice');</pre>
<h2><a class='u'
name="DESCRIPTION"
>DESCRIPTION</a></h2>
-<p>Token is the fundamental unit used by Apache Lucy’s Analyzer subclasses. Each Token has 5 attributes: <code>text</code>, <code>start_offset</code>, <code>end_offset</code>, <code>boost</code>, and <code>pos_inc</code>.</p>
+<p>Token is the fundamental unit used by Apache Lucy’s Analyzer subclasses.
+Each Token has 5 attributes: <code>text</code>,
+<code>start_offset</code>,
+<code>end_offset</code>,
+<code>boost</code>,
+and <code>pos_inc</code>.</p>
<p>The <code>text</code> attribute is a Unicode string encoded as UTF-8.</p>
-<p><code>start_offset</code> is the start point of the token text, measured in Unicode code points from the top of the stored field; <code>end_offset</code> delimits the corresponding closing boundary. <code>start_offset</code> and <code>end_offset</code> locate the Token within a larger context, even if the Token’s text attribute gets modified – by stemming, for instance. The Token for “beating” in the text “beating a dead horse” begins life with a start_offset of 0 and an end_offset of 7; after stemming, the text is “beat”, but the start_offset is still 0 and the end_offset is still 7. This allows “beating” to be highlighted correctly after a search matches “beat”.</p>
-
-<p><code>boost</code> is a per-token weight. Use this when you want to assign more or less importance to a particular token, as you might for emboldened text within an HTML document, for example. (Note: The field this token belongs to must be spec’d to use a posting of type RichPosting.)</p>
-
-<p><code>pos_inc</code> is the POSition INCrement, measured in Tokens. This attribute, which defaults to 1, is a an advanced tool for manipulating phrase matching. Ordinarily, Tokens are assigned consecutive position numbers: 0, 1, and 2 for <code>"three blind mice"</code>. However, if you set the position increment for “blind” to, say, 1000, then the three tokens will end up assigned to positions 0, 1, and 1001 – and will no longer produce a phrase match for the query <code>"three blind mice"</code>.</p>
+<p><code>start_offset</code> is the start point of the token text,
+measured in Unicode code points from the top of the stored field; <code>end_offset</code> delimits the corresponding closing boundary.
+<code>start_offset</code> and <code>end_offset</code> locate the Token within a larger context,
+even if the Token’s text attribute gets modified – by stemming,
+for instance.
+The Token for “beating” in the text “beating a dead horse” begins life with a start_offset of 0 and an end_offset of 7; after stemming,
+the text is “beat”,
+but the start_offset is still 0 and the end_offset is still 7.
+This allows “beating” to be highlighted correctly after a search matches “beat”.</p>
+
+<p><code>boost</code> is a per-token weight.
+Use this when you want to assign more or less importance to a particular token,
+as you might for emboldened text within an HTML document,
+for example.
+(Note: The field this token belongs to must be spec’d to use a posting of type RichPosting.)</p>
+
+<p><code>pos_inc</code> is the POSition INCrement,
+measured in Tokens.
+This attribute,
+which defaults to 1,
+is a an advanced tool for manipulating phrase matching.
+Ordinarily,
+Tokens are assigned consecutive position numbers: 0,
+1,
+and 2 for <code>"three blind mice"</code>.
+However,
+if you set the position increment for “blind” to,
+say,
+1000,
+then the three tokens will end up assigned to positions 0,
+1,
+and 1001 – and will no longer produce a phrase match for the query <code>"three blind mice"</code>.</p>
<h2><a class='u'
name="CONSTRUCTORS"
@@ -43,13 +75,13 @@ name="CONSTRUCTORS"
name="new"
>new</a></h3>
-<pre> my $token = Lucy::Analysis::Token->new(
- text => $text, # required
- start_offset => $start_offset, # required
- end_offset => $end_offset, # required
- boost => 1.0, # optional
- pos_inc => 1, # optional
- );</pre>
+<pre>my $token = Lucy::Analysis::Token->new(
+ text => $text, # required
+ start_offset => $start_offset, # required
+ end_offset => $end_offset, # required
+ boost => 1.0, # optional
+ pos_inc => 1, # optional
+);</pre>
<ul>
<li><b>text</b> - A string.</li>
@@ -71,7 +103,7 @@ name="METHODS"
name="get_text"
>get_text</a></h3>
-<pre> my $text = $token->get_text;</pre>
+<pre>my $text = $token->get_text;</pre>
<p>Get the token's text.</p>
@@ -79,7 +111,7 @@ name="get_text"
name="set_text"
>set_text</a></h3>
-<pre> $token->set_text($text);</pre>
+<pre>$token->set_text($text);</pre>
<p>Set the token's text.</p>
@@ -87,31 +119,31 @@ name="set_text"
name="get_start_offset"
>get_start_offset</a></h3>
-<pre> my $retval = $token->get_start_offset();</pre>
+<pre>my $retval = $token->get_start_offset();</pre>
<h3><a class='u'
name="get_end_offset"
>get_end_offset</a></h3>
-<pre> my $retval = $token->get_end_offset();</pre>
+<pre>my $retval = $token->get_end_offset();</pre>
<h3><a class='u'
name="get_boost"
>get_boost</a></h3>
-<pre> my $retval = $token->get_boost();</pre>
+<pre>my $retval = $token->get_boost();</pre>
<h3><a class='u'
name="get_pos_inc"
>get_pos_inc</a></h3>
-<pre> my $retval = $token->get_pos_inc();</pre>
+<pre>my $retval = $token->get_pos_inc();</pre>
<h3><a class='u'
name="get_len"
>get_len</a></h3>
-<pre> my $retval = $token->get_len();</pre>
+<pre>my $retval = $token->get_len();</pre>
<h2><a class='u'
name="INHERITANCE"
Modified: lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook.mdtext Fri Feb 26 13:44:34 2016
@@ -45,8 +45,8 @@ name="Materials"
>Tutorial</a> application.
These materials can be found in the <code>sample</code> directory at the root of the Lucy distribution:</p>
-<pre> sample/indexer.pl # indexing app
- sample/search.cgi # search app
- sample/us_constitution # corpus</pre>
+<pre>sample/indexer.pl # indexing app
+sample/search.cgi # search app
+sample/us_constitution # corpus</pre>
</div>
Modified: lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQuery.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQuery.mdtext?rev=1732480&r1=1732479&r2=1732480&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQuery.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQuery.mdtext Fri Feb 26 13:44:34 2016
@@ -15,47 +15,56 @@ name="DESCRIPTION"
<p>Explore Apache Lucy’s support for custom query types by creating a “PrefixQuery” class to handle trailing wildcards.</p>
-<pre> my $prefix_query = PrefixQuery->new(
- field => 'content',
- query_string => 'foo*',
- );
- my $hits = $searcher->hits( query => $prefix_query );
- ...</pre>
+<pre>my $prefix_query = PrefixQuery->new(
+ field => 'content',
+ query_string => 'foo*',
+);
+my $hits = $searcher->hits( query => $prefix_query );
+...</pre>
<h3><a class='u'
name="Query,_Compiler,_and_Matcher"
->Query, Compiler, and Matcher</a></h3>
-
-<p>To add support for a new query type, we need three classes: a Query, a Compiler, and a Matcher.</p>
+>Query,
+Compiler,
+and Matcher</a></h3>
+
+<p>To add support for a new query type,
+we need three classes: a Query,
+a Compiler,
+and a Matcher.</p>
<ul>
<li>PrefixQuery - a subclass of <a href="../../../Lucy/Search/Query.html" class="podlinkpod"
->Query</a>, and the only class that client code will deal with directly.</li>
+>Query</a>,
+and the only class that client code will deal with directly.</li>
<li>PrefixCompiler - a subclass of <a href="../../../Lucy/Search/Compiler.html" class="podlinkpod"
->Compiler</a>, whose primary role is to compile a PrefixQuery to a PrefixMatcher.</li>
+>Compiler</a>,
+whose primary role is to compile a PrefixQuery to a PrefixMatcher.</li>
<li>PrefixMatcher - a subclass of <a href="../../../Lucy/Search/Matcher.html" class="podlinkpod"
->Matcher</a>, which does the heavy lifting: it applies the query to individual documents and assigns a score to each match.</li>
+>Matcher</a>,
+which does the heavy lifting: it applies the query to individual documents and assigns a score to each match.</li>
</ul>
-<p>The PrefixQuery class on its own isn’t enough because a Query object’s role is limited to expressing an abstract specification for the search. A Query is basically nothing but metadata; execution is left to the Query’s companion Compiler and Matcher.</p>
+<p>The PrefixQuery class on its own isn’t enough because a Query object’s role is limited to expressing an abstract specification for the search.
+A Query is basically nothing but metadata; execution is left to the Query’s companion Compiler and Matcher.</p>
<p>Here’s a simplified sketch illustrating how a Searcher’s hits() method ties together the three classes.</p>
-<pre> sub hits {
- my ( $self, $query ) = @_;
- my $compiler = $query->make_compiler(
- searcher => $self,
- boost => $query->get_boost,
- );
- my $matcher = $compiler->make_matcher(
- reader => $self->get_reader,
- need_score => 1,
- );
- my @hits = $matcher->capture_hits;
- return \@hits;
- }</pre>
+<pre>sub hits {
+ my ( $self, $query ) = @_;
+ my $compiler = $query->make_compiler(
+ searcher => $self,
+ boost => $query->get_boost,
+ );
+ my $matcher = $compiler->make_matcher(
+ reader => $self->get_reader,
+ need_score => 1,
+ );
+ my @hits = $matcher->capture_hits;
+ return \@hits;
+}</pre>
<h4><a class='u'
name="PrefixQuery"
@@ -63,65 +72,66 @@ name="PrefixQuery"
<p>Our PrefixQuery class will have two attributes: a query string and a field name.</p>
-<pre> package PrefixQuery;
- use base qw( Lucy::Search::Query );
- use Carp;
- use Scalar::Util qw( blessed );
-
- # Inside-out member vars and hand-rolled accessors.
- my %query_string;
- my %field;
- sub get_query_string { my $self = shift; return $query_string{$$self} }
- sub get_field { my $self = shift; return $field{$$self} }</pre>
+<pre>package PrefixQuery;
+use base qw( Lucy::Search::Query );
+use Carp;
+use Scalar::Util qw( blessed );
+
+# Inside-out member vars and hand-rolled accessors.
+my %query_string;
+my %field;
+sub get_query_string { my $self = shift; return $query_string{$$self} }
+sub get_field { my $self = shift; return $field{$$self} }</pre>
<p>PrefixQuery’s constructor collects and validates the attributes.</p>
-<pre> sub new {
- my ( $class, %args ) = @_;
- my $query_string = delete $args{query_string};
- my $field = delete $args{field};
- my $self = $class->SUPER::new(%args);
- confess("'query_string' param is required")
- unless defined $query_string;
- confess("Invalid query_string: '$query_string'")
- unless $query_string =~ /\*\s*$/;
- confess("'field' param is required")
- unless defined $field;
- $query_string{$$self} = $query_string;
- $field{$$self} = $field;
- return $self;
- }</pre>
-
-<p>Since this is an inside-out class, we’ll need a destructor:</p>
-
-<pre> sub DESTROY {
- my $self = shift;
- delete $query_string{$$self};
- delete $field{$$self};
- $self->SUPER::DESTROY;
- }</pre>
+<pre>sub new {
+ my ( $class, %args ) = @_;
+ my $query_string = delete $args{query_string};
+ my $field = delete $args{field};
+ my $self = $class->SUPER::new(%args);
+ confess("'query_string' param is required")
+ unless defined $query_string;
+ confess("Invalid query_string: '$query_string'")
+ unless $query_string =~ /\*\s*$/;
+ confess("'field' param is required")
+ unless defined $field;
+ $query_string{$$self} = $query_string;
+ $field{$$self} = $field;
+ return $self;
+}</pre>
+
+<p>Since this is an inside-out class,
+we’ll need a destructor:</p>
+
+<pre>sub DESTROY {
+ my $self = shift;
+ delete $query_string{$$self};
+ delete $field{$$self};
+ $self->SUPER::DESTROY;
+}</pre>
<p>The equals() method determines whether two Queries are logically equivalent:</p>
-<pre> sub equals {
- my ( $self, $other ) = @_;
- return 0 unless blessed($other);
- return 0 unless $other->isa("PrefixQuery");
- return 0 unless $field{$$self} eq $field{$$other};
- return 0 unless $query_string{$$self} eq $query_string{$$other};
- return 1;
- }</pre>
+<pre>sub equals {
+ my ( $self, $other ) = @_;
+ return 0 unless blessed($other);
+ return 0 unless $other->isa("PrefixQuery");
+ return 0 unless $field{$$self} eq $field{$$other};
+ return 0 unless $query_string{$$self} eq $query_string{$$other};
+ return 1;
+}</pre>
<p>The last thing we’ll need is a make_compiler() factory method which kicks out a subclass of <a href="../../../Lucy/Search/Compiler.html" class="podlinkpod"
>Compiler</a>.</p>
-<pre> sub make_compiler {
- my ( $self, %args ) = @_;
- my $subordinate = delete $args{subordinate};
- my $compiler = PrefixCompiler->new( %args, parent => $self );
- $compiler->normalize unless $subordinate;
- return $compiler;
- }</pre>
+<pre>sub make_compiler {
+ my ( $self, %args ) = @_;
+ my $subordinate = delete $args{subordinate};
+ my $compiler = PrefixCompiler->new( %args, parent => $self );
+ $compiler->normalize unless $subordinate;
+ return $compiler;
+}</pre>
<h4><a class='u'
name="PrefixCompiler"
@@ -131,67 +141,80 @@ name="PrefixCompiler"
>Searcher</a> – such as <a href="../../../Lucy/Search/IndexSearcher.html" class="podlinkpod"
>IndexSearchers</a>.</p>
-<p>A Searcher is associated with a particular collection of documents. These documents may all reside in one index, as with IndexSearcher, or they may be spread out across multiple indexes on one or more machines, as with LucyX::Remote::ClusterSearcher.</p>
+<p>A Searcher is associated with a particular collection of documents.
+These documents may all reside in one index,
+as with IndexSearcher,
+or they may be spread out across multiple indexes on one or more machines,
+as with LucyX::Remote::ClusterSearcher.</p>
-<p>Searcher objects have access to certain statistical information about the collections they represent; for instance, a Searcher can tell you how many documents are in the collection…</p>
+<p>Searcher objects have access to certain statistical information about the collections they represent; for instance,
+a Searcher can tell you how many documents are in the collection…</p>
-<pre> my $maximum_number_of_docs_in_collection = $searcher->doc_max;</pre>
+<pre>my $maximum_number_of_docs_in_collection = $searcher->doc_max;</pre>
<p>… or how many documents a specific term appears in:</p>
-<pre> my $term_appears_in_this_many_docs = $searcher->doc_freq(
- field => 'content',
- term => 'foo',
- );</pre>
-
-<p>Such information can be used by sophisticated Compiler implementations to assign more or less heft to individual queries or sub-queries. However, we’re not going to bother with weighting for this demo; we’ll just assign a fixed score of 1.0 to each matching document.</p>
-
-<p>We don’t need to write a constructor, as it will suffice to inherit new() from Lucy::Search::Compiler. The only method we need to implement for PrefixCompiler is make_matcher().</p>
-
-<pre> package PrefixCompiler;
- use base qw( Lucy::Search::Compiler );
+<pre>my $term_appears_in_this_many_docs = $searcher->doc_freq(
+ field => 'content',
+ term => 'foo',
+);</pre>
+
+<p>Such information can be used by sophisticated Compiler implementations to assign more or less heft to individual queries or sub-queries.
+However,
+we’re not going to bother with weighting for this demo; we’ll just assign a fixed score of 1.0 to each matching document.</p>
+
+<p>We don’t need to write a constructor,
+as it will suffice to inherit new() from Lucy::Search::Compiler.
+The only method we need to implement for PrefixCompiler is make_matcher().</p>
+
+<pre>package PrefixCompiler;
+use base qw( Lucy::Search::Compiler );
+
+sub make_matcher {
+ my ( $self, %args ) = @_;
+ my $seg_reader = $args{reader};
+
+ # Retrieve low-level components LexiconReader and PostingListReader.
+ my $lex_reader
+ = $seg_reader->obtain("Lucy::Index::LexiconReader");
+ my $plist_reader
+ = $seg_reader->obtain("Lucy::Index::PostingListReader");
- sub make_matcher {
- my ( $self, %args ) = @_;
- my $seg_reader = $args{reader};
+ # Acquire a Lexicon and seek it to our query string.
+ my $substring = $self->get_parent->get_query_string;
+ $substring =~ s/\*.\s*$//;
+ my $field = $self->get_parent->get_field;
+ my $lexicon = $lex_reader->lexicon( field => $field );
+ return unless $lexicon;
+ $lexicon->seek($substring);
- # Retrieve low-level components LexiconReader and PostingListReader.
- my $lex_reader
- = $seg_reader->obtain("Lucy::Index::LexiconReader");
- my $plist_reader
- = $seg_reader->obtain("Lucy::Index::PostingListReader");
-
- # Acquire a Lexicon and seek it to our query string.
- my $substring = $self->get_parent->get_query_string;
- $substring =~ s/\*.\s*$//;
- my $field = $self->get_parent->get_field;
- my $lexicon = $lex_reader->lexicon( field => $field );
- return unless $lexicon;
- $lexicon->seek($substring);
-
- # Accumulate PostingLists for each matching term.
- my @posting_lists;
- while ( defined( my $term = $lexicon->get_term ) ) {
- last unless $term =~ /^\Q$substring/;
- my $posting_list = $plist_reader->posting_list(
- field => $field,
- term => $term,
- );
- if ($posting_list) {
- push @posting_lists, $posting_list;
- }
- last unless $lexicon->next;
+ # Accumulate PostingLists for each matching term.
+ my @posting_lists;
+ while ( defined( my $term = $lexicon->get_term ) ) {
+ last unless $term =~ /^\Q$substring/;
+ my $posting_list = $plist_reader->posting_list(
+ field => $field,
+ term => $term,
+ );
+ if ($posting_list) {
+ push @posting_lists, $posting_list;
}
- return unless @posting_lists;
-
- return PrefixMatcher->new( posting_lists => \@posting_lists );
- }</pre>
+ last unless $lexicon->next;
+ }
+ return unless @posting_lists;
+
+ return PrefixMatcher->new( posting_lists => \@posting_lists );
+}</pre>
<p>PrefixCompiler gets access to a <a href="../../../Lucy/Index/SegReader.html" class="podlinkpod"
->SegReader</a> object when make_matcher() gets called. From the SegReader and its sub-components <a href="../../../Lucy/Index/LexiconReader.html" class="podlinkpod"
+>SegReader</a> object when make_matcher() gets called.
+From the SegReader and its sub-components <a href="../../../Lucy/Index/LexiconReader.html" class="podlinkpod"
>LexiconReader</a> and <a href="../../../Lucy/Index/PostingListReader.html" class="podlinkpod"
->PostingListReader</a>, we acquire a <a href="../../../Lucy/Index/Lexicon.html" class="podlinkpod"
->Lexicon</a>, scan through the Lexicon’s unique terms, and acquire a <a href="../../../Lucy/Index/PostingList.html" class="podlinkpod"
+>PostingListReader</a>,
+we acquire a <a href="../../../Lucy/Index/Lexicon.html" class="podlinkpod"
+>Lexicon</a>,
+scan through the Lexicon’s unique terms,
+and acquire a <a href="../../../Lucy/Index/PostingList.html" class="podlinkpod"
>PostingList</a> for each term that matches our prefix.</p>
<p>Each of these PostingList objects represents a set of documents which match the query.</p>
@@ -202,87 +225,97 @@ name="PrefixMatcher"
<p>The Matcher subclass is the most involved.</p>
-<pre> package PrefixMatcher;
- use base qw( Lucy::Search::Matcher );
-
- # Inside-out member vars.
- my %doc_ids;
- my %tick;
+<pre>package PrefixMatcher;
+use base qw( Lucy::Search::Matcher );
+
+# Inside-out member vars.
+my %doc_ids;
+my %tick;
+
+sub new {
+ my ( $class, %args ) = @_;
+ my $posting_lists = delete $args{posting_lists};
+ my $self = $class->SUPER::new(%args);
- sub new {
- my ( $class, %args ) = @_;
- my $posting_lists = delete $args{posting_lists};
- my $self = $class->SUPER::new(%args);
-
- # Cheesy but simple way of interleaving PostingList doc sets.
- my %all_doc_ids;
- for my $posting_list (@$posting_lists) {
- while ( my $doc_id = $posting_list->next ) {
- $all_doc_ids{$doc_id} = undef;
- }
+ # Cheesy but simple way of interleaving PostingList doc sets.
+ my %all_doc_ids;
+ for my $posting_list (@$posting_lists) {
+ while ( my $doc_id = $posting_list->next ) {
+ $all_doc_ids{$doc_id} = undef;
}
- my @doc_ids = sort { $a <=> $b } keys %all_doc_ids;
- $doc_ids{$$self} = \@doc_ids;
-
- # Track our position within the array of doc ids.
- $tick{$$self} = -1;
-
- return $self;
}
+ my @doc_ids = sort { $a <=> $b } keys %all_doc_ids;
+ $doc_ids{$$self} = \@doc_ids;
+
+ # Track our position within the array of doc ids.
+ $tick{$$self} = -1;
- sub DESTROY {
- my $self = shift;
- delete $doc_ids{$$self};
- delete $tick{$$self};
- $self->SUPER::DESTROY;
- }</pre>
+ return $self;
+}
-<p>The doc ids must be in order, or some will be ignored; hence the <code>sort</code> above.</p>
+sub DESTROY {
+ my $self = shift;
+ delete $doc_ids{$$self};
+ delete $tick{$$self};
+ $self->SUPER::DESTROY;
+}</pre>
-<p>In addition to the constructor and destructor, there are three methods that must be overridden.</p>
+<p>The doc ids must be in order,
+or some will be ignored; hence the <code>sort</code> above.</p>
+
+<p>In addition to the constructor and destructor,
+there are three methods that must be overridden.</p>
<p>next() advances the Matcher to the next valid matching doc.</p>
-<pre> sub next {
- my $self = shift;
- my $doc_ids = $doc_ids{$$self};
- my $tick = ++$tick{$$self};
- return 0 if $tick >= scalar @$doc_ids;
- return $doc_ids->[$tick];
- }</pre>
-
-<p>get_doc_id() returns the current document id, or 0 if the Matcher is exhausted. (<a href="../../../Lucy/Docs/DocIDs.html" class="podlinkpod"
->Document numbers</a> start at 1, so 0 is a sentinel.)</p>
-
-<pre> sub get_doc_id {
- my $self = shift;
- my $tick = $tick{$$self};
- my $doc_ids = $doc_ids{$$self};
- return $tick < scalar @$doc_ids ? $doc_ids->[$tick] : 0;
- }</pre>
+<pre>sub next {
+ my $self = shift;
+ my $doc_ids = $doc_ids{$$self};
+ my $tick = ++$tick{$$self};
+ return 0 if $tick >= scalar @$doc_ids;
+ return $doc_ids->[$tick];
+}</pre>
+
+<p>get_doc_id() returns the current document id,
+or 0 if the Matcher is exhausted.
+(<a href="../../../Lucy/Docs/DocIDs.html" class="podlinkpod"
+>Document numbers</a> start at 1,
+so 0 is a sentinel.)</p>
+
+<pre>sub get_doc_id {
+ my $self = shift;
+ my $tick = $tick{$$self};
+ my $doc_ids = $doc_ids{$$self};
+ return $tick < scalar @$doc_ids ? $doc_ids->[$tick] : 0;
+}</pre>
-<p>score() conveys the relevance score of the current match. We’ll just return a fixed score of 1.0:</p>
+<p>score() conveys the relevance score of the current match.
+We’ll just return a fixed score of 1.0:</p>
-<pre> sub score { 1.0 }</pre>
+<pre>sub score { 1.0 }</pre>
<h3><a class='u'
name="Usage"
>Usage</a></h3>
-<p>To get a basic feel for PrefixQuery, insert the FlatQueryParser module described in <a href="../../../Lucy/Docs/Cookbook/CustomQueryParser.html" class="podlinkpod"
+<p>To get a basic feel for PrefixQuery,
+insert the FlatQueryParser module described in <a href="../../../Lucy/Docs/Cookbook/CustomQueryParser.html" class="podlinkpod"
>CustomQueryParser</a> (which supports PrefixQuery) into the search.cgi sample app.</p>
-<pre> my $parser = FlatQueryParser->new( schema => $searcher->get_schema );
- my $query = $parser->parse($q);</pre>
-
-<p>If you’re planning on using PrefixQuery in earnest, though, you may want to change up analyzers to avoid stemming, because stemming – another approach to prefix conflation – is not perfectly compatible with prefix searches.</p>
+<pre>my $parser = FlatQueryParser->new( schema => $searcher->get_schema );
+my $query = $parser->parse($q);</pre>
-<pre> # Polyanalyzer with no SnowballStemmer.
- my $analyzer = Lucy::Analysis::PolyAnalyzer->new(
- analyzers => [
- Lucy::Analysis::StandardTokenizer->new,
- Lucy::Analysis::Normalizer->new,
- ],
- );</pre>
+<p>If you’re planning on using PrefixQuery in earnest,
+though,
+you may want to change up analyzers to avoid stemming,
+because stemming – another approach to prefix conflation – is not perfectly compatible with prefix searches.</p>
+
+<pre># Polyanalyzer with no SnowballStemmer.
+my $analyzer = Lucy::Analysis::PolyAnalyzer->new(
+ analyzers => [
+ Lucy::Analysis::StandardTokenizer->new,
+ Lucy::Analysis::Normalizer->new,
+ ],
+);</pre>
</div>