You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucy.apache.org by nw...@apache.org on 2016/02/26 13:52:27 UTC
svn commit: r1732471 [1/6] - in /lucy/site/trunk/content/docs/test: ./ Lucy/
Lucy/Analysis/ Lucy/Docs/ Lucy/Docs/Cookbook/ Lucy/Docs/Tutorial/
Lucy/Document/ Lucy/Highlight/ Lucy/Index/ Lucy/Object/ Lucy/Plan/
Lucy/Search/ Lucy/Search/Collector/ Lucy/S...
Author: nwellnhof
Date: Fri Feb 26 12:52:25 2016
New Revision: 1732471
URL: http://svn.apache.org/viewvc?rev=1732471&view=rev
Log:
Update generated mdtext for Perl documentation
Added:
lucy/site/trunk/content/docs/test/Lucy/Analysis/Inversion.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/Token.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/AnalysisTutorial.mdtext
- copied, changed from r1730822, lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/Analysis.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/BeyondSimpleTutorial.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/FieldTypeTutorial.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/HighlighterTutorial.mdtext
- copied, changed from r1730822, lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/Highlighter.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/QueryObjectsTutorial.mdtext
- copied, changed from r1730822, lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/QueryObjects.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/SimpleTutorial.mdtext
- copied, changed from r1730822, lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/Simple.mdtext
lucy/site/trunk/content/docs/test/index.mdtext
Removed:
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/Analysis.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/BeyondSimple.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/FieldType.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/Highlighter.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/QueryObjects.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial/Simple.mdtext
Modified:
lucy/site/trunk/content/docs/test/Lucy.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/Analyzer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/CaseFolder.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/EasyAnalyzer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/Normalizer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/PolyAnalyzer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/RegexTokenizer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStemmer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStopFilter.mdtext
lucy/site/trunk/content/docs/test/Lucy/Analysis/StandardTokenizer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQueryParser.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/FastUpdates.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/DevGuide.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/DocIDs.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/FileFormat.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/FileLocking.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/IRTheory.mdtext
lucy/site/trunk/content/docs/test/Lucy/Docs/Tutorial.mdtext
lucy/site/trunk/content/docs/test/Lucy/Document/Doc.mdtext
lucy/site/trunk/content/docs/test/Lucy/Document/HitDoc.mdtext
lucy/site/trunk/content/docs/test/Lucy/Highlight/Highlighter.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/BackgroundMerger.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/DataReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/DataWriter.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/DeletionsWriter.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/DocReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/IndexManager.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/IndexReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/Indexer.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/Lexicon.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/LexiconReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/PolyReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/PostingList.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/PostingListReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/SegReader.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/SegWriter.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/Segment.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/Similarity.mdtext
lucy/site/trunk/content/docs/test/Lucy/Index/Snapshot.mdtext
lucy/site/trunk/content/docs/test/Lucy/Object/BitVector.mdtext
lucy/site/trunk/content/docs/test/Lucy/Object/Obj.mdtext
lucy/site/trunk/content/docs/test/Lucy/Plan/Architecture.mdtext
lucy/site/trunk/content/docs/test/Lucy/Plan/BlobType.mdtext
lucy/site/trunk/content/docs/test/Lucy/Plan/FieldType.mdtext
lucy/site/trunk/content/docs/test/Lucy/Plan/FullTextType.mdtext
lucy/site/trunk/content/docs/test/Lucy/Plan/Schema.mdtext
lucy/site/trunk/content/docs/test/Lucy/Plan/StringType.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/ANDQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Collector.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Collector/BitCollector.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Compiler.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Hits.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/IndexSearcher.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/LeafQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/MatchAllQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Matcher.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/NOTQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/NoMatchQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/ORQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/PhraseQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/PolyQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/PolySearcher.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Query.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/QueryParser.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/RangeQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/RequiredOptionalQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Searcher.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/SortRule.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/SortSpec.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/Span.mdtext
lucy/site/trunk/content/docs/test/Lucy/Search/TermQuery.mdtext
lucy/site/trunk/content/docs/test/Lucy/Simple.mdtext
lucy/site/trunk/content/docs/test/Lucy/Store/FSFolder.mdtext
lucy/site/trunk/content/docs/test/Lucy/Store/Folder.mdtext
lucy/site/trunk/content/docs/test/Lucy/Store/Lock.mdtext
lucy/site/trunk/content/docs/test/Lucy/Store/LockErr.mdtext
lucy/site/trunk/content/docs/test/Lucy/Store/LockFactory.mdtext
lucy/site/trunk/content/docs/test/Lucy/Store/RAMFolder.mdtext
lucy/site/trunk/content/docs/test/LucyX/Index/ByteBufDocReader.mdtext
lucy/site/trunk/content/docs/test/LucyX/Index/ByteBufDocWriter.mdtext
lucy/site/trunk/content/docs/test/LucyX/Index/LongFieldSim.mdtext
lucy/site/trunk/content/docs/test/LucyX/Index/ZlibDocReader.mdtext
lucy/site/trunk/content/docs/test/LucyX/Index/ZlibDocWriter.mdtext
lucy/site/trunk/content/docs/test/LucyX/Remote/ClusterSearcher.mdtext
lucy/site/trunk/content/docs/test/LucyX/Remote/SearchClient.mdtext
lucy/site/trunk/content/docs/test/LucyX/Remote/SearchServer.mdtext
lucy/site/trunk/content/docs/test/LucyX/Search/Filter.mdtext
lucy/site/trunk/content/docs/test/LucyX/Search/MockMatcher.mdtext
lucy/site/trunk/content/docs/test/LucyX/Search/ProximityQuery.mdtext
Modified: lucy/site/trunk/content/docs/test/Lucy.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy.mdtext Fri Feb 26 12:52:25 2016
@@ -3,21 +3,21 @@ Title: Lucy - Apache Lucy Documentation
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
<p>Lucy - Apache Lucy search engine library.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="VERSION"
->VERSION</a></h1>
+>VERSION</a></h2>
-<p>0.3.0</p>
+<p>0.4.0</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="SYNOPSIS"
->SYNOPSIS</a></h1>
+>SYNOPSIS</a></h2>
<p>First,
plan out your index structure,
@@ -70,15 +70,15 @@ and add documents:</p>
print "$hit->{title}\n";
}</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="DESCRIPTION"
->DESCRIPTION</a></h1>
+>DESCRIPTION</a></h2>
<p>The Apache Lucy search engine library delivers high-performance, modular full-text search.</p>
-<h2><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Features"
->Features</a></h2>
+>Features</a></h3>
<ul>
<li>Extremely fast. A single machine can handle millions of documents.</li>
@@ -106,9 +106,9 @@ name="Features"
<li>Stoplists.</li>
</ul>
-<h2><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Getting_Started"
->Getting Started</a></h2>
+>Getting Started</a></h3>
<p><a href="./Lucy/Simple.html" class="podlinkpod"
>Lucy::Simple</a> provides a stripped down API which may suffice for many tasks.</p>
@@ -135,9 +135,9 @@ name="Getting_Started"
>Lucy::Analysis::EasyAnalyzer</a> - A one-size-fits-all parser/tokenizer.</li>
</ul>
-<h2><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Delving_Deeper"
->Delving Deeper</a></h2>
+>Delving Deeper</a></h3>
<p><a href="./Lucy/Docs/Cookbook.html" class="podlinkpod"
>Lucy::Docs::Cookbook</a> augments the tutorial with more advanced recipes.</p>
@@ -159,39 +159,39 @@ name="Delving_Deeper"
>LucyX::Remote::SearchClient</a>, and <a href="./LucyX/Remote/ClusterSearcher.html" class="podlinkpod"
>LucyX::Remote::ClusterSearcher</a>.</p>
-<h2><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Backwards_Compatibility_Policy"
->Backwards Compatibility Policy</a></h2>
+>Backwards Compatibility Policy</a></h3>
<p>Lucy will spin off stable forks into new namespaces periodically. The first will be named "Lucy1". Users who require strong backwards compatibility should use a stable fork.</p>
<p>The main namespace, "Lucy", is an API-unstable development branch (as hinted at by its 0.x.x version number). Superficial interface changes happen frequently. Hard file format compatibility breaks which require reindexing are rare, as we generally try to provide continuity across multiple releases, but we reserve the right to make such changes.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="CLASS_METHODS"
->CLASS METHODS</a></h1>
+>CLASS METHODS</a></h2>
<p>The Lucy module itself does not have a large interface, providing only a single public class method.</p>
-<h2><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="error"
->error</a></h2>
+>error</a></h3>
<pre> my $instream = $folder->open_in( file => 'foo' ) or die Clownfish->error;</pre>
<p>Access a shared variable which is set by some routines on failure. It will always be either a <a href="./Clownfish/Err.html" class="podlinkpod"
>Clownfish::Err</a> object or undef.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="SUPPORT"
->SUPPORT</a></h1>
+>SUPPORT</a></h2>
<p>The Apache Lucy homepage, where you'll find links to our mailing lists and so on, is <a href="http://lucy.apache.org" class="podlinkurl"
>http://lucy.apache.org</a>. Please direct support questions to the Lucy users mailing list.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="BUGS"
->BUGS</a></h1>
+>BUGS</a></h2>
<p>Not thread-safe.</p>
@@ -200,9 +200,9 @@ name="BUGS"
<p>If you find a bug, please inquire on the Lucy users mailing list about it, then report it on the Lucy issue tracker once it has been confirmed: <a href="https://issues.apache.org/jira/browse/LUCY" class="podlinkurl"
>https://issues.apache.org/jira/browse/LUCY</a>.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="COPYRIGHT"
->COPYRIGHT</a></h1>
+>COPYRIGHT</a></h2>
<p>Apache Lucy is distributed under the Apache License, Version 2.0, as described in the file <code>LICENSE</code> included with the distribution.</p>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/Analyzer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/Analyzer.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/Analyzer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/Analyzer.mdtext Fri Feb 26 12:52:25 2016
@@ -3,29 +3,127 @@ Title: Lucy::Analysis::Analyzer - Apache
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
<p>Lucy::Analysis::Analyzer - Tokenize/modify/filter text.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="SYNOPSIS"
->SYNOPSIS</a></h1>
+>SYNOPSIS</a></h2>
<pre> # Abstract base class.</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="DESCRIPTION"
->DESCRIPTION</a></h1>
+>DESCRIPTION</a></h2>
<p>An Analyzer is a filter which processes text, transforming it from one form into another. For instance, an analyzer might break up a long text into smaller pieces (<a href="../../Lucy/Analysis/RegexTokenizer.html" class="podlinkpod"
>RegexTokenizer</a>), or it might perform case folding to facilitate case-insensitive search (<a href="../../Lucy/Analysis/Normalizer.html" class="podlinkpod"
>Normalizer</a>).</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
+name="CONSTRUCTORS"
+>CONSTRUCTORS</a></h2>
+
+<h3><a class='u'
+name="new"
+>new</a></h3>
+
+<pre> package MyAnalyzer;
+ use base qw( Lucy::Analysis::Analyzer );
+ our %foo;
+ sub new {
+ my $self = shift->SUPER::new;
+ my %args = @_;
+ $foo{$$self} = $args{foo};
+ return $self;
+ }</pre>
+
+<p>Abstract constructor. Takes no arguments.</p>
+
+<h2><a class='u'
+name="ABSTRACT_METHODS"
+>ABSTRACT METHODS</a></h2>
+
+<h3><a class='u'
+name="transform"
+>transform</a></h3>
+
+<pre> my $retval = $analyzer->transform($inversion);</pre>
+
+<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
+>Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+
+<ul>
+<li><b>inversion</b> - An inversion.</li>
+</ul>
+
+<h2><a class='u'
+name="METHODS"
+>METHODS</a></h2>
+
+<h3><a class='u'
+name="transform_text"
+>transform_text</a></h3>
+
+<pre> my $retval = $analyzer->transform_text($text);</pre>
+
+<p>Kick off an analysis chain, creating an Inversion from string input. The default implementation simply creates an initial Inversion with a single Token, then calls <a href="#transform" class="podlinkpod"
+>transform()</a>, but occasionally subclasses will provide an optimized implementation which minimizes string copies.</p>
+
+<ul>
+<li><b>text</b> - A string.</li>
+</ul>
+
+<h3><a class='u'
+name="split"
+>split</a></h3>
+
+<pre> my $retval = $analyzer->split($text);</pre>
+
+<p>Analyze text and return an array of token texts.</p>
+
+<ul>
+<li><b>text</b> - A string.</li>
+</ul>
+
+<h3><a class='u'
+name="dump"
+>dump</a></h3>
+
+<pre> my $retval = $analyzer->dump();</pre>
+
+<p>Dump the analyzer as hash.</p>
+
+<p>Subclasses should call <a href="#dump" class="podlinkpod"
+>dump()</a> on the superclass. The returned object is a hash which should be populated with parameters of the analyzer.</p>
+
+<p>Returns: A hash containing a description of the analyzer.</p>
+
+<h3><a class='u'
+name="load"
+>load</a></h3>
+
+<pre> my $retval = $analyzer->load($dump);</pre>
+
+<p>Reconstruct an analyzer from a dump.</p>
+
+<p>Subclasses should first call <a href="#load" class="podlinkpod"
+>load()</a> on the superclass. The returned object is an analyzer which should be reconstructed by setting the dumped parameters from the hash contained in <code>dump</code>.</p>
+
+<p>Note that the invocant analyzer is unused.</p>
+
+<ul>
+<li><b>dump</b> - A hash.</li>
+</ul>
+
+<p>Returns: An analyzer.</p>
+
+<h2><a class='u'
name="INHERITANCE"
->INHERITANCE</a></h1>
+>INHERITANCE</a></h2>
<p>Lucy::Analysis::Analyzer isa Clownfish::Obj.</p>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/CaseFolder.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/CaseFolder.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/CaseFolder.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/CaseFolder.mdtext Fri Feb 26 12:52:25 2016
@@ -3,16 +3,16 @@ Title: Lucy::Analysis::CaseFolder - Apac
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
<p>Lucy::Analysis::CaseFolder - Normalize case,
facilitating case-insensitive search.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="SYNOPSIS"
->SYNOPSIS</a></h1>
+>SYNOPSIS</a></h2>
<pre> my $case_folder = Lucy::Analysis::CaseFolder->new;
@@ -20,30 +20,47 @@ name="SYNOPSIS"
analyzers => [ $tokenizer, $case_folder, $stemmer ],
);</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="DESCRIPTION"
->DESCRIPTION</a></h1>
+>DESCRIPTION</a></h2>
<p>CaseFolder is DEPRECATED. Use <a href="../../Lucy/Analysis/Normalizer.html" class="podlinkpod"
>Normalizer</a> instead.</p>
<p>CaseFolder normalizes text according to Unicode case-folding rules, so that searches will be case-insensitive.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="CONSTRUCTORS"
->CONSTRUCTORS</a></h1>
+>CONSTRUCTORS</a></h2>
-<h2><a class='u' href='#___top' title='click to go to top of document'
-name="new()"
->new()</a></h2>
+<h3><a class='u'
+name="new"
+>new</a></h3>
<pre> my $case_folder = Lucy::Analysis::CaseFolder->new;</pre>
<p>Constructor. Takes no arguments.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
+name="METHODS"
+>METHODS</a></h2>
+
+<h3><a class='u'
+name="transform"
+>transform</a></h3>
+
+<pre> my $retval = $case_folder->transform($inversion);</pre>
+
+<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
+>Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+
+<ul>
+<li><b>inversion</b> - An inversion.</li>
+</ul>
+
+<h2><a class='u'
name="INHERITANCE"
->INHERITANCE</a></h1>
+>INHERITANCE</a></h2>
<p>Lucy::Analysis::CaseFolder isa <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
>Lucy::Analysis::Analyzer</a> isa Clownfish::Obj.</p>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/EasyAnalyzer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/EasyAnalyzer.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/EasyAnalyzer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/EasyAnalyzer.mdtext Fri Feb 26 12:52:25 2016
@@ -3,15 +3,15 @@ Title: Lucy::Analysis::EasyAnalyzer - Ap
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
<p>Lucy::Analysis::EasyAnalyzer - A simple analyzer chain.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="SYNOPSIS"
->SYNOPSIS</a></h1>
+>SYNOPSIS</a></h2>
<pre> my $schema = Lucy::Plan::Schema->new;
my $analyzer = Lucy::Analysis::EasyAnalyzer->new(
@@ -23,9 +23,9 @@ name="SYNOPSIS"
$schema->spec_field( name => 'title', type => $type );
$schema->spec_field( name => 'content', type => $type );</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="DESCRIPTION"
->DESCRIPTION</a></h1>
+>DESCRIPTION</a></h2>
<p>EasyAnalyzer is an analyzer chain consisting of a <a href="../../Lucy/Analysis/StandardTokenizer.html" class="podlinkpod"
>StandardTokenizer</a>, a <a href="../../Lucy/Analysis/Normalizer.html" class="podlinkpod"
@@ -50,25 +50,44 @@ name="DESCRIPTION"
sv => Swedish,
tr => Turkish,</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="CONSTRUCTORS"
->CONSTRUCTORS</a></h1>
+>CONSTRUCTORS</a></h2>
-<h2><a class='u' href='#___top' title='click to go to top of document'
-name="new(_[labeled_params]_)"
->new( <i>[labeled params]</i> )</a></h2>
+<h3><a class='u'
+name="new"
+>new</a></h3>
<pre> my $analyzer = Lucy::Analysis::EasyAnalyzer->new(
language => 'es',
);</pre>
+<p>Create a new EasyAnalyzer.</p>
+
<ul>
<li><b>language</b> - An ISO code from the list of supported languages.</li>
</ul>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
+name="METHODS"
+>METHODS</a></h2>
+
+<h3><a class='u'
+name="transform"
+>transform</a></h3>
+
+<pre> my $retval = $easy_analyzer->transform($inversion);</pre>
+
+<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
+>Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+
+<ul>
+<li><b>inversion</b> - An inversion.</li>
+</ul>
+
+<h2><a class='u'
name="INHERITANCE"
->INHERITANCE</a></h1>
+>INHERITANCE</a></h2>
<p>Lucy::Analysis::EasyAnalyzer isa <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
>Lucy::Analysis::Analyzer</a> isa Clownfish::Obj.</p>
Added: lucy/site/trunk/content/docs/test/Lucy/Analysis/Inversion.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/Inversion.mdtext?rev=1732471&view=auto
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/Inversion.mdtext (added)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/Inversion.mdtext Fri Feb 26 12:52:25 2016
@@ -0,0 +1,84 @@
+Title: Lucy::Analysis::Inversion - Apache Lucy Documentation
+
+<div>
+<a name='___top' class='dummyTopAnchor' ></a>
+
+<h2><a class='u'
+name="NAME"
+>NAME</a></h2>
+
+<p>Lucy::Analysis::Inversion - A collection of Tokens.</p>
+
+<h2><a class='u'
+name="SYNOPSIS"
+>SYNOPSIS</a></h2>
+
+<pre> my $result = Lucy::Analysis::Inversion->new;
+
+ while (my $token = $inversion->next) {
+ $result->append($token);
+ }</pre>
+
+<h2><a class='u'
+name="DESCRIPTION"
+>DESCRIPTION</a></h2>
+
+<p>An Inversion is a collection of Token objects which you can add to, then iterate over.</p>
+
+<h2><a class='u'
+name="CONSTRUCTORS"
+>CONSTRUCTORS</a></h2>
+
+<h3><a class='u'
+name="new"
+>new</a></h3>
+
+<pre> my $inversion = Lucy::Analysis::Inversion->new(
+ $seed, # optional
+ );</pre>
+
+<p>Create a new Inversion.</p>
+
+<ul>
+<li><b>seed</b> - An initial Token to start things off, which may be undef.</li>
+</ul>
+
+<h2><a class='u'
+name="METHODS"
+>METHODS</a></h2>
+
+<h3><a class='u'
+name="append"
+>append</a></h3>
+
+<pre> $inversion->append($token);</pre>
+
+<p>Tack a token onto the end of the Inversion.</p>
+
+<ul>
+<li><b>token</b> - A Token.</li>
+</ul>
+
+<h3><a class='u'
+name="next"
+>next</a></h3>
+
+<pre> my $retval = $inversion->next();</pre>
+
+<p>Return the next token in the Inversion until out of tokens.</p>
+
+<h3><a class='u'
+name="reset"
+>reset</a></h3>
+
+<pre> $inversion->reset();</pre>
+
+<p>Reset the Inversion’s iterator, so that the next call to next() returns the first Token in the inversion.</p>
+
+<h2><a class='u'
+name="INHERITANCE"
+>INHERITANCE</a></h2>
+
+<p>Lucy::Analysis::Inversion isa Clownfish::Obj.</p>
+
+</div>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/Normalizer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/Normalizer.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/Normalizer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/Normalizer.mdtext Fri Feb 26 12:52:25 2016
@@ -3,16 +3,16 @@ Title: Lucy::Analysis::Normalizer - Apac
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
<p>Lucy::Analysis::Normalizer - Unicode normalization,
case folding and accent stripping.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="SYNOPSIS"
->SYNOPSIS</a></h1>
+>SYNOPSIS</a></h2>
<pre> my $normalizer = Lucy::Analysis::Normalizer->new;
@@ -20,22 +20,22 @@ name="SYNOPSIS"
analyzers => [ $tokenizer, $normalizer, $stemmer ],
);</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="DESCRIPTION"
->DESCRIPTION</a></h1>
+>DESCRIPTION</a></h2>
<p>Normalizer is an <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
>Analyzer</a> which normalizes tokens to one of the Unicode normalization forms. Optionally, it performs Unicode case folding and converts accented characters to their base character.</p>
<p>If you use highlighting, Normalizer should be run after tokenization because it might add or remove characters.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="CONSTRUCTORS"
->CONSTRUCTORS</a></h1>
+>CONSTRUCTORS</a></h2>
-<h2><a class='u' href='#___top' title='click to go to top of document'
-name="new(_[labeled_params]_)"
->new( <i>[labeled params]</i> )</a></h2>
+<h3><a class='u'
+name="new"
+>new</a></h3>
<pre> my $normalizer = Lucy::Analysis::Normalizer->new(
normalization_form => 'NFKC',
@@ -43,17 +43,36 @@ name="new(_[labeled_params]_)"
strip_accents => 0,
);</pre>
+<p>Create a new Normalizer.</p>
+
<ul>
-<li><b>normalization_form</b> - Unicode normalization form, can be one of 'NFC', 'NFKC', 'NFD', 'NFKD'. Defaults to 'NFKC'.</li>
+<li><b>normalization_form</b> - Unicode normalization form, can be one of ‘NFC’, ‘NFKC’, ‘NFD’, ‘NFKD’. Defaults to ‘NFKC’.</li>
<li><b>case_fold</b> - Perform case folding, default is true.</li>
<li><b>strip_accents</b> - Strip accents, default is false.</li>
</ul>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
+name="METHODS"
+>METHODS</a></h2>
+
+<h3><a class='u'
+name="transform"
+>transform</a></h3>
+
+<pre> my $retval = $normalizer->transform($inversion);</pre>
+
+<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
+>Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+
+<ul>
+<li><b>inversion</b> - An inversion.</li>
+</ul>
+
+<h2><a class='u'
name="INHERITANCE"
->INHERITANCE</a></h1>
+>INHERITANCE</a></h2>
<p>Lucy::Analysis::Normalizer isa <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
>Lucy::Analysis::Analyzer</a> isa Clownfish::Obj.</p>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/PolyAnalyzer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/PolyAnalyzer.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/PolyAnalyzer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/PolyAnalyzer.mdtext Fri Feb 26 12:52:25 2016
@@ -3,15 +3,15 @@ Title: Lucy::Analysis::PolyAnalyzer - Ap
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
<p>Lucy::Analysis::PolyAnalyzer - Multiple Analyzers in series.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="SYNOPSIS"
->SYNOPSIS</a></h1>
+>SYNOPSIS</a></h2>
<pre> my $schema = Lucy::Plan::Schema->new;
my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
@@ -23,12 +23,12 @@ name="SYNOPSIS"
$schema->spec_field( name => 'title', type => $type );
$schema->spec_field( name => 'content', type => $type );</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="DESCRIPTION"
->DESCRIPTION</a></h1>
+>DESCRIPTION</a></h2>
<p>A PolyAnalyzer is a series of <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
->Analyzers</a>, each of which will be called upon to "analyze" text in turn. You can either provide the Analyzers yourself, or you can specify a supported language, in which case a PolyAnalyzer consisting of a <a href="../../Lucy/Analysis/CaseFolder.html" class="podlinkpod"
+>Analyzers</a>, each of which will be called upon to “analyze” text in turn. You can either provide the Analyzers yourself, or you can specify a supported language, in which case a PolyAnalyzer consisting of a <a href="../../Lucy/Analysis/CaseFolder.html" class="podlinkpod"
>CaseFolder</a>, a <a href="../../Lucy/Analysis/RegexTokenizer.html" class="podlinkpod"
>RegexTokenizer</a>, and a <a href="../../Lucy/Analysis/SnowballStemmer.html" class="podlinkpod"
>SnowballStemmer</a> will be generated for you.</p>
@@ -54,13 +54,13 @@ name="DESCRIPTION"
sv => Swedish,
tr => Turkish,</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="CONSTRUCTORS"
->CONSTRUCTORS</a></h1>
+>CONSTRUCTORS</a></h2>
-<h2><a class='u' href='#___top' title='click to go to top of document'
-name="new(_[labeled_params]_)"
->new( <i>[labeled params]</i> )</a></h2>
+<h3><a class='u'
+name="new"
+>new</a></h3>
<pre> my $tokenizer = Lucy::Analysis::StandardTokenizer->new;
my $normalizer = Lucy::Analysis::Normalizer->new;
@@ -68,26 +68,43 @@ name="new(_[labeled_params]_)"
my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
analyzers => [ $tokenizer, $normalizer, $stemmer, ], );</pre>
+<p>Create a new PolyAnalyzer.</p>
+
<ul>
<li><b>language</b> - An ISO code from the list of supported languages. DEPRECATED, use <a href="../../Lucy/Analysis/EasyAnalyzer.html" class="podlinkpod"
>EasyAnalyzer</a> instead.</li>
-<li><b>analyzers</b> - An array of Analyzers. The order of the analyzers matters. Don't put a SnowballStemmer before a RegexTokenizer (can't stem whole documents or paragraphs -- just individual words), or a SnowballStopFilter after a SnowballStemmer (stemmed words, e.g. "themselv", will not appear in a stoplist). In general, the sequence should be: tokenize, normalize, stopalize, stem.</li>
+<li><b>analyzers</b> - An array of Analyzers. The order of the analyzers matters. Don’t put a SnowballStemmer before a RegexTokenizer (can’t stem whole documents or paragraphs – just individual words), or a SnowballStopFilter after a SnowballStemmer (stemmed words, e.g. “themselv”, will not appear in a stoplist). In general, the sequence should be: tokenize, normalize, stopalize, stem.</li>
</ul>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="METHODS"
->METHODS</a></h1>
+>METHODS</a></h2>
+
+<h3><a class='u'
+name="get_analyzers"
+>get_analyzers</a></h3>
+
+<pre> my $retval = $poly_analyzer->get_analyzers();</pre>
+
+<p>Getter for “analyzers” member.</p>
-<h2><a class='u' href='#___top' title='click to go to top of document'
-name="get_analyzers()"
->get_analyzers()</a></h2>
+<h3><a class='u'
+name="transform"
+>transform</a></h3>
-<p>Getter for "analyzers" member.</p>
+<pre> my $retval = $poly_analyzer->transform($inversion);</pre>
+
+<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
+>Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+
+<ul>
+<li><b>inversion</b> - An inversion.</li>
+</ul>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="INHERITANCE"
->INHERITANCE</a></h1>
+>INHERITANCE</a></h2>
<p>Lucy::Analysis::PolyAnalyzer isa <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
>Lucy::Analysis::Analyzer</a> isa Clownfish::Obj.</p>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/RegexTokenizer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/RegexTokenizer.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/RegexTokenizer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/RegexTokenizer.mdtext Fri Feb 26 12:52:25 2016
@@ -3,15 +3,15 @@ Title: Lucy::Analysis::RegexTokenizer -
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
<p>Lucy::Analysis::RegexTokenizer - Split a string into tokens.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="SYNOPSIS"
->SYNOPSIS</a></h1>
+>SYNOPSIS</a></h2>
<pre> my $whitespace_tokenizer
= Lucy::Analysis::RegexTokenizer->new( pattern => '\S+' );
@@ -27,51 +27,70 @@ name="SYNOPSIS"
my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
analyzers => [ $word_char_tokenizer, $normalizer, $stemmer ], );</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="DESCRIPTION"
->DESCRIPTION</a></h1>
+>DESCRIPTION</a></h2>
-<p>Generically, "tokenizing" is a process of breaking up a string into an array of "tokens". For instance, the string "three blind mice" might be tokenized into "three", "blind", "mice".</p>
+<p>Generically, “tokenizing” is a process of breaking up a string into an array of “tokens”. For instance, the string “three blind mice” might be tokenized into “three”, “blind”, “mice”.</p>
-<p>Lucy::Analysis::RegexTokenizer decides where it should break up the text based on a regular expression compiled from a supplied <code>pattern</code> matching one token. If our source string is...</p>
+<p>Lucy::Analysis::RegexTokenizer decides where it should break up the text based on a regular expression compiled from a supplied <code>pattern</code> matching one token. If our source string is…</p>
<pre> "Eats, Shoots and Leaves."</pre>
-<p>... then a "whitespace tokenizer" with a <code>pattern</code> of <code>"\\S+"</code> produces...</p>
+<p>… then a “whitespace tokenizer” with a <code>pattern</code> of <code>"\\S+"</code> produces…</p>
<pre> Eats,
Shoots
and
Leaves.</pre>
-<p>... while a "word character tokenizer" with a <code>pattern</code> of <code>"\\w+"</code> produces...</p>
+<p>… while a “word character tokenizer” with a <code>pattern</code> of <code>"\\w+"</code> produces…</p>
<pre> Eats
Shoots
and
Leaves</pre>
-<p>... the difference being that the word character tokenizer skips over punctuation as well as whitespace when determining token boundaries.</p>
+<p>… the difference being that the word character tokenizer skips over punctuation as well as whitespace when determining token boundaries.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="CONSTRUCTORS"
->CONSTRUCTORS</a></h1>
+>CONSTRUCTORS</a></h2>
-<h2><a class='u' href='#___top' title='click to go to top of document'
-name="new(_[labeled_params]_)"
->new( <i>[labeled params]</i> )</a></h2>
+<h3><a class='u'
+name="new"
+>new</a></h3>
<pre> my $word_char_tokenizer = Lucy::Analysis::RegexTokenizer->new(
pattern => '\w+', # required
);</pre>
+<p>Create a new RegexTokenizer.</p>
+
+<ul>
+<li><b>pattern</b> - A string specifying a Perl-syntax regular expression which should match one token. The default value is <code>\w+(?:[\x{2019}']\w+)*</code>, which matches “it’s” as well as “it” and “O’Henry’s” as well as “Henry”.</li>
+</ul>
+
+<h2><a class='u'
+name="METHODS"
+>METHODS</a></h2>
+
+<h3><a class='u'
+name="transform"
+>transform</a></h3>
+
+<pre> my $retval = $regex_tokenizer->transform($inversion);</pre>
+
+<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
+>Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+
<ul>
-<li><b>pattern</b> - A string specifying a Perl-syntax regular expression which should match one token. The default value is <code>\w+(?:[\x{2019}']\w+)*</code>, which matches "it's" as well as "it" and "O'Henry's" as well as "Henry".</li>
+<li><b>inversion</b> - An inversion.</li>
</ul>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="INHERITANCE"
->INHERITANCE</a></h1>
+>INHERITANCE</a></h2>
<p>Lucy::Analysis::RegexTokenizer isa <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
>Lucy::Analysis::Analyzer</a> isa Clownfish::Obj.</p>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStemmer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStemmer.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStemmer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStemmer.mdtext Fri Feb 26 12:52:25 2016
@@ -3,15 +3,15 @@ Title: Lucy::Analysis::SnowballStemmer -
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
<p>Lucy::Analysis::SnowballStemmer - Reduce related words to a shared root.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="SYNOPSIS"
->SYNOPSIS</a></h1>
+>SYNOPSIS</a></h2>
<pre> my $stemmer = Lucy::Analysis::SnowballStemmer->new( language => 'es' );
@@ -21,30 +21,49 @@ name="SYNOPSIS"
<p>This class is a wrapper around the Snowball stemming library, so it supports the same languages.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="DESCRIPTION"
->DESCRIPTION</a></h1>
+>DESCRIPTION</a></h2>
<p>SnowballStemmer is an <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
->Analyzer</a> which reduces related words to a root form (using the "Snowball" stemming library). For instance, "horse", "horses", and "horsing" all become "hors" -- so that a search for 'horse' will also match documents containing 'horses' and 'horsing'.</p>
+>Analyzer</a> which reduces related words to a root form (using the “Snowball” stemming library). For instance, “horse”, “horses”, and “horsing” all become “hors” – so that a search for ‘horse’ will also match documents containing ‘horses’ and ‘horsing’.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="CONSTRUCTORS"
->CONSTRUCTORS</a></h1>
+>CONSTRUCTORS</a></h2>
-<h2><a class='u' href='#___top' title='click to go to top of document'
-name="new(_[labeled_params]_)"
->new( <i>[labeled params]</i> )</a></h2>
+<h3><a class='u'
+name="new"
+>new</a></h3>
<pre> my $stemmer = Lucy::Analysis::SnowballStemmer->new( language => 'es' );</pre>
+<p>Create a new SnowballStemmer.</p>
+
<ul>
<li><b>language</b> - A two-letter ISO code identifying a language supported by Snowball.</li>
</ul>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
+name="METHODS"
+>METHODS</a></h2>
+
+<h3><a class='u'
+name="transform"
+>transform</a></h3>
+
+<pre> my $retval = $snowball_stemmer->transform($inversion);</pre>
+
+<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
+>Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+
+<ul>
+<li><b>inversion</b> - An inversion.</li>
+</ul>
+
+<h2><a class='u'
name="INHERITANCE"
->INHERITANCE</a></h1>
+>INHERITANCE</a></h2>
<p>Lucy::Analysis::SnowballStemmer isa <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
>Lucy::Analysis::Analyzer</a> isa Clownfish::Obj.</p>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStopFilter.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStopFilter.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStopFilter.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/SnowballStopFilter.mdtext Fri Feb 26 12:52:25 2016
@@ -3,15 +3,15 @@ Title: Lucy::Analysis::SnowballStopFilte
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
-<p>Lucy::Analysis::SnowballStopFilter - Suppress a "stoplist" of common words.</p>
+<p>Lucy::Analysis::SnowballStopFilter - Suppress a “stoplist” of common words.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="SYNOPSIS"
->SYNOPSIS</a></h1>
+>SYNOPSIS</a></h2>
<pre> my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
language => 'fr',
@@ -20,11 +20,11 @@ name="SYNOPSIS"
analyzers => [ $tokenizer, $normalizer, $stopfilter, $stemmer ],
);</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="DESCRIPTION"
->DESCRIPTION</a></h1>
+>DESCRIPTION</a></h2>
-<p>A "stoplist" is collection of "stopwords": words which are common enough to be of little value when determining search results. For example, so many documents in English contain "the", "if", and "maybe" that it may improve both performance and relevance to block them.</p>
+<p>A “stoplist” is collection of “stopwords”: words which are common enough to be of little value when determining search results. For example, so many documents in English contain “the”, “if”, and “maybe” that it may improve both performance and relevance to block them.</p>
<p>Before filtering stopwords:</p>
@@ -34,7 +34,8 @@ name="DESCRIPTION"
<pre> ("walrus")</pre>
-<p>SnowballStopFilter provides default stoplists for several languages, courtesy of the Snowball project (<http://snowball.tartarus.org>), or you may supply your own.</p>
+<p>SnowballStopFilter provides default stoplists for several languages, courtesy of the <a href="http://snowball.tartarus.org" class="podlinkurl"
+>Snowball project</a>, or you may supply your own.</p>
<pre> |-----------------------|
| ISO CODE | LANGUAGE |
@@ -54,13 +55,13 @@ name="DESCRIPTION"
| ru | Russian |
|-----------------------|</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="CONSTRUCTORS"
->CONSTRUCTORS</a></h1>
+>CONSTRUCTORS</a></h2>
-<h2><a class='u' href='#___top' title='click to go to top of document'
-name="new(_[labeled_params]_)"
->new( <i>[labeled params]</i> )</a></h2>
+<h3><a class='u'
+name="new"
+>new</a></h3>
<pre> my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
language => 'de',
@@ -71,15 +72,34 @@ name="new(_[labeled_params]_)"
stoplist => \%stoplist,
);</pre>
+<p>Create a new SnowballStopFilter.</p>
+
<ul>
<li><b>stoplist</b> - A hash with stopwords as the keys.</li>
<li><b>language</b> - The ISO code for a supported language.</li>
</ul>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
+name="METHODS"
+>METHODS</a></h2>
+
+<h3><a class='u'
+name="transform"
+>transform</a></h3>
+
+<pre> my $retval = $snowball_stop_filter->transform($inversion);</pre>
+
+<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
+>Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+
+<ul>
+<li><b>inversion</b> - An inversion.</li>
+</ul>
+
+<h2><a class='u'
name="INHERITANCE"
->INHERITANCE</a></h1>
+>INHERITANCE</a></h2>
<p>Lucy::Analysis::SnowballStopFilter isa <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
>Lucy::Analysis::Analyzer</a> isa Clownfish::Obj.</p>
Modified: lucy/site/trunk/content/docs/test/Lucy/Analysis/StandardTokenizer.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/StandardTokenizer.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/StandardTokenizer.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/StandardTokenizer.mdtext Fri Feb 26 12:52:25 2016
@@ -3,15 +3,15 @@ Title: Lucy::Analysis::StandardTokenizer
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
<p>Lucy::Analysis::StandardTokenizer - Split a string into tokens.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="SYNOPSIS"
->SYNOPSIS</a></h1>
+>SYNOPSIS</a></h2>
<pre> my $tokenizer = Lucy::Analysis::StandardTokenizer->new;
@@ -19,29 +19,46 @@ name="SYNOPSIS"
my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
analyzers => [ $tokenizer, $normalizer, $stemmer ], );</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="DESCRIPTION"
->DESCRIPTION</a></h1>
+>DESCRIPTION</a></h2>
-<p>Generically, "tokenizing" is a process of breaking up a string into an array of "tokens". For instance, the string "three blind mice" might be tokenized into "three", "blind", "mice".</p>
+<p>Generically, “tokenizing” is a process of breaking up a string into an array of “tokens”. For instance, the string “three blind mice” might be tokenized into “three”, “blind”, “mice”.</p>
-<p>Lucy::Analysis::StandardTokenizer breaks up the text at the word boundaries defined in Unicode Standard Annex #29. It then returns those words that start with an alphabetic or numeric character.</p>
+<p>Lucy::Analysis::StandardTokenizer breaks up the text at the word boundaries defined in Unicode Standard Annex #29. It then returns those words that contain alphabetic or numeric characters.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="CONSTRUCTORS"
->CONSTRUCTORS</a></h1>
+>CONSTRUCTORS</a></h2>
-<h2><a class='u' href='#___top' title='click to go to top of document'
-name="new()"
->new()</a></h2>
+<h3><a class='u'
+name="new"
+>new</a></h3>
<pre> my $tokenizer = Lucy::Analysis::StandardTokenizer->new;</pre>
<p>Constructor. Takes no arguments.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
+name="METHODS"
+>METHODS</a></h2>
+
+<h3><a class='u'
+name="transform"
+>transform</a></h3>
+
+<pre> my $retval = $standard_tokenizer->transform($inversion);</pre>
+
+<p>Take a single <a href="../../Lucy/Analysis/Inversion.html" class="podlinkpod"
+>Inversion</a> as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.</p>
+
+<ul>
+<li><b>inversion</b> - An inversion.</li>
+</ul>
+
+<h2><a class='u'
name="INHERITANCE"
->INHERITANCE</a></h1>
+>INHERITANCE</a></h2>
<p>Lucy::Analysis::StandardTokenizer isa <a href="../../Lucy/Analysis/Analyzer.html" class="podlinkpod"
>Lucy::Analysis::Analyzer</a> isa Clownfish::Obj.</p>
Added: lucy/site/trunk/content/docs/test/Lucy/Analysis/Token.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Analysis/Token.mdtext?rev=1732471&view=auto
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Analysis/Token.mdtext (added)
+++ lucy/site/trunk/content/docs/test/Lucy/Analysis/Token.mdtext Fri Feb 26 12:52:25 2016
@@ -0,0 +1,122 @@
+Title: Lucy::Analysis::Token - Apache Lucy Documentation
+
+<div>
+<a name='___top' class='dummyTopAnchor' ></a>
+
+<h2><a class='u'
+name="NAME"
+>NAME</a></h2>
+
+<p>Lucy::Analysis::Token - Unit of text.</p>
+
+<h2><a class='u'
+name="SYNOPSIS"
+>SYNOPSIS</a></h2>
+
+<pre> my $token = Lucy::Analysis::Token->new(
+ text => 'blind',
+ start_offset => 8,
+ end_offset => 13,
+ );
+
+ $token->set_text('mice');</pre>
+
+<h2><a class='u'
+name="DESCRIPTION"
+>DESCRIPTION</a></h2>
+
+<p>Token is the fundamental unit used by Apache Lucy’s Analyzer subclasses. Each Token has 5 attributes: <code>text</code>, <code>start_offset</code>, <code>end_offset</code>, <code>boost</code>, and <code>pos_inc</code>.</p>
+
+<p>The <code>text</code> attribute is a Unicode string encoded as UTF-8.</p>
+
+<p><code>start_offset</code> is the start point of the token text, measured in Unicode code points from the top of the stored field; <code>end_offset</code> delimits the corresponding closing boundary. <code>start_offset</code> and <code>end_offset</code> locate the Token within a larger context, even if the Token’s text attribute gets modified – by stemming, for instance. The Token for “beating” in the text “beating a dead horse” begins life with a start_offset of 0 and an end_offset of 7; after stemming, the text is “beat”, but the start_offset is still 0 and the end_offset is still 7. This allows “beating” to be highlighted correctly after a search matches “beat”.</p>
+
+<p><code>boost</code> is a per-token weight. Use this when you want to assign more or less importance to a particular token, as you might for emboldened text within an HTML document, for example. (Note: The field this token belongs to must be spec’d to use a posting of type RichPosting.)</p>
+
+<p><code>pos_inc</code> is the POSition INCrement, measured in Tokens. This attribute, which defaults to 1, is a an advanced tool for manipulating phrase matching. Ordinarily, Tokens are assigned consecutive position numbers: 0, 1, and 2 for <code>"three blind mice"</code>. However, if you set the position increment for “blind” to, say, 1000, then the three tokens will end up assigned to positions 0, 1, and 1001 – and will no longer produce a phrase match for the query <code>"three blind mice"</code>.</p>
+
+<h2><a class='u'
+name="CONSTRUCTORS"
+>CONSTRUCTORS</a></h2>
+
+<h3><a class='u'
+name="new"
+>new</a></h3>
+
+<pre> my $token = Lucy::Analysis::Token->new(
+ text => $text, # required
+ start_offset => $start_offset, # required
+ end_offset => $end_offset, # required
+ boost => 1.0, # optional
+ pos_inc => 1, # optional
+ );</pre>
+
+<ul>
+<li><b>text</b> - A string.</li>
+
+<li><b>start_offset</b> - Start offset into the original document in Unicode code points.</li>
+
+<li><b>start_offset</b> - End offset into the original document in Unicode code points.</li>
+
+<li><b>boost</b> - Per-token weight.</li>
+
+<li><b>pos_inc</b> - Position increment for phrase matching.</li>
+</ul>
+
+<h2><a class='u'
+name="METHODS"
+>METHODS</a></h2>
+
+<h3><a class='u'
+name="get_text"
+>get_text</a></h3>
+
+<pre> my $text = $token->get_text;</pre>
+
+<p>Get the token's text.</p>
+
+<h3><a class='u'
+name="set_text"
+>set_text</a></h3>
+
+<pre> $token->set_text($text);</pre>
+
+<p>Set the token's text.</p>
+
+<h3><a class='u'
+name="get_start_offset"
+>get_start_offset</a></h3>
+
+<pre> my $retval = $token->get_start_offset();</pre>
+
+<h3><a class='u'
+name="get_end_offset"
+>get_end_offset</a></h3>
+
+<pre> my $retval = $token->get_end_offset();</pre>
+
+<h3><a class='u'
+name="get_boost"
+>get_boost</a></h3>
+
+<pre> my $retval = $token->get_boost();</pre>
+
+<h3><a class='u'
+name="get_pos_inc"
+>get_pos_inc</a></h3>
+
+<pre> my $retval = $token->get_pos_inc();</pre>
+
+<h3><a class='u'
+name="get_len"
+>get_len</a></h3>
+
+<pre> my $retval = $token->get_len();</pre>
+
+<h2><a class='u'
+name="INHERITANCE"
+>INHERITANCE</a></h2>
+
+<p>Lucy::Analysis::Token isa Clownfish::Obj.</p>
+
+</div>
Modified: lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook.mdtext Fri Feb 26 12:52:25 2016
@@ -3,43 +3,43 @@ Title: Lucy::Docs::Cookbook - Apache Luc
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
-<p>Lucy::Docs::Cookbook - Apache Lucy recipes.</p>
+<p>Lucy::Docs::Cookbook - Apache Lucy recipes</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="DESCRIPTION"
->DESCRIPTION</a></h1>
+>DESCRIPTION</a></h2>
-<p>The Cookbook provides thematic documentation covering some of Apache Lucy's more sophisticated features.
+<p>The Cookbook provides thematic documentation covering some of Apache Lucy’s more sophisticated features.
For a step-by-step introduction to Lucy,
see <a href="../../Lucy/Docs/Tutorial.html" class="podlinkpod"
->Lucy::Docs::Tutorial</a>.</p>
+>Tutorial</a>.</p>
-<h2><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Chapters"
->Chapters</a></h2>
+>Chapters</a></h3>
<ul>
<li><a href="../../Lucy/Docs/Cookbook/FastUpdates.html" class="podlinkpod"
->Lucy::Docs::Cookbook::FastUpdates</a> - While index updates are fast on average,
+>FastUpdates</a> - While index updates are fast on average,
worst-case update performance may be significantly slower.
To make index updates consistently quick,
we must manually intervene to control the process of index segment consolidation.</li>
<li><a href="../../Lucy/Docs/Cookbook/CustomQuery.html" class="podlinkpod"
->Lucy::Docs::Cookbook::CustomQuery</a> - Explore Lucy's support for custom query types by creating a "PrefixQuery" class to handle trailing wildcards.</li>
+>CustomQuery</a> - Explore Lucy’s support for custom query types by creating a “PrefixQuery” class to handle trailing wildcards.</li>
<li><a href="../../Lucy/Docs/Cookbook/CustomQueryParser.html" class="podlinkpod"
->Lucy::Docs::Cookbook::CustomQueryParser</a> - Define your own custom search query syntax using Lucy::Search::QueryParser and <a href="../../Parse/RecDescent.html" class="podlinkpod"
->Parse::RecDescent</a>.</li>
+>CustomQueryParser</a> - Define your own custom search query syntax using <a href="../../Lucy/Search/QueryParser.html" class="podlinkpod"
+>QueryParser</a> and Parse::RecDescent.</li>
</ul>
-<h2><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Materials"
->Materials</a></h2>
+>Materials</a></h3>
<p>Some of the recipes in the Cookbook reference the completed <a href="../../Lucy/Docs/Tutorial.html" class="podlinkpod"
>Tutorial</a> application.
Modified: lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQuery.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQuery.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQuery.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQuery.mdtext Fri Feb 26 12:52:25 2016
@@ -3,17 +3,17 @@ Title: Lucy::Docs::Cookbook::CustomQuery
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
-<p>Lucy::Docs::Cookbook::CustomQuery - Sample subclass of Query.</p>
+<p>Lucy::Docs::Cookbook::CustomQuery - Sample subclass of Query</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
-name="ABSTRACT"
->ABSTRACT</a></h1>
+<h2><a class='u'
+name="DESCRIPTION"
+>DESCRIPTION</a></h2>
-<p>Explore Apache Lucy's support for custom query types by creating a "PrefixQuery" class to handle trailing wildcards.</p>
+<p>Explore Apache Lucy’s support for custom query types by creating a “PrefixQuery” class to handle trailing wildcards.</p>
<pre> my $prefix_query = PrefixQuery->new(
field => 'content',
@@ -22,30 +22,33 @@ name="ABSTRACT"
my $hits = $searcher->hits( query => $prefix_query );
...</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Query,_Compiler,_and_Matcher"
->Query, Compiler, and Matcher</a></h1>
+>Query, Compiler, and Matcher</a></h3>
<p>To add support for a new query type, we need three classes: a Query, a Compiler, and a Matcher.</p>
<ul>
<li>PrefixQuery - a subclass of <a href="../../../Lucy/Search/Query.html" class="podlinkpod"
->Lucy::Search::Query</a>, and the only class that client code will deal with directly.</li>
+>Query</a>, and the only class that client code will deal with directly.</li>
<li>PrefixCompiler - a subclass of <a href="../../../Lucy/Search/Compiler.html" class="podlinkpod"
->Lucy::Search::Compiler</a>, whose primary role is to compile a PrefixQuery to a PrefixMatcher.</li>
+>Compiler</a>, whose primary role is to compile a PrefixQuery to a PrefixMatcher.</li>
<li>PrefixMatcher - a subclass of <a href="../../../Lucy/Search/Matcher.html" class="podlinkpod"
->Lucy::Search::Matcher</a>, which does the heavy lifting: it applies the query to individual documents and assigns a score to each match.</li>
+>Matcher</a>, which does the heavy lifting: it applies the query to individual documents and assigns a score to each match.</li>
</ul>
-<p>The PrefixQuery class on its own isn't enough because a Query object's role is limited to expressing an abstract specification for the search. A Query is basically nothing but metadata; execution is left to the Query's companion Compiler and Matcher.</p>
+<p>The PrefixQuery class on its own isn’t enough because a Query object’s role is limited to expressing an abstract specification for the search. A Query is basically nothing but metadata; execution is left to the Query’s companion Compiler and Matcher.</p>
-<p>Here's a simplified sketch illustrating how a Searcher's hits() method ties together the three classes.</p>
+<p>Here’s a simplified sketch illustrating how a Searcher’s hits() method ties together the three classes.</p>
<pre> sub hits {
my ( $self, $query ) = @_;
- my $compiler = $query->make_compiler( searcher => $self );
+ my $compiler = $query->make_compiler(
+ searcher => $self,
+ boost => $query->get_boost,
+ );
my $matcher = $compiler->make_matcher(
reader => $self->get_reader,
need_score => 1,
@@ -54,9 +57,9 @@ name="Query,_Compiler,_and_Matcher"
return \@hits;
}</pre>
-<h2><a class='u' href='#___top' title='click to go to top of document'
+<h4><a class='u'
name="PrefixQuery"
->PrefixQuery</a></h2>
+>PrefixQuery</a></h4>
<p>Our PrefixQuery class will have two attributes: a query string and a field name.</p>
@@ -71,7 +74,7 @@ name="PrefixQuery"
sub get_query_string { my $self = shift; return $query_string{$$self} }
sub get_field { my $self = shift; return $field{$$self} }</pre>
-<p>PrefixQuery's constructor collects and validates the attributes.</p>
+<p>PrefixQuery’s constructor collects and validates the attributes.</p>
<pre> sub new {
my ( $class, %args ) = @_;
@@ -89,7 +92,7 @@ name="PrefixQuery"
return $self;
}</pre>
-<p>Since this is an inside-out class, we'll need a destructor:</p>
+<p>Since this is an inside-out class, we’ll need a destructor:</p>
<pre> sub DESTROY {
my $self = shift;
@@ -109,7 +112,7 @@ name="PrefixQuery"
return 1;
}</pre>
-<p>The last thing we'll need is a make_compiler() factory method which kicks out a subclass of <a href="../../../Lucy/Search/Compiler.html" class="podlinkpod"
+<p>The last thing we’ll need is a make_compiler() factory method which kicks out a subclass of <a href="../../../Lucy/Search/Compiler.html" class="podlinkpod"
>Compiler</a>.</p>
<pre> sub make_compiler {
@@ -120,39 +123,38 @@ name="PrefixQuery"
return $compiler;
}</pre>
-<h2><a class='u' href='#___top' title='click to go to top of document'
+<h4><a class='u'
name="PrefixCompiler"
->PrefixCompiler</a></h2>
+>PrefixCompiler</a></h4>
-<p>PrefixQuery's make_compiler() method will be called internally at search-time by objects which subclass <a href="../../../Lucy/Search/Searcher.html" class="podlinkpod"
->Lucy::Search::Searcher</a> -- such as <a href="../../../Lucy/Search/IndexSearcher.html" class="podlinkpod"
+<p>PrefixQuery’s make_compiler() method will be called internally at search-time by objects which subclass <a href="../../../Lucy/Search/Searcher.html" class="podlinkpod"
+>Searcher</a> – such as <a href="../../../Lucy/Search/IndexSearcher.html" class="podlinkpod"
>IndexSearchers</a>.</p>
-<p>A Searcher is associated with a particular collection of documents. These documents may all reside in one index, as with IndexSearcher, or they may be spread out across multiple indexes on one or more machines, as with <a href="../../../LucyX/Remote/ClusterSearcher.html" class="podlinkpod"
->LucyX::Remote::ClusterSearcher</a>.</p>
+<p>A Searcher is associated with a particular collection of documents. These documents may all reside in one index, as with IndexSearcher, or they may be spread out across multiple indexes on one or more machines, as with LucyX::Remote::ClusterSearcher.</p>
-<p>Searcher objects have access to certain statistical information about the collections they represent; for instance, a Searcher can tell you how many documents are in the collection...</p>
+<p>Searcher objects have access to certain statistical information about the collections they represent; for instance, a Searcher can tell you how many documents are in the collection…</p>
<pre> my $maximum_number_of_docs_in_collection = $searcher->doc_max;</pre>
-<p>... or how many documents a specific term appears in:</p>
+<p>… or how many documents a specific term appears in:</p>
<pre> my $term_appears_in_this_many_docs = $searcher->doc_freq(
field => 'content',
term => 'foo',
);</pre>
-<p>Such information can be used by sophisticated Compiler implementations to assign more or less heft to individual queries or sub-queries. However, we're not going to bother with weighting for this demo; we'll just assign a fixed score of 1.0 to each matching document.</p>
+<p>Such information can be used by sophisticated Compiler implementations to assign more or less heft to individual queries or sub-queries. However, we’re not going to bother with weighting for this demo; we’ll just assign a fixed score of 1.0 to each matching document.</p>
-<p>We don't need to write a constructor, as it will suffice to inherit new() from Lucy::Search::Compiler. The only method we need to implement for PrefixCompiler is make_matcher().</p>
+<p>We don’t need to write a constructor, as it will suffice to inherit new() from Lucy::Search::Compiler. The only method we need to implement for PrefixCompiler is make_matcher().</p>
<pre> package PrefixCompiler;
use base qw( Lucy::Search::Compiler );
-
+
sub make_matcher {
my ( $self, %args ) = @_;
my $seg_reader = $args{reader};
-
+
# Retrieve low-level components LexiconReader and PostingListReader.
my $lex_reader
= $seg_reader->obtain("Lucy::Index::LexiconReader");
@@ -189,14 +191,14 @@ name="PrefixCompiler"
>SegReader</a> object when make_matcher() gets called. From the SegReader and its sub-components <a href="../../../Lucy/Index/LexiconReader.html" class="podlinkpod"
>LexiconReader</a> and <a href="../../../Lucy/Index/PostingListReader.html" class="podlinkpod"
>PostingListReader</a>, we acquire a <a href="../../../Lucy/Index/Lexicon.html" class="podlinkpod"
->Lexicon</a>, scan through the Lexicon's unique terms, and acquire a <a href="../../../Lucy/Index/PostingList.html" class="podlinkpod"
+>Lexicon</a>, scan through the Lexicon’s unique terms, and acquire a <a href="../../../Lucy/Index/PostingList.html" class="podlinkpod"
>PostingList</a> for each term that matches our prefix.</p>
<p>Each of these PostingList objects represents a set of documents which match the query.</p>
-<h2><a class='u' href='#___top' title='click to go to top of document'
+<h4><a class='u'
name="PrefixMatcher"
->PrefixMatcher</a></h2>
+>PrefixMatcher</a></h4>
<p>The Matcher subclass is the most involved.</p>
@@ -259,21 +261,21 @@ name="PrefixMatcher"
return $tick < scalar @$doc_ids ? $doc_ids->[$tick] : 0;
}</pre>
-<p>score() conveys the relevance score of the current match. We'll just return a fixed score of 1.0:</p>
+<p>score() conveys the relevance score of the current match. We’ll just return a fixed score of 1.0:</p>
<pre> sub score { 1.0 }</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Usage"
->Usage</a></h1>
+>Usage</a></h3>
<p>To get a basic feel for PrefixQuery, insert the FlatQueryParser module described in <a href="../../../Lucy/Docs/Cookbook/CustomQueryParser.html" class="podlinkpod"
->Lucy::Docs::Cookbook::CustomQueryParser</a> (which supports PrefixQuery) into the search.cgi sample app.</p>
+>CustomQueryParser</a> (which supports PrefixQuery) into the search.cgi sample app.</p>
<pre> my $parser = FlatQueryParser->new( schema => $searcher->get_schema );
my $query = $parser->parse($q);</pre>
-<p>If you're planning on using PrefixQuery in earnest, though, you may want to change up analyzers to avoid stemming, because stemming -- another approach to prefix conflation -- is not perfectly compatible with prefix searches.</p>
+<p>If you’re planning on using PrefixQuery in earnest, though, you may want to change up analyzers to avoid stemming, because stemming – another approach to prefix conflation – is not perfectly compatible with prefix searches.</p>
<pre> # Polyanalyzer with no SnowballStemmer.
my $analyzer = Lucy::Analysis::PolyAnalyzer->new(
Modified: lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQueryParser.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQueryParser.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQueryParser.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/CustomQueryParser.mdtext Fri Feb 26 12:52:25 2016
@@ -3,43 +3,43 @@ Title: Lucy::Docs::Cookbook::CustomQuery
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
<p>Lucy::Docs::Cookbook::CustomQueryParser - Sample subclass of QueryParser.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
-name="ABSTRACT"
->ABSTRACT</a></h1>
+<h2><a class='u'
+name="DESCRIPTION"
+>DESCRIPTION</a></h2>
<p>Implement a custom search query language using a subclass of <a href="../../../Lucy/Search/QueryParser.html" class="podlinkpod"
->Lucy::Search::QueryParser</a>.</p>
+>QueryParser</a>.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="The_language"
->The language</a></h1>
+>The language</a></h3>
<p>At first,
our query language will support only simple term queries and phrases delimited by double quotes.
-For simplicity's sake,
+For simplicity’s sake,
it will not support parenthetical groupings,
boolean operators,
or prepended plus/minus.
-The results for all subqueries will be unioned together -- i.e.
-joined using an OR -- which is usually the best approach for small-to-medium-sized document collections.</p>
+The results for all subqueries will be unioned together – i.e.
+joined using an OR – which is usually the best approach for small-to-medium-sized document collections.</p>
<p>Later,
-we'll add support for trailing wildcards.</p>
+we’ll add support for trailing wildcards.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Single-field_parser"
->Single-field parser</a></h1>
+>Single-field parser</a></h3>
<p>Our initial parser implentation will generate queries against a single fixed field,
-"content",
+“content”,
and it will analyze text using a fixed choice of English EasyAnalyzer.
-We won't subclass Lucy::Search::QueryParser just yet.</p>
+We won’t subclass Lucy::Search::QueryParser just yet.</p>
<pre> package FlatQueryParser;
use Lucy::Search::TermQuery;
@@ -95,7 +95,7 @@ We won't subclass Lucy::Search::Quer
return \@tokens;
}</pre>
-<p>The main parsing routine creates an array of tokens by calling _tokenize(), runs the tokens through through the EasyAnalyzer, creates TermQuery or PhraseQuery objects according to how many tokens emerge from the EasyAnalyzer's split() method, and adds each of the sub-queries to the primary ORQuery.</p>
+<p>The main parsing routine creates an array of tokens by calling _tokenize(), runs the tokens through through the EasyAnalyzer, creates TermQuery or PhraseQuery objects according to how many tokens emerge from the EasyAnalyzer’s split() method, and adds each of the sub-queries to the primary ORQuery.</p>
<pre> sub parse {
my ( $self, $query_string ) = @_;
@@ -126,21 +126,21 @@ We won't subclass Lucy::Search::Quer
return $or_query;
}</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Multi-field_parser"
->Multi-field parser</a></h1>
+>Multi-field parser</a></h3>
-<p>Most often, the end user will want their search query to match not only a single 'content' field, but also 'title' and so on. To make that happen, we have to turn queries such as this...</p>
+<p>Most often, the end user will want their search query to match not only a single ‘content’ field, but also ‘title’ and so on. To make that happen, we have to turn queries such as this…</p>
<pre> foo AND NOT bar</pre>
-<p>... into the logical equivalent of this:</p>
+<p>… into the logical equivalent of this:</p>
<pre> (title:foo OR content:foo) AND NOT (title:bar OR content:bar)</pre>
-<p>Rather than continue with our own from-scratch parser class and write the routines to accomplish that expansion, we're now going to subclass Lucy::Search::QueryParser and take advantage of some of its existing methods.</p>
+<p>Rather than continue with our own from-scratch parser class and write the routines to accomplish that expansion, we’re now going to subclass Lucy::Search::QueryParser and take advantage of some of its existing methods.</p>
-<p>Our first parser implementation had the "content" field name and the choice of English EasyAnalyzer hard-coded for simplicity, but we don't need to do that once we subclass Lucy::Search::QueryParser. QueryParser's constructor -- which we will inherit, allowing us to eliminate our own constructor -- requires a Schema which conveys field and Analyzer information, so we can just defer to that.</p>
+<p>Our first parser implementation had the “content” field name and the choice of English EasyAnalyzer hard-coded for simplicity, but we don’t need to do that once we subclass Lucy::Search::QueryParser. QueryParser’s constructor – which we will inherit, allowing us to eliminate our own constructor – requires a Schema which conveys field and Analyzer information, so we can just defer to that.</p>
<pre> package FlatQueryParser;
use base qw( Lucy::Search::QueryParser );
@@ -152,7 +152,7 @@ name="Multi-field_parser"
# Inherit new()</pre>
-<p>We're also going to jettison our _make_term_query() and _make_phrase_query() helper subs and chop our parse() subroutine way down. Our revised parse() routine will generate Lucy::Search::LeafQuery objects instead of TermQueries and PhraseQueries:</p>
+<p>We’re also going to jettison our _make_term_query() and _make_phrase_query() helper subs and chop our parse() subroutine way down. Our revised parse() routine will generate Lucy::Search::LeafQuery objects instead of TermQueries and PhraseQueries:</p>
<pre> sub parse {
my ( $self, $query_string ) = @_;
@@ -165,11 +165,11 @@ name="Multi-field_parser"
return $self->expand($or_query);
}</pre>
-<p>The magic happens in QueryParser's expand() method, which walks the ORQuery object we supply to it looking for LeafQuery objects, and calls expand_leaf() for each one it finds. expand_leaf() performs field-specific analysis, decides whether each query should be a TermQuery or a PhraseQuery, and if multiple fields are required, creates an ORQuery which mults out e.g. <code>foo</code> into <code>(title:foo OR content:foo)</code>.</p>
+<p>The magic happens in QueryParser’s expand() method, which walks the ORQuery object we supply to it looking for LeafQuery objects, and calls expand_leaf() for each one it finds. expand_leaf() performs field-specific analysis, decides whether each query should be a TermQuery or a PhraseQuery, and if multiple fields are required, creates an ORQuery which mults out e.g. <code>foo</code> into <code>(title:foo OR content:foo)</code>.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Extending_the_query_language"
->Extending the query language</a></h1>
+>Extending the query language</a></h3>
<p>To add support for trailing wildcards to our query language, we need to override expand_leaf() to accommodate PrefixQuery, while deferring to the parent class implementation on TermQuery and PhraseQuery.</p>
@@ -192,11 +192,11 @@ name="Extending_the_query_language"
}
}</pre>
-<p>Ordinarily, those asterisks would have been stripped when running tokens through the EasyAnalyzer -- query strings containing "foo*" would produce TermQueries for the term "foo". Our override intercepts tokens with trailing asterisks and processes them as PrefixQueries before <code>SUPER::expand_leaf</code> can discard them, so that a search for "foo*" can match "food", "foosball", and so on.</p>
+<p>Ordinarily, those asterisks would have been stripped when running tokens through the EasyAnalyzer – query strings containing “foo*” would produce TermQueries for the term “foo”. Our override intercepts tokens with trailing asterisks and processes them as PrefixQueries before <code>SUPER::expand_leaf</code> can discard them, so that a search for “foo*” can match “food”, “foosball”, and so on.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Usage"
->Usage</a></h1>
+>Usage</a></h3>
<p>Insert our custom parser into the search.cgi sample app to get a feel for how it behaves:</p>
Modified: lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/FastUpdates.mdtext
URL: http://svn.apache.org/viewvc/lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/FastUpdates.mdtext?rev=1732471&r1=1732470&r2=1732471&view=diff
==============================================================================
--- lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/FastUpdates.mdtext (original)
+++ lucy/site/trunk/content/docs/test/Lucy/Docs/Cookbook/FastUpdates.mdtext Fri Feb 26 12:52:25 2016
@@ -3,24 +3,24 @@ Title: Lucy::Docs::Cookbook::FastUpdates
<div>
<a name='___top' class='dummyTopAnchor' ></a>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h2><a class='u'
name="NAME"
->NAME</a></h1>
+>NAME</a></h2>
-<p>Lucy::Docs::Cookbook::FastUpdates - Near real-time index updates.</p>
+<p>Lucy::Docs::Cookbook::FastUpdates - Near real-time index updates</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
-name="ABSTRACT"
->ABSTRACT</a></h1>
+<h2><a class='u'
+name="DESCRIPTION"
+>DESCRIPTION</a></h2>
<p>While index updates are fast on average,
worst-case update performance may be significantly slower.
To make index updates consistently quick,
we must manually intervene to control the process of index segment consolidation.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="The_problem"
->The problem</a></h1>
+>The problem</a></h3>
<p>Ordinarily,
modifying an index is cheap.
@@ -28,7 +28,7 @@ New data is added to new segments,
and the time to write a new segment scales more or less linearly with the number of documents added during the indexing session.</p>
<p>Deletions are also cheap most of the time,
-because we don't remove documents immediately but instead mark them as deleted,
+because we don’t remove documents immediately but instead mark them as deleted,
and adding the deletion mark is cheap.</p>
<p>However,
@@ -44,15 +44,16 @@ Every once in a while,
though,
a large amount of data must be rewritten.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Procrastinating_and_playing_catch-up"
->Procrastinating and playing catch-up</a></h1>
+>Procrastinating and playing catch-up</a></h3>
<p>The simplest way to force fast index updates is to avoid rewriting anything.</p>
<p>Indexer relies upon <a href="../../../Lucy/Index/IndexManager.html" class="podlinkpod"
->IndexManager</a>'s recycle() method to tell it which segments should be consolidated.
-If we subclass IndexManager and override recycle() so that it always returns an empty array,
+>IndexManager</a>’s <a href="../../../Lucy/Index/IndexManager.html#recycle" class="podlinkpod"
+>recycle()</a> method to tell it which segments should be consolidated.
+If we subclass IndexManager and override the method so that it always returns an empty array,
we get consistently quick performance:</p>
<pre> package NoMergeManager;
@@ -67,7 +68,7 @@ we get consistently quick performance:</
...
$indexer->commit;</pre>
-<p>However, we can't procrastinate forever. Eventually, we'll have to run an ordinary, uncontrolled indexing session, potentially triggering a large rewrite of lots of small and/or degraded segments:</p>
+<p>However, we can’t procrastinate forever. Eventually, we’ll have to run an ordinary, uncontrolled indexing session, potentially triggering a large rewrite of lots of small and/or degraded segments:</p>
<pre> my $indexer = Lucy::Index::Indexer->new(
index => '/path/to/index',
@@ -76,9 +77,9 @@ we get consistently quick performance:</
...
$indexer->commit;</pre>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Acceptable_worst-case_update_time,_slower_degradation"
->Acceptable worst-case update time, slower degradation</a></h1>
+>Acceptable worst-case update time, slower degradation</a></h3>
<p>Never merging anything at all in the main indexing process is probably overkill. Small segments are relatively cheap to merge; we just need to guard against the big rewrites.</p>
@@ -96,13 +97,14 @@ name="Acceptable_worst-case_update_time,
<p>However, we still have to consolidate every once in a while, and while that happens content updates will be locked out.</p>
-<h1><a class='u' href='#___top' title='click to go to top of document'
+<h3><a class='u'
name="Background_merging"
->Background merging</a></h1>
+>Background merging</a></h3>
-<p>If it's not acceptable to lock out updates while the index consolidation process runs, the alternative is to move the consolidation process out of band, using Lucy::Index::BackgroundMerger.</p>
+<p>If it’s not acceptable to lock out updates while the index consolidation process runs, the alternative is to move the consolidation process out of band, using <a href="../../../Lucy/Index/BackgroundMerger.html" class="podlinkpod"
+>BackgroundMerger</a>.</p>
-<p>It's never safe to have more than one Indexer attempting to modify the content of an index at the same time, but a BackgroundMerger and an Indexer can operate simultaneously:</p>
+<p>It’s never safe to have more than one Indexer attempting to modify the content of an index at the same time, but a BackgroundMerger and an Indexer can operate simultaneously:</p>
<pre> # Indexing process.
use Scalar::Util qw( blessed );
@@ -126,7 +128,7 @@ name="Background_merging"
die "Write failed: $@";
}
}
-
+
# Background merge process.
my $manager = Lucy::Index::IndexManager->new;
$manager->set_write_lock_timeout(60_000);
@@ -136,6 +138,8 @@ name="Background_merging"
);
$bg_merger->commit;</pre>
-<p>The exception handling code becomes useful once you have more than one index modification process happening simultaneously. By default, Indexer tries several times to acquire a write lock over the span of one second, then holds it until commit() completes. BackgroundMerger handles most of its work without the write lock, but it does need it briefly once at the beginning and once again near the end. Under normal loads, the internal retry logic will resolve conflicts, but if it's not acceptable to miss an insert, you probably want to catch LockErr exceptions thrown by Indexer. In contrast, a LockErr from BackgroundMerger probably just needs to be logged.</p>
+<p>The exception handling code becomes useful once you have more than one index modification process happening simultaneously. By default, Indexer tries several times to acquire a write lock over the span of one second, then holds it until <a href="../../../Lucy/Index/Indexer.html#commit" class="podlinkpod"
+>commit()</a> completes. BackgroundMerger handles most of its work without the write lock, but it does need it briefly once at the beginning and once again near the end. Under normal loads, the internal retry logic will resolve conflicts, but if it’s not acceptable to miss an insert, you probably want to catch <a href="../../../Lucy/Store/LockErr.html" class="podlinkpod"
+>LockErr</a> exceptions thrown by Indexer. In contrast, a LockErr from BackgroundMerger probably just needs to be logged.</p>
</div>