You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mahout.apache.org by bu...@apache.org on 2014/04/30 22:53:14 UTC
svn commit: r907508 - in /websites/staging/mahout/trunk/content: ./
users/classification/bayesian.html users/sparkbindings/ScalaSparkBindings.pdf
Author: buildbot
Date: Wed Apr 30 20:53:13 2014
New Revision: 907508
Log:
Staging update by buildbot for mahout
Modified:
websites/staging/mahout/trunk/content/ (props changed)
websites/staging/mahout/trunk/content/users/classification/bayesian.html
websites/staging/mahout/trunk/content/users/sparkbindings/ScalaSparkBindings.pdf
Propchange: websites/staging/mahout/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Wed Apr 30 20:53:13 2014
@@ -1 +1 @@
-1590502
+1591479
Modified: websites/staging/mahout/trunk/content/users/classification/bayesian.html
==============================================================================
--- websites/staging/mahout/trunk/content/users/classification/bayesian.html (original)
+++ websites/staging/mahout/trunk/content/users/classification/bayesian.html Wed Apr 30 20:53:13 2014
@@ -262,11 +262,11 @@
</ul>
<p>As we can see, the main difference between Bayes and CBayes is the weight calculation step. Where Bayes weighs terms more heavily based on the likelihood that they belong to class <code>\(c\)</code>, CBayes seeks to maximize term weights on the likelihood that they do not belong to any other class. </p>
<h3 id="running-from-the-command-line">Running from the command line</h3>
-<p>Mahout provides CLI drivers for all above steps. Here we will give a simple overview of Mahout CLI commands used to preprocess the data, train the model and assign labels to the training set. An <a href="http://svn.apache.org/repos/asf/mahout/trunk/examples/bin/classify-20newsgroups.sh">example script</a> is given for the full process from data acquisition through classification of the classic <a href="https://mahout.apache.org/users/classification/twenty-newsgroups.html">20 Newsgroups corpus</a>. </p>
+<p>Mahout provides CLI drivers for all above steps. Here we will give a simple overview of Mahout CLI commands used to preprocess the data, train the model and assign labels to the training set. An <a href="http://svn.apache.org/repos/asf/mahout/trunk/examples/bin/classify-20newsgroups.sh">example script</a> is given for the full process from data acquisition through classification of the classic <a href="http://mahout.apache.org/users/classification/twenty-newsgroups.html">20 Newsgroups corpus</a>. </p>
<ul>
<li>
<p><strong>Preprocessing:</strong>
-For a set of Sequence File Formatted documents in PATH_TO_SEQUENCE_FILES the <a href="https://mahout.apache.org/users/basics/creating-vectors-from-text.html">mahout seq2sparse</a> command performs the TF-IDF transformations (-wt tfidf option) and L2 length normalization (-n 2 option) as follows:</p>
+For a set of Sequence File Formatted documents in PATH_TO_SEQUENCE_FILES the <a href="http://mahout.apache.org/users/basics/creating-vectors-from-text.html">mahout seq2sparse</a> command performs the TF-IDF transformations (-wt tfidf option) and L2 length normalization (-n 2 option) as follows:</p>
<div class="codehilite"><pre>mahout seq2sparse
-i <span class="cp">${</span><span class="n">PATH_TO_SEQUENCE_FILES</span><span class="cp">}</span>
-o <span class="cp">${</span><span class="n">PATH_TO_TFIDF_VECTORS</span><span class="cp">}</span>
@@ -307,11 +307,52 @@ Classification and testing on a holdout
</li>
</ul>
+<h3 id="command-line-options">Command line options</h3>
+<ul>
+<li>
+<p><strong>Training</strong>:</p>
+<div class="codehilite"><pre><span class="n">mahout</span> <span class="n">trainnb</span>
+ <span class="o">--</span><span class="n">input</span> <span class="p">(</span><span class="o">-</span><span class="nb">i</span><span class="p">)</span> <span class="n">input</span> <span class="n">Path</span> <span class="n">to</span> <span class="n">job</span> <span class="n">input</span> <span class="n">directory</span><span class="p">.</span>
+ <span class="o">--</span><span class="n">output</span> <span class="p">(</span><span class="o">-</span><span class="n">o</span><span class="p">)</span> <span class="n">output</span> <span class="n">The</span> <span class="n">directory</span> <span class="n">pathname</span> <span class="k">for</span> <span class="n">output</span><span class="p">.</span>
+ <span class="o">--</span><span class="n">labels</span> <span class="p">(</span><span class="o">-</span><span class="n">l</span><span class="p">)</span> <span class="n">labels</span> <span class="n">comma</span><span class="o">-</span><span class="n">separated</span> <span class="n">list</span> <span class="n">of</span> <span class="n">labels</span> <span class="n">to</span> <span class="n">include</span> <span class="n">in</span>
+ <span class="n">training</span>
+ <span class="o">--</span><span class="n">extractLabels</span> <span class="p">(</span><span class="o">-</span><span class="n">el</span><span class="p">)</span> <span class="n">Extract</span> <span class="n">the</span> <span class="n">labels</span> <span class="n">from</span> <span class="n">the</span> <span class="n">input</span>
+ <span class="o">--</span><span class="n">trainComplementary</span> <span class="p">(</span><span class="o">-</span><span class="n">c</span><span class="p">)</span> <span class="n">train</span> <span class="n">complementary</span>?
+ <span class="o">--</span><span class="n">labelIndex</span> <span class="p">(</span><span class="o">-</span><span class="n">li</span><span class="p">)</span> <span class="n">labelIndex</span> <span class="n">The</span> <span class="n">path</span> <span class="n">to</span> <span class="n">store</span> <span class="n">the</span> <span class="n">label</span> <span class="n">index</span> <span class="n">in</span>
+ <span class="o">--</span><span class="n">overwrite</span> <span class="p">(</span><span class="o">-</span><span class="n">ow</span><span class="p">)</span> <span class="n">If</span> <span class="n">present</span><span class="p">,</span> <span class="n">overwrite</span> <span class="n">the</span> <span class="n">output</span> <span class="n">directory</span>
+ <span class="n">before</span> <span class="n">running</span> <span class="n">job</span>
+ <span class="o">--</span><span class="n">help</span> <span class="p">(</span><span class="o">-</span><span class="n">h</span><span class="p">)</span> <span class="n">Print</span> <span class="n">out</span> <span class="n">help</span>
+ <span class="o">--</span><span class="n">tempDir</span> <span class="n">tempDir</span> <span class="n">Intermediate</span> <span class="n">output</span> <span class="n">directory</span>
+ <span class="o">--</span><span class="n">startPhase</span> <span class="n">startPhase</span> <span class="n">First</span> <span class="n">phase</span> <span class="n">to</span> <span class="n">run</span>
+ <span class="o">--</span><span class="n">endPhase</span> <span class="n">endPhase</span> <span class="n">Last</span> <span class="n">phase</span> <span class="n">to</span> <span class="n">run</span>
+</pre></div>
+
+
+</li>
+<li>
+<p><strong>Testing</strong></p>
+<div class="codehilite"><pre><span class="n">mahout</span> <span class="n">testnb</span>
+ <span class="o">--</span><span class="n">input</span> <span class="p">(</span><span class="o">-</span><span class="nb">i</span><span class="p">)</span> <span class="n">input</span> <span class="n">Path</span> <span class="n">to</span> <span class="n">job</span> <span class="n">input</span> <span class="n">directory</span><span class="p">.</span>
+ <span class="o">--</span><span class="n">output</span> <span class="p">(</span><span class="o">-</span><span class="n">o</span><span class="p">)</span> <span class="n">output</span> <span class="n">The</span> <span class="n">directory</span> <span class="n">pathname</span> <span class="k">for</span> <span class="n">output</span><span class="p">.</span>
+ <span class="o">--</span><span class="n">overwrite</span> <span class="p">(</span><span class="o">-</span><span class="n">ow</span><span class="p">)</span> <span class="n">If</span> <span class="n">present</span><span class="p">,</span> <span class="n">overwrite</span> <span class="n">the</span> <span class="n">output</span> <span class="n">directory</span>
+ <span class="n">before</span> <span class="n">running</span> <span class="n">job</span>
+ <span class="o">--</span><span class="n">model</span> <span class="p">(</span><span class="o">-</span><span class="n">m</span><span class="p">)</span> <span class="n">model</span> <span class="n">The</span> <span class="n">path</span> <span class="n">to</span> <span class="n">the</span> <span class="n">model</span> <span class="n">built</span> <span class="n">during</span> <span class="n">training</span>
+ <span class="o">--</span><span class="n">testComplementary</span> <span class="p">(</span><span class="o">-</span><span class="n">c</span><span class="p">)</span> <span class="n">test</span> <span class="n">complementary</span>?
+ <span class="o">--</span><span class="n">runSequential</span> <span class="p">(</span><span class="o">-</span><span class="n">seq</span><span class="p">)</span> <span class="n">run</span> <span class="n">sequential</span>?
+ <span class="o">--</span><span class="n">labelIndex</span> <span class="p">(</span><span class="o">-</span><span class="n">l</span><span class="p">)</span> <span class="n">labelIndex</span> <span class="n">The</span> <span class="n">path</span> <span class="n">to</span> <span class="n">the</span> <span class="n">location</span> <span class="n">of</span> <span class="n">the</span> <span class="n">label</span> <span class="n">index</span>
+ <span class="o">--</span><span class="n">help</span> <span class="p">(</span><span class="o">-</span><span class="n">h</span><span class="p">)</span> <span class="n">Print</span> <span class="n">out</span> <span class="n">help</span>
+ <span class="o">--</span><span class="n">tempDir</span> <span class="n">tempDir</span> <span class="n">Intermediate</span> <span class="n">output</span> <span class="n">directory</span>
+ <span class="o">--</span><span class="n">startPhase</span> <span class="n">startPhase</span> <span class="n">First</span> <span class="n">phase</span> <span class="n">to</span> <span class="n">run</span>
+ <span class="o">--</span><span class="n">endPhase</span> <span class="n">endPhase</span> <span class="n">Last</span> <span class="n">phase</span> <span class="n">to</span> <span class="n">run</span>
+</pre></div>
+
+
+</li>
+</ul>
<h3 id="examples">Examples</h3>
-<p>Mahout provides 2 examples for Naive Bayes classification:</p>
+<p>Mahout provides an example for Naive Bayes classification:</p>
<ol>
<li><a href="twenty-newsgroups.html">Classify 20 Newsgroups</a></li>
-<li><a href="wikipedia-bayes-example.html">Wikipedia Naive Bayes Example</a></li>
</ol>
<h3 id="references">References</h3>
<p>[1]: Jason D. M. Rennie, Lawerence Shih, Jamie Teevan, David Karger (2003). <a href="http://people.csail.mit.edu/jrennie/papers/icml03-nb.pdf">Tackling the Poor Assumptions of Naive Bayes Text Classifiers</a>. Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003).</p>
Modified: websites/staging/mahout/trunk/content/users/sparkbindings/ScalaSparkBindings.pdf
==============================================================================
Binary files - no diff available.