You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mahout.apache.org by bu...@apache.org on 2014/04/30 22:53:14 UTC

svn commit: r907508 - in /websites/staging/mahout/trunk/content: ./ users/classification/bayesian.html users/sparkbindings/ScalaSparkBindings.pdf

Author: buildbot
Date: Wed Apr 30 20:53:13 2014
New Revision: 907508

Log:
Staging update by buildbot for mahout

Modified:
    websites/staging/mahout/trunk/content/   (props changed)
    websites/staging/mahout/trunk/content/users/classification/bayesian.html
    websites/staging/mahout/trunk/content/users/sparkbindings/ScalaSparkBindings.pdf

Propchange: websites/staging/mahout/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Wed Apr 30 20:53:13 2014
@@ -1 +1 @@
-1590502
+1591479

Modified: websites/staging/mahout/trunk/content/users/classification/bayesian.html
==============================================================================
--- websites/staging/mahout/trunk/content/users/classification/bayesian.html (original)
+++ websites/staging/mahout/trunk/content/users/classification/bayesian.html Wed Apr 30 20:53:13 2014
@@ -262,11 +262,11 @@
 </ul>
 <p>As we can see, the main difference between Bayes and CBayes is the weight calculation step.  Where Bayes weighs terms more heavily based on the likelihood that they belong to class <code>\(c\)</code>, CBayes seeks to maximize term weights on the likelihood that they do not belong to any other class.  </p>
 <h3 id="running-from-the-command-line">Running from the command line</h3>
-<p>Mahout provides CLI drivers for all above steps.  Here we will give a simple overview of Mahout CLI commands used to preprocess the data, train the model and assign labels to the training set. An <a href="http://svn.apache.org/repos/asf/mahout/trunk/examples/bin/classify-20newsgroups.sh">example script</a> is given for the full process from data acquisition through classification of the classic <a href="https://mahout.apache.org/users/classification/twenty-newsgroups.html">20 Newsgroups corpus</a>.  </p>
+<p>Mahout provides CLI drivers for all above steps.  Here we will give a simple overview of Mahout CLI commands used to preprocess the data, train the model and assign labels to the training set. An <a href="http://svn.apache.org/repos/asf/mahout/trunk/examples/bin/classify-20newsgroups.sh">example script</a> is given for the full process from data acquisition through classification of the classic <a href="http://mahout.apache.org/users/classification/twenty-newsgroups.html">20 Newsgroups corpus</a>.  </p>
 <ul>
 <li>
 <p><strong>Preprocessing:</strong>
-For a set of Sequence File Formatted documents in PATH_TO_SEQUENCE_FILES the <a href="https://mahout.apache.org/users/basics/creating-vectors-from-text.html">mahout seq2sparse</a> command performs the TF-IDF transformations (-wt tfidf option) and L2 length normalization (-n 2 option) as follows:</p>
+For a set of Sequence File Formatted documents in PATH_TO_SEQUENCE_FILES the <a href="http://mahout.apache.org/users/basics/creating-vectors-from-text.html">mahout seq2sparse</a> command performs the TF-IDF transformations (-wt tfidf option) and L2 length normalization (-n 2 option) as follows:</p>
 <div class="codehilite"><pre>mahout seq2sparse 
   -i <span class="cp">${</span><span class="n">PATH_TO_SEQUENCE_FILES</span><span class="cp">}</span> 
   -o <span class="cp">${</span><span class="n">PATH_TO_TFIDF_VECTORS</span><span class="cp">}</span> 
@@ -307,11 +307,52 @@ Classification and testing on a holdout 
 
 </li>
 </ul>
+<h3 id="command-line-options">Command line options</h3>
+<ul>
+<li>
+<p><strong>Training</strong>:</p>
+<div class="codehilite"><pre><span class="n">mahout</span> <span class="n">trainnb</span>
+  <span class="o">--</span><span class="n">input</span> <span class="p">(</span><span class="o">-</span><span class="nb">i</span><span class="p">)</span> <span class="n">input</span>               <span class="n">Path</span> <span class="n">to</span> <span class="n">job</span> <span class="n">input</span> <span class="n">directory</span><span class="p">.</span>                 
+  <span class="o">--</span><span class="n">output</span> <span class="p">(</span><span class="o">-</span><span class="n">o</span><span class="p">)</span> <span class="n">output</span>             <span class="n">The</span> <span class="n">directory</span> <span class="n">pathname</span> <span class="k">for</span> <span class="n">output</span><span class="p">.</span>           
+  <span class="o">--</span><span class="n">labels</span> <span class="p">(</span><span class="o">-</span><span class="n">l</span><span class="p">)</span> <span class="n">labels</span>             <span class="n">comma</span><span class="o">-</span><span class="n">separated</span> <span class="n">list</span> <span class="n">of</span> <span class="n">labels</span> <span class="n">to</span> <span class="n">include</span> <span class="n">in</span> 
+                                     <span class="n">training</span>                                     
+  <span class="o">--</span><span class="n">extractLabels</span> <span class="p">(</span><span class="o">-</span><span class="n">el</span><span class="p">)</span>            <span class="n">Extract</span> <span class="n">the</span> <span class="n">labels</span> <span class="n">from</span> <span class="n">the</span> <span class="n">input</span>            
+  <span class="o">--</span><span class="n">trainComplementary</span> <span class="p">(</span><span class="o">-</span><span class="n">c</span><span class="p">)</span>        <span class="n">train</span> <span class="n">complementary</span>?                         
+  <span class="o">--</span><span class="n">labelIndex</span> <span class="p">(</span><span class="o">-</span><span class="n">li</span><span class="p">)</span> <span class="n">labelIndex</span>    <span class="n">The</span> <span class="n">path</span> <span class="n">to</span> <span class="n">store</span> <span class="n">the</span> <span class="n">label</span> <span class="n">index</span> <span class="n">in</span>         
+  <span class="o">--</span><span class="n">overwrite</span> <span class="p">(</span><span class="o">-</span><span class="n">ow</span><span class="p">)</span>                <span class="n">If</span> <span class="n">present</span><span class="p">,</span> <span class="n">overwrite</span> <span class="n">the</span> <span class="n">output</span> <span class="n">directory</span>   
+                                     <span class="n">before</span> <span class="n">running</span> <span class="n">job</span>                           
+  <span class="o">--</span><span class="n">help</span> <span class="p">(</span><span class="o">-</span><span class="n">h</span><span class="p">)</span>                      <span class="n">Print</span> <span class="n">out</span> <span class="n">help</span>                               
+  <span class="o">--</span><span class="n">tempDir</span> <span class="n">tempDir</span>                <span class="n">Intermediate</span> <span class="n">output</span> <span class="n">directory</span>                
+  <span class="o">--</span><span class="n">startPhase</span> <span class="n">startPhase</span>          <span class="n">First</span> <span class="n">phase</span> <span class="n">to</span> <span class="n">run</span>                           
+  <span class="o">--</span><span class="n">endPhase</span> <span class="n">endPhase</span>              <span class="n">Last</span> <span class="n">phase</span> <span class="n">to</span> <span class="n">run</span>
+</pre></div>
+
+
+</li>
+<li>
+<p><strong>Testing</strong></p>
+<div class="codehilite"><pre><span class="n">mahout</span> <span class="n">testnb</span>   
+  <span class="o">--</span><span class="n">input</span> <span class="p">(</span><span class="o">-</span><span class="nb">i</span><span class="p">)</span> <span class="n">input</span>              <span class="n">Path</span> <span class="n">to</span> <span class="n">job</span> <span class="n">input</span> <span class="n">directory</span><span class="p">.</span>                  
+  <span class="o">--</span><span class="n">output</span> <span class="p">(</span><span class="o">-</span><span class="n">o</span><span class="p">)</span> <span class="n">output</span>            <span class="n">The</span> <span class="n">directory</span> <span class="n">pathname</span> <span class="k">for</span> <span class="n">output</span><span class="p">.</span>            
+  <span class="o">--</span><span class="n">overwrite</span> <span class="p">(</span><span class="o">-</span><span class="n">ow</span><span class="p">)</span>               <span class="n">If</span> <span class="n">present</span><span class="p">,</span> <span class="n">overwrite</span> <span class="n">the</span> <span class="n">output</span> <span class="n">directory</span>    
+                                    <span class="n">before</span> <span class="n">running</span> <span class="n">job</span>                                                      
+  <span class="o">--</span><span class="n">model</span> <span class="p">(</span><span class="o">-</span><span class="n">m</span><span class="p">)</span> <span class="n">model</span>              <span class="n">The</span> <span class="n">path</span> <span class="n">to</span> <span class="n">the</span> <span class="n">model</span> <span class="n">built</span> <span class="n">during</span> <span class="n">training</span>   
+  <span class="o">--</span><span class="n">testComplementary</span> <span class="p">(</span><span class="o">-</span><span class="n">c</span><span class="p">)</span>        <span class="n">test</span> <span class="n">complementary</span>?                           
+  <span class="o">--</span><span class="n">runSequential</span> <span class="p">(</span><span class="o">-</span><span class="n">seq</span><span class="p">)</span>          <span class="n">run</span> <span class="n">sequential</span>?                               
+  <span class="o">--</span><span class="n">labelIndex</span> <span class="p">(</span><span class="o">-</span><span class="n">l</span><span class="p">)</span> <span class="n">labelIndex</span>    <span class="n">The</span> <span class="n">path</span> <span class="n">to</span> <span class="n">the</span> <span class="n">location</span> <span class="n">of</span> <span class="n">the</span> <span class="n">label</span> <span class="n">index</span>   
+  <span class="o">--</span><span class="n">help</span> <span class="p">(</span><span class="o">-</span><span class="n">h</span><span class="p">)</span>                     <span class="n">Print</span> <span class="n">out</span> <span class="n">help</span>                                
+  <span class="o">--</span><span class="n">tempDir</span> <span class="n">tempDir</span>               <span class="n">Intermediate</span> <span class="n">output</span> <span class="n">directory</span>                 
+  <span class="o">--</span><span class="n">startPhase</span> <span class="n">startPhase</span>         <span class="n">First</span> <span class="n">phase</span> <span class="n">to</span> <span class="n">run</span>                            
+  <span class="o">--</span><span class="n">endPhase</span> <span class="n">endPhase</span>             <span class="n">Last</span> <span class="n">phase</span> <span class="n">to</span> <span class="n">run</span>
+</pre></div>
+
+
+</li>
+</ul>
 <h3 id="examples">Examples</h3>
-<p>Mahout provides 2 examples for Naive Bayes classification:</p>
+<p>Mahout provides an example for Naive Bayes classification:</p>
 <ol>
 <li><a href="twenty-newsgroups.html">Classify 20 Newsgroups</a></li>
-<li><a href="wikipedia-bayes-example.html">Wikipedia Naive Bayes Example</a></li>
 </ol>
 <h3 id="references">References</h3>
 <p>[1]: Jason D. M. Rennie, Lawerence Shih, Jamie Teevan, David Karger (2003). <a href="http://people.csail.mit.edu/jrennie/papers/icml03-nb.pdf">Tackling the Poor Assumptions of Naive Bayes Text Classifiers</a>. Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003).</p>

Modified: websites/staging/mahout/trunk/content/users/sparkbindings/ScalaSparkBindings.pdf
==============================================================================
Binary files - no diff available.