You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mahout.apache.org by bu...@apache.org on 2014/05/25 13:48:09 UTC
svn commit: r909889 - in /websites/staging/mahout/trunk/content: ./
users/classification/twenty-newsgroups.html
Author: buildbot
Date: Sun May 25 11:48:09 2014
New Revision: 909889
Log:
Staging update by buildbot for mahout
Modified:
websites/staging/mahout/trunk/content/ (props changed)
websites/staging/mahout/trunk/content/users/classification/twenty-newsgroups.html
Propchange: websites/staging/mahout/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Sun May 25 11:48:09 2014
@@ -1 +1 @@
-1597416
+1597417
Modified: websites/staging/mahout/trunk/content/users/classification/twenty-newsgroups.html
==============================================================================
--- websites/staging/mahout/trunk/content/users/classification/twenty-newsgroups.html (original)
+++ websites/staging/mahout/trunk/content/users/classification/twenty-newsgroups.html Sun May 25 11:48:09 2014
@@ -241,27 +241,29 @@
<h2 id="twenty-newsgroups-classification-example">Twenty Newsgroups Classification Example</h2>
<p><a name="TwentyNewsgroups-Introduction"></a></p>
<h2 id="introduction">Introduction</h2>
-<p>The 20 Newsgroups data set is a collection of approximately 20,000
+<p>The 20 newsgroups dataset is a collection of approximately 20,000
newsgroup documents, partitioned (nearly) evenly across 20 different
newsgroups. The 20 newsgroups collection has become a popular data set for
experiments in text applications of machine learning techniques, such as
text classification and text clustering. We will use the <a href="http://mahout.apache.org/users/classification/bayesian.html">Mahout CBayes</a>
classifier to create a model that would classify a new document into one of
-the 20 newsgroup.</p>
+the 20 newsgroups.</p>
<p><a name="TwentyNewsgroups-Prerequisites"></a></p>
<h3 id="prerequisites">Prerequisites</h3>
<ul>
<li>Mahout has been downloaded (<a href="https://mahout.apache.org/general/downloads.html">instructions here</a>)</li>
<li>Maven is available</li>
-<li>Your environment has the following variables:</li>
+<li>Your environment has the following variables:<ul>
<li><strong>HADOOP_HOME</strong> Environment variables refers to where Hadoop lives </li>
<li><strong>MAHOUT_HOME</strong> Environment variables refers to where Mahout lives</li>
</ul>
+</li>
+</ul>
<p><a name="TwentyNewsgroups-Instructionsforrunningtheexample"></a></p>
<h3 id="instructions-for-running-the-example">Instructions for running the example</h3>
<ol>
<li>
-<p>If running Hadoop in cluster mode, Start the hadoop daemons by executing the following commands</p>
+<p>If running Hadoop in cluster mode, start the hadoop daemons by executing the following commands:</p>
<div class="codehilite"><pre> $ <span class="n">cd</span> $<span class="n">HADOOP_HOME</span><span class="o">/</span><span class="n">bin</span>
$ <span class="o">./</span><span class="n">start</span><span class="o">-</span><span class="n">all</span><span class="p">.</span><span class="n">sh</span>
</pre></div>
@@ -274,7 +276,7 @@ the 20 newsgroup.</p>
</li>
<li>
-<p>In the trunk directory of mahout, compile and install mahout:</p>
+<p>In the trunk directory of Mahout, compile and install Mahout:</p>
<div class="codehilite"><pre> $ <span class="n">cd</span> $<span class="n">MAHOUT_HOME</span>
$ <span class="n">mvn</span> <span class="o">-</span><span class="n">DskipTests</span> <span class="n">clean</span> <span class="n">install</span>
</pre></div>
@@ -282,7 +284,7 @@ the 20 newsgroup.</p>
</li>
<li>
-<p>Run the <a href="http://svn.apache.org/repos/asf/mahout/trunk/examples/bin/classify-20newsgroups.sh">20 newsgroup example script</a> by executing:</p>
+<p>Run the <a href="http://svn.apache.org/repos/asf/mahout/trunk/examples/bin/classify-20newsgroups.sh">20 newsgroups example script</a> by executing:</p>
<div class="codehilite"><pre> $ <span class="o">./</span><span class="n">examples</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">classify</span><span class="o">-</span>20<span class="n">newsgroups</span><span class="p">.</span><span class="n">sh</span>
</pre></div>
@@ -301,38 +303,38 @@ the 20 newsgroup.</p>
<p>Select 1 and the the script will perform the following:</p>
<ol>
<li>Create a working directory for the dataset and all input/output.</li>
-<li>Download and extract the <em>20news-bydate.tar.gz</em> from the <a href="http://people.csail.mit.edu/jrennie/20Newsgroups/20news-bydate.tar.gz">20newsgroups dataset</a> to the working directory.</li>
-<li>Convert the full 20newsgroups dataset into a < Text, Text > sequence file. </li>
-<li>Convert and preprocesses the dataset into a < Text, VectorWritable > sequence file containing term frequencies for each document.</li>
+<li>Download and extract the <em>20news-bydate.tar.gz</em> from the <a href="http://people.csail.mit.edu/jrennie/20Newsgroups/20news-bydate.tar.gz">20 newsgroups dataset</a> to the working directory.</li>
+<li>Convert the full 20 newsgroups dataset into a < Text, Text > SequenceFile. </li>
+<li>Convert and preprocesses the dataset into a < Text, VectorWritable > SequenceFile containing term frequencies for each document.</li>
<li>Split the preprocessed dataset into training and testing sets. </li>
<li>Train the classifier.</li>
<li>Test the classifier.</li>
</ol>
-<p>Output might look like:</p>
+<p>Output should look something like:</p>
<div class="codehilite"><pre><span class="o">=======================================================</span>
Confusion Matrix
<span class="o">-------------------------------------------------------</span>
-a b c d e f g h i j k l m n o p q r s t <span class="o"><--</span>Classified as
-<span class="m">381</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">9</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">3</span> <span class="m">0</span> <span class="o">|</span> <span class="m">398</span> a <span class="o">=</span> rec.motorcycles
-<span class="m">1</span> <span class="m">284</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">6</span> <span class="m">3</span> <span class="m">11</span> <span class="m">0</span> <span class="m">66</span> <span class="m">3</span> <span class="m">0</span> <span class="m">1</span> <span class="m">6</span> <span class="m">0</span> <span class="m">4</span> <span class="m">9</span> <span class="o">|</span> <span class="m">395</span> b <span class="o">=</span> comp.windows.x
-<span class="m">2</span> <span class="m">0</span> <span class="m">339</span> <span class="m">2</span> <span class="m">0</span> <span class="m">3</span> <span class="m">5</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">1</span> <span class="m">12</span> <span class="m">1</span> <span class="m">7</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="o">|</span> <span class="m">376</span> c <span class="o">=</span> talk.politics.mideast
-<span class="m">4</span> <span class="m">0</span> <span class="m">1</span> <span class="m">327</span> <span class="m">0</span> <span class="m">2</span> <span class="m">2</span> <span class="m">0</span> <span class="m">0</span> <span class="m">2</span> <span class="m">1</span> <span class="m">1</span> <span class="m">0</span> <span class="m">5</span> <span class="m">1</span> <span class="m">4</span> <span class="m">12</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="o">|</span> <span class="m">364</span> d <span class="o">=</span> talk.politics.guns
-<span class="m">7</span> <span class="m">0</span> <span class="m">4</span> <span class="m">32</span> <span class="m">27</span> <span class="m">7</span> <span class="m">7</span> <span class="m">2</span> <span class="m">0</span> <span class="m">12</span> <span class="m">0</span> <span class="m">0</span> <span class="m">6</span> <span class="m">0</span> <span class="m">100</span> <span class="m">9</span> <span class="m">7</span> <span class="m">31</span> <span class="m">0</span> <span class="m">0</span> <span class="o">|</span> <span class="m">251</span> e <span class="o">=</span> talk.religion.misc
-<span class="m">10</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">359</span> <span class="m">2</span> <span class="m">2</span> <span class="m">0</span> <span class="m">1</span> <span class="m">3</span> <span class="m">0</span> <span class="m">1</span> <span class="m">6</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">11</span> <span class="m">0</span> <span class="o">|</span> <span class="m">396</span> f <span class="o">=</span> rec.autos
-<span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">383</span> <span class="m">9</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">3</span> <span class="m">0</span> <span class="o">|</span> <span class="m">397</span> g <span class="o">=</span> rec.sport.baseball
-<span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">9</span> <span class="m">382</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">1</span> <span class="m">1</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="o">|</span> <span class="m">399</span> h <span class="o">=</span> rec.sport.hockey
-<span class="m">2</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">4</span> <span class="m">3</span> <span class="m">0</span> <span class="m">330</span> <span class="m">4</span> <span class="m">4</span> <span class="m">0</span> <span class="m">5</span> <span class="m">12</span> <span class="m">0</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="m">12</span> <span class="m">7</span> <span class="o">|</span> <span class="m">385</span> i <span class="o">=</span> comp.sys.mac.hardware
-<span class="m">0</span> <span class="m">3</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">368</span> <span class="m">0</span> <span class="m">0</span> <span class="m">10</span> <span class="m">4</span> <span class="m">1</span> <span class="m">3</span> <span class="m">2</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="o">|</span> <span class="m">394</span> j <span class="o">=</span> sci.space
-<span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">3</span> <span class="m">1</span> <span class="m">0</span> <span class="m">27</span> <span class="m">2</span> <span class="m">291</span> <span class="m">0</span> <span class="m">11</span> <span class="m">25</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">13</span> <span class="m">18</span> <span class="o">|</span> <span class="m">392</span> k <span class="o">=</span> comp.sys.ibm.pc.hardware
-<span class="m">8</span> <span class="m">0</span> <span class="m">1</span> <span class="m">109</span> <span class="m">0</span> <span class="m">6</span> <span class="m">11</span> <span class="m">4</span> <span class="m">1</span> <span class="m">18</span> <span class="m">0</span> <span class="m">98</span> <span class="m">1</span> <span class="m">3</span> <span class="m">11</span> <span class="m">10</span> <span class="m">27</span> <span class="m">1</span> <span class="m">1</span> <span class="m">0</span> <span class="o">|</span> <span class="m">310</span> l <span class="o">=</span> talk.politics.misc
-<span class="m">0</span> <span class="m">11</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">3</span> <span class="m">6</span> <span class="m">0</span> <span class="m">10</span> <span class="m">6</span> <span class="m">11</span> <span class="m">0</span> <span class="m">299</span> <span class="m">13</span> <span class="m">0</span> <span class="m">2</span> <span class="m">13</span> <span class="m">0</span> <span class="m">7</span> <span class="m">8</span> <span class="o">|</span> <span class="m">389</span> m <span class="o">=</span> comp.graphics
-<span class="m">6</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">4</span> <span class="m">2</span> <span class="m">0</span> <span class="m">5</span> <span class="m">2</span> <span class="m">12</span> <span class="m">0</span> <span class="m">8</span> <span class="m">321</span> <span class="m">0</span> <span class="m">4</span> <span class="m">14</span> <span class="m">0</span> <span class="m">8</span> <span class="m">6</span> <span class="o">|</span> <span class="m">393</span> n <span class="o">=</span> sci.electronics
-<span class="m">2</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">4</span> <span class="m">1</span> <span class="m">0</span> <span class="m">3</span> <span class="m">1</span> <span class="m">0</span> <span class="m">3</span> <span class="m">1</span> <span class="m">372</span> <span class="m">6</span> <span class="m">0</span> <span class="m">2</span> <span class="m">1</span> <span class="m">2</span> <span class="o">|</span> <span class="m">398</span> o <span class="o">=</span> soc.religion.christian
-<span class="m">4</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">2</span> <span class="m">3</span> <span class="m">3</span> <span class="m">0</span> <span class="m">4</span> <span class="m">2</span> <span class="m">0</span> <span class="m">7</span> <span class="m">12</span> <span class="m">6</span> <span class="m">342</span> <span class="m">1</span> <span class="m">0</span> <span class="m">9</span> <span class="m">0</span> <span class="o">|</span> <span class="m">396</span> p <span class="o">=</span> sci.med
-<span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">1</span> <span class="m">4</span> <span class="m">0</span> <span class="m">3</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">8</span> <span class="m">4</span> <span class="m">0</span> <span class="m">2</span> <span class="m">369</span> <span class="m">0</span> <span class="m">1</span> <span class="m">1</span> <span class="o">|</span> <span class="m">396</span> q <span class="o">=</span> sci.crypt
-<span class="m">10</span> <span class="m">0</span> <span class="m">4</span> <span class="m">10</span> <span class="m">1</span> <span class="m">5</span> <span class="m">6</span> <span class="m">2</span> <span class="m">2</span> <span class="m">6</span> <span class="m">2</span> <span class="m">0</span> <span class="m">2</span> <span class="m">1</span> <span class="m">86</span> <span class="m">15</span> <span class="m">14</span> <span class="m">152</span> <span class="m">0</span> <span class="m">1</span> <span class="o">|</span> <span class="m">319</span> r <span class="o">=</span> alt.atheism
-<span class="m">4</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">9</span> <span class="m">1</span> <span class="m">1</span> <span class="m">8</span> <span class="m">1</span> <span class="m">12</span> <span class="m">0</span> <span class="m">3</span> <span class="m">6</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="m">0</span> <span class="m">341</span> <span class="m">2</span> <span class="o">|</span> <span class="m">390</span> s <span class="o">=</span> misc.forsale
-<span class="m">8</span> <span class="m">5</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">6</span> <span class="m">0</span> <span class="m">8</span> <span class="m">5</span> <span class="m">50</span> <span class="m">0</span> <span class="m">40</span> <span class="m">2</span> <span class="m">1</span> <span class="m">0</span> <span class="m">9</span> <span class="m">0</span> <span class="m">3</span> <span class="m">256</span> <span class="o">|</span> <span class="m">394</span> t <span class="o">=</span> comp.os.ms<span class="o">-</span>windows.misc
+ a b c d e f g h i j k l m n o p q r s t <span class="o"><--</span>Classified as
+<span class="m">381</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">9</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">3</span> <span class="m">0</span> <span class="o">|</span><span class="m">398</span> a<span class="o">=</span>rec.motorcycles
+ <span class="m">1</span> <span class="m">284</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">6</span> <span class="m">3</span> <span class="m">11</span> <span class="m">0</span> <span class="m">66</span> <span class="m">3</span> <span class="m">0</span> <span class="m">6</span> <span class="m">0</span> <span class="m">4</span> <span class="m">9</span> <span class="m">0</span> <span class="o">|</span><span class="m">395</span> b<span class="o">=</span>comp.windows.x
+ <span class="m">2</span> <span class="m">0</span> <span class="m">339</span> <span class="m">2</span> <span class="m">0</span> <span class="m">3</span> <span class="m">5</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">1</span> <span class="m">12</span> <span class="m">1</span> <span class="m">7</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="o">|</span><span class="m">376</span> c<span class="o">=</span>talk.politics.mideast
+ <span class="m">4</span> <span class="m">0</span> <span class="m">1</span> <span class="m">327</span> <span class="m">0</span> <span class="m">2</span> <span class="m">2</span> <span class="m">0</span> <span class="m">0</span> <span class="m">2</span> <span class="m">1</span> <span class="m">1</span> <span class="m">0</span> <span class="m">5</span> <span class="m">1</span> <span class="m">4</span> <span class="m">12</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="o">|</span><span class="m">364</span> d<span class="o">=</span>talk.politics.guns
+ <span class="m">7</span> <span class="m">0</span> <span class="m">4</span> <span class="m">32</span> <span class="m">27</span> <span class="m">7</span> <span class="m">7</span> <span class="m">2</span> <span class="m">0</span> <span class="m">12</span> <span class="m">0</span> <span class="m">0</span> <span class="m">6</span> <span class="m">0</span> <span class="m">100</span> <span class="m">9</span> <span class="m">7</span> <span class="m">31</span> <span class="m">0</span> <span class="m">0</span> <span class="o">|</span><span class="m">251</span> e<span class="o">=</span>talk.religion.misc
+ <span class="m">10</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">359</span> <span class="m">2</span> <span class="m">2</span> <span class="m">0</span> <span class="m">0</span> <span class="m">3</span> <span class="m">0</span> <span class="m">1</span> <span class="m">6</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">11</span> <span class="m">0</span> <span class="o">|</span><span class="m">396</span> f<span class="o">=</span>rec.autos
+ <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">383</span> <span class="m">9</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">3</span> <span class="m">0</span> <span class="m">0</span> <span class="o">|</span><span class="m">397</span> g<span class="o">=</span>rec.sport.baseball
+ <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">9</span> <span class="m">382</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">1</span> <span class="m">1</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="o">|</span><span class="m">399</span> h<span class="o">=</span>rec.sport.hockey
+ <span class="m">2</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">4</span> <span class="m">3</span> <span class="m">0</span> <span class="m">330</span> <span class="m">4</span> <span class="m">4</span> <span class="m">0</span> <span class="m">5</span> <span class="m">12</span> <span class="m">0</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="m">12</span> <span class="m">7</span> <span class="o">|</span><span class="m">385</span> i<span class="o">=</span>comp.sys.mac.hardware
+ <span class="m">0</span> <span class="m">3</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">368</span> <span class="m">0</span> <span class="m">0</span> <span class="m">10</span> <span class="m">4</span> <span class="m">1</span> <span class="m">3</span> <span class="m">2</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="o">|</span><span class="m">394</span> j<span class="o">=</span>sci.space
+ <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">3</span> <span class="m">1</span> <span class="m">0</span> <span class="m">27</span> <span class="m">2</span> <span class="m">291</span> <span class="m">0</span> <span class="m">11</span> <span class="m">25</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">13</span> <span class="m">18</span><span class="o">|</span><span class="m">392</span> k<span class="o">=</span>comp.sys.ibm.pc.hardware
+ <span class="m">8</span> <span class="m">0</span> <span class="m">1</span> <span class="m">109</span> <span class="m">0</span> <span class="m">6</span> <span class="m">11</span> <span class="m">4</span> <span class="m">1</span> <span class="m">18</span> <span class="m">0</span> <span class="m">98</span> <span class="m">1</span> <span class="m">3</span> <span class="m">11</span> <span class="m">10</span> <span class="m">27</span> <span class="m">1</span> <span class="m">1</span> <span class="m">0</span> <span class="o">|</span><span class="m">310</span> l<span class="o">=</span>talk.politics.misc
+ <span class="m">0</span> <span class="m">11</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">3</span> <span class="m">6</span> <span class="m">0</span> <span class="m">10</span> <span class="m">6</span> <span class="m">11</span> <span class="m">0</span> <span class="m">299</span> <span class="m">13</span> <span class="m">0</span> <span class="m">2</span> <span class="m">13</span> <span class="m">0</span> <span class="m">7</span> <span class="m">8</span> <span class="o">|</span><span class="m">389</span> m<span class="o">=</span>comp.graphics
+ <span class="m">6</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">0</span> <span class="m">4</span> <span class="m">2</span> <span class="m">0</span> <span class="m">5</span> <span class="m">2</span> <span class="m">12</span> <span class="m">0</span> <span class="m">8</span> <span class="m">321</span> <span class="m">0</span> <span class="m">4</span> <span class="m">14</span> <span class="m">0</span> <span class="m">8</span> <span class="m">6</span> <span class="o">|</span><span class="m">393</span> n<span class="o">=</span>sci.electronics
+ <span class="m">2</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">4</span> <span class="m">1</span> <span class="m">0</span> <span class="m">3</span> <span class="m">1</span> <span class="m">0</span> <span class="m">3</span> <span class="m">1</span> <span class="m">372</span> <span class="m">6</span> <span class="m">0</span> <span class="m">2</span> <span class="m">1</span> <span class="m">2</span> <span class="o">|</span><span class="m">398</span> o<span class="o">=</span>soc.religion.christian
+ <span class="m">4</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">2</span> <span class="m">3</span> <span class="m">3</span> <span class="m">0</span> <span class="m">4</span> <span class="m">2</span> <span class="m">0</span> <span class="m">7</span> <span class="m">12</span> <span class="m">6</span> <span class="m">342</span> <span class="m">1</span> <span class="m">0</span> <span class="m">9</span> <span class="m">0</span> <span class="o">|</span><span class="m">396</span> p<span class="o">=</span>sci.med
+ <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">1</span> <span class="m">4</span> <span class="m">0</span> <span class="m">3</span> <span class="m">0</span> <span class="m">1</span> <span class="m">0</span> <span class="m">8</span> <span class="m">4</span> <span class="m">0</span> <span class="m">2</span> <span class="m">369</span> <span class="m">0</span> <span class="m">1</span> <span class="m">1</span> <span class="o">|</span><span class="m">396</span> q<span class="o">=</span>sci.crypt
+ <span class="m">10</span> <span class="m">0</span> <span class="m">4</span> <span class="m">10</span> <span class="m">1</span> <span class="m">5</span> <span class="m">6</span> <span class="m">2</span> <span class="m">2</span> <span class="m">6</span> <span class="m">2</span> <span class="m">0</span> <span class="m">2</span> <span class="m">1</span> <span class="m">86</span> <span class="m">15</span> <span class="m">14</span> <span class="m">152</span> <span class="m">0</span> <span class="m">1</span> <span class="o">|</span><span class="m">319</span> r<span class="o">=</span>alt.atheism
+ <span class="m">4</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">9</span> <span class="m">1</span> <span class="m">1</span> <span class="m">8</span> <span class="m">1</span> <span class="m">12</span> <span class="m">0</span> <span class="m">3</span> <span class="m">0</span> <span class="m">2</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">341</span> <span class="m">2</span> <span class="o">|</span><span class="m">390</span> s<span class="o">=</span>misc.forsale
+ <span class="m">8</span> <span class="m">5</span> <span class="m">0</span> <span class="m">0</span> <span class="m">0</span> <span class="m">1</span> <span class="m">6</span> <span class="m">0</span> <span class="m">8</span> <span class="m">5</span> <span class="m">50</span> <span class="m">0</span> <span class="m">40</span> <span class="m">2</span> <span class="m">1</span> <span class="m">0</span> <span class="m">9</span> <span class="m">0</span> <span class="m">3</span> <span class="m">256</span><span class="o">|</span><span class="m">394</span> t<span class="o">=</span>comp.os.ms<span class="o">-</span>windows.misc
<span class="o">=======================================================</span>
Statistics
<span class="o">-------------------------------------------------------</span>
@@ -344,8 +346,8 @@ Reliability <span class="p">(</span>stan
<p><a name="TwentyNewsgroups-ComplementaryNaiveBayes"></a></p>
-<h2 id="end-to-end-commands-to-build-a-cbayes-model-for-20-newsgroups">End to end commands to build a CBayes model for 20 Newsgroups:</h2>
-<p>The <a href="http://svn.apache.org/repos/asf/mahout/trunk/examples/bin/classify-20newsgroups.sh">20 newsgroup example script</a> issues the following commands as outlined above. We can build a CBayes classifier from the command line by following the process in the script: </p>
+<h2 id="end-to-end-commands-to-build-a-cbayes-model-for-20-newsgroups">End to end commands to build a CBayes model for 20 newsgroups</h2>
+<p>The <a href="http://svn.apache.org/repos/asf/mahout/trunk/examples/bin/classify-20newsgroups.sh">20 newsgroups example script</a> issues the following commands as outlined above. We can build a CBayes classifier from the command line by following the process in the script: </p>
<p><em>Be sure that <strong>MAHOUT_HOME</strong>/bin and <strong>HADOOP_HOME</strong>/bin are in your <strong>$PATH</strong></em></p>
<ol>
<li>
@@ -369,7 +371,7 @@ Reliability <span class="p">(</span>stan
<ul>
<li>
-<p>If you're running on a hadoop cluster</p>
+<p>If you're running on a Hadoop cluster:</p>
<div class="codehilite"><pre>$ hadoop dfs -put <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/20news-all <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/20news-all
</pre></div>
@@ -378,16 +380,17 @@ Reliability <span class="p">(</span>stan
</ul>
</li>
<li>
-<p>Convert the full 20newsgroups dataset into a < Text, Text > sequence file. </p>
+<p>Convert the full 20 newsgroups dataset into a < Text, Text > SequenceFile. </p>
<div class="codehilite"><pre> $ mahout seqdirectory
-i <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/20news-all
- -o <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/20news-seq -ow
+ -o <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/20news-seq
+ -ow
</pre></div>
</li>
<li>
-<p>Convert and preprocesses the dataset into a < Text, VectorWritable > sequence file containing term frequencies for each document. </p>
+<p>Convert and preprocesses the dataset into a < Text, VectorWritable > SequenceFile containing term frequencies for each document. </p>
<div class="codehilite"><pre> $ mahout seq2sparse
-i <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/20news-seq
-o <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/20news-vectors
@@ -397,7 +400,7 @@ Reliability <span class="p">(</span>stan
</pre></div>
-<p>If we wanted to use different parsing methods or transformations on the term frequency vectors we could supply different options here e.g.: -ng 2 for bi-grams or -n 2 for L2 length normalization. See the <a href="http://mahout.apache.org/users/basics/creating-vectors-from-text.html">Creating vectors from text</a> for a list of all se2sparse options. </p>
+<p>If we wanted to use different parsing methods or transformations on the term frequency vectors we could supply different options here e.g.: -ng 2 for bigrams or -n 2 for L2 length normalization. See the <a href="http://mahout.apache.org/users/basics/creating-vectors-from-text.html">Creating vectors from text</a> page for a list of all seq2sparse options. </p>
</li>
<li>
<p>Split the preprocessed dataset into training and testing sets.</p>
@@ -414,7 +417,8 @@ Reliability <span class="p">(</span>stan
<li>
<p>Train the classifier.</p>
<div class="codehilite"><pre> $ mahout trainnb
- -i <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/20news-train-vectors -el
+ -i <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/20news-train-vectors
+ -el
-o <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/model
-li <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/labelindex
-ow
@@ -425,7 +429,7 @@ Reliability <span class="p">(</span>stan
</li>
<li>
<p>Test the classifier.</p>
-<div class="codehilite"><pre> $ mahhout testnb
+<div class="codehilite"><pre> $ mahout testnb
-i <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/20news-test-vectors
-m <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/model
-l <span class="cp">${</span><span class="n">WORK_DIR</span><span class="cp">}</span>/labelindex