You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mahout.apache.org by ra...@apache.org on 2019/03/07 22:14:59 UTC
[mahout] branch asf-site updated: Version bump 0.13.1
This is an automated email from the ASF dual-hosted git repository.
rawkintrevo pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/mahout.git
The following commit(s) were added to refs/heads/asf-site by this push:
new d49c053 Version bump 0.13.1
d49c053 is described below
commit d49c053656f792f352b269b7f726a11d375daa97
Author: Trevor a.k.a @rawkintrevo <tr...@gmail.com>
AuthorDate: Thu Mar 7 16:14:12 2019 -0600
Version bump 0.13.1
---
docs/0.13.1/api/docs/404.html | 1 +
docs/0.13.1/api/docs/Gemfile | 5 +
docs/0.13.1/api/docs/Gemfile.lock | 60 +
.../docs/algorithms/clustering/canopy/Canopy.png | Bin 0 -> 48061 bytes
.../docs/algorithms/clustering/canopy/Canopy10.png | Bin 0 -> 42414 bytes
.../algorithms/clustering/canopy/SampleData.png | Bin 0 -> 32587 bytes
.../docs/algorithms/clustering/canopy/index.html | 318 +
.../algorithms/clustering/distance-metrics.html | 269 +
.../api/docs/algorithms/clustering/index.html | 194 +
.../api/docs/algorithms/linear-algebra/d-qr.html | 244 +
.../api/docs/algorithms/linear-algebra/d-spca.html | 365 +
.../api/docs/algorithms/linear-algebra/d-ssvd.html | 330 +
.../api/docs/algorithms/linear-algebra/index.html | 199 +
.../map-reduce/classification/bayesian.html | 365 +
.../map-reduce/classification/class-discovery.html | 343 +
.../classification/classifyingyourdata.html | 212 +
.../map-reduce/classification/collocations.html | 567 ++
.../gaussian-discriminative-analysis.html | 204 +
.../classification/hidden-markov-models.html | 299 +
.../independent-component-analysis.html | 201 +
.../locally-weighted-linear-regression.html | 209 +
.../classification/logistic-regression.html | 319 +
.../classification/mahout-collections.html | 239 +
.../algorithms/map-reduce/classification/mlp.html | 430 ++
.../map-reduce/classification/naivebayes.html | 230 +
.../map-reduce/classification/neural-network.html | 205 +
.../classification/partial-implementation.html | 333 +
.../map-reduce/classification/random-forests.html | 427 ++
.../restricted-boltzmann-machines.html | 237 +
.../classification/support-vector-machines.html | 227 +
.../map-reduce/clustering/canopy-clustering.html | 382 ++
.../map-reduce/clustering/cluster-dumper.html | 299 +
.../clustering/expectation-maximization.html | 247 +
.../map-reduce/clustering/fuzzy-k-means.html | 380 ++
.../clustering/hierarchical-clustering.html | 200 +
.../map-reduce/clustering/k-means-clustering.html | 371 +
.../clustering/latent-dirichlet-allocation.html | 337 +
.../clustering/llr---log-likelihood-ratio.html | 231 +
.../map-reduce/clustering/spectral-clustering.html | 281 +
.../map-reduce/clustering/streaming-k-means.html | 376 ++
.../api/docs/algorithms/map-reduce/index.html | 226 +
.../docs/algorithms/preprocessors/AsFactor.html | 217 +
.../docs/algorithms/preprocessors/MeanCenter.html | 212 +
.../algorithms/preprocessors/StandardScaler.html | 226 +
.../api/docs/algorithms/preprocessors/index.html | 197 +
.../api/docs/algorithms/reccomenders/cco.html | 657 ++
.../api/docs/algorithms/reccomenders/d-als.html | 239 +
.../api/docs/algorithms/reccomenders/index.html | 218 +
.../api/docs/algorithms/recommenders/cco.html | 662 ++
.../api/docs/algorithms/recommenders/d-als.html | 239 +
.../api/docs/algorithms/recommenders/index.html | 226 +
.../docs/algorithms/regression/fittness-tests.html | 200 +
.../api/docs/algorithms/regression/index.html | 206 +
.../0.13.1/api/docs/algorithms/regression/ols.html | 252 +
.../serial-correlation/cochrane-orcutt.html | 334 +
.../regression/serial-correlation/dw-test.html | 241 +
docs/0.13.1/api/docs/algorithms/template.html | 200 +
docs/0.13.1/api/docs/assets/img/2dDirichletASN.png | Bin 0 -> 61964 bytes
.../api/docs/assets/img/2dDirichletASN4040.png | Bin 0 -> 80547 bytes
docs/0.13.1/api/docs/assets/img/2dDirichletSN.png | Bin 0 -> 70586 bytes
docs/0.13.1/api/docs/assets/img/2dFuzzyKMeans.png | Bin 0 -> 59029 bytes
docs/0.13.1/api/docs/assets/img/2dKMeans.png | Bin 0 -> 56553 bytes
docs/0.13.1/api/docs/assets/img/2dMeanShift.png | Bin 0 -> 42083 bytes
.../api/docs/assets/img/AsymmetricSampleData.png | Bin 0 -> 37665 bytes
docs/0.13.1/api/docs/assets/img/Canopy.png | Bin 0 -> 48061 bytes
docs/0.13.1/api/docs/assets/img/Canopy10.png | Bin 0 -> 42414 bytes
docs/0.13.1/api/docs/assets/img/DirichletASN.png | Bin 0 -> 50460 bytes
docs/0.13.1/api/docs/assets/img/DirichletN.png | Bin 0 -> 85831 bytes
docs/0.13.1/api/docs/assets/img/DirichletSN.png | Bin 0 -> 75029 bytes
docs/0.13.1/api/docs/assets/img/DirichletSN40.png | Bin 0 -> 84339 bytes
...lementation of k-Means provided with Mahout.png | Bin 0 -> 244837 bytes
docs/0.13.1/api/docs/assets/img/FuzzyKMeans.png | Bin 0 -> 50444 bytes
docs/0.13.1/api/docs/assets/img/KMeans.png | Bin 0 -> 62848 bytes
.../api/docs/assets/img/Mahout-logo-164x200.png | Bin 0 -> 10577 bytes
.../api/docs/assets/img/Mahout-logo-245x300.png | Bin 0 -> 16214 bytes
.../api/docs/assets/img/Mahout-logo-327x400.png | Bin 0 -> 22115 bytes
.../api/docs/assets/img/Mahout-logo-82x100.png | Bin 0 -> 5140 bytes
docs/0.13.1/api/docs/assets/img/MeanShift.png | Bin 0 -> 40236 bytes
docs/0.13.1/api/docs/assets/img/SampleData.png | Bin 0 -> 32587 bytes
docs/0.13.1/api/docs/assets/img/bg.png | Bin 0 -> 3894 bytes
docs/0.13.1/api/docs/assets/img/debug-config-2.png | Bin 0 -> 118953 bytes
docs/0.13.1/api/docs/assets/img/debug-config.png | Bin 0 -> 48956 bytes
.../0.13.1/api/docs/assets/img/download-mahout.png | Bin 0 -> 10154 bytes
.../api/docs/assets/img/download-original.png | Bin 0 -> 56689 bytes
docs/0.13.1/api/docs/assets/img/download.png | Bin 0 -> 6151 bytes
.../docs/assets/img/flink_squirrel_100_color.png | Bin 0 -> 16446 bytes
docs/0.13.1/api/docs/assets/img/generic page.png | Bin 0 -> 306779 bytes
.../api/docs/assets/img/home-wrapper-original.png | Bin 0 -> 435604 bytes
.../0.13.1/api/docs/assets/img/home-wrapper-v2.png | Bin 0 -> 59828 bytes
.../0.13.1/api/docs/assets/img/home-wrapper-v3.png | Bin 0 -> 111463 bytes
docs/0.13.1/api/docs/assets/img/home-wrapper.png | Bin 0 -> 57264 bytes
.../api/docs/assets/img/instruction_arrow.png | Bin 0 -> 285 bytes
docs/0.13.1/api/docs/assets/img/logo.png | Bin 0 -> 25615 bytes
.../0.13.1/api/docs/assets/img/mahout-logo-100.png | Bin 0 -> 19477 bytes
.../0.13.1/api/docs/assets/img/mahout-logo-200.png | Bin 0 -> 46360 bytes
.../0.13.1/api/docs/assets/img/mahout-logo-300.png | Bin 0 -> 70139 bytes
.../0.13.1/api/docs/assets/img/mahout-logo-400.png | Bin 0 -> 55468 bytes
.../api/docs/assets/img/mahout-logo-brudman.png | Bin 0 -> 59233 bytes
.../docs/assets/img/mahout-logo-poweredby-100.png | Bin 0 -> 24623 bytes
.../docs/assets/img/mahout-logo-poweredby-55.png | Bin 0 -> 11684 bytes
.../api/docs/assets/img/mahout-logo-poweredby.png | Bin 0 -> 75777 bytes
.../assets/img/mahout-logo-transparent-400.png | Bin 0 -> 61970 bytes
docs/0.13.1/api/docs/assets/img/mahout-logo.png | Bin 0 -> 79779 bytes
docs/0.13.1/api/docs/assets/img/mahout-lupe.png | Bin 0 -> 479 bytes
docs/0.13.1/api/docs/assets/img/mahout2_theme.png | Bin 0 -> 285686 bytes
docs/0.13.1/api/docs/assets/img/mahout_theme.png | Bin 0 -> 271601 bytes
.../api/docs/assets/img/mantle-1-original.png | Bin 0 -> 183285 bytes
docs/0.13.1/api/docs/assets/img/mantle-asf.png | Bin 0 -> 43710 bytes
.../api/docs/assets/img/mantle-community.png | Bin 0 -> 47094 bytes
docs/0.13.1/api/docs/assets/img/mantle-hadoop.png | Bin 0 -> 34392 bytes
docs/0.13.1/api/docs/assets/img/mantle-mahout.png | Bin 0 -> 32609 bytes
docs/0.13.1/api/docs/assets/img/screen.png | Bin 0 -> 267592 bytes
.../api/docs/assets/img/sgd-class-hierarchy.png | Bin 0 -> 259654 bytes
.../api/docs/assets/img/sidebar-original.png | Bin 0 -> 48060 bytes
docs/0.13.1/api/docs/assets/img/sidebar.png | Bin 0 -> 401 bytes
.../api/docs/assets/img/taste-architecture.png | Bin 0 -> 101546 bytes
.../api/docs/assets/img/vector-class-hierarchy.png | Bin 0 -> 142705 bytes
docs/0.13.1/api/docs/assets/img/wiki-bg.png | Bin 0 -> 56041 bytes
.../api/docs/assets/img/wiki-wrapper-original.png | Bin 0 -> 398249 bytes
docs/0.13.1/api/docs/assets/img/wiki-wrapper.png | Bin 0 -> 59718 bytes
.../mahout-retro/css/bootstrap-responsive.css | 1109 +++
.../mahout-retro/css/bootstrap-responsive.css.bu | 1109 +++
.../mahout-retro/css/bootstrap-responsive.min.css | 9 +
.../assets/themes/mahout-retro/css/bootstrap.css | 6165 +++++++++++++++++
.../themes/mahout-retro/css/bootstrap.min.css | 9 +
.../docs/assets/themes/mahout-retro/css/global.css | 938 +++
.../assets/themes/mahout-retro/css/global__.css | 886 +++
.../docs/assets/themes/mahout-retro/css/main.css | 4 +
.../assets/themes/mahout-retro/js/bootstrap.js | 2276 +++++++
.../assets/themes/mahout-retro/js/bootstrap.min.js | 6 +
.../docs/assets/themes/mahout-retro/js/effects.js | 1130 ++++
.../themes/mahout-retro/js/jquery-1.9.1.min.js | 5 +
.../assets/themes/mahout-retro/js/prototype.js | 4320 ++++++++++++
.../docs/assets/themes/mahout-retro/js/search.js | 21 +
.../docs/assets/themes/mahout-retro/js/slides.js | 109 +
.../docs/assets/themes/mahout-retro/js/widgets.js | 45 +
.../mahout/bootstrap/css/bootstrap-theme.css | 459 ++
.../mahout/bootstrap/css/bootstrap-theme.min.css | 9 +
.../themes/mahout/bootstrap/css/bootstrap.css | 7100 ++++++++++++++++++++
.../fonts/glyphicons-halflings-regular.eot | Bin 0 -> 20290 bytes
.../fonts/glyphicons-halflings-regular.svg | 229 +
.../fonts/glyphicons-halflings-regular.ttf | Bin 0 -> 41236 bytes
.../fonts/glyphicons-halflings-regular.woff | Bin 0 -> 23292 bytes
.../bootstrap/img/glyphicons-halflings-white.png | Bin 0 -> 8777 bytes
.../mahout/bootstrap/img/glyphicons-halflings.png | Bin 0 -> 12799 bytes
.../assets/themes/mahout/bootstrap/js/bootstrap.js | 2002 ++++++
.../themes/mahout/bootstrap/js/bootstrap.min.js | 9 +
.../api/docs/assets/themes/mahout/css/style.css | 1128 ++++
.../api/docs/assets/themes/mahout/css/syntax.css | 60 +
.../docs/assets/themes/mahout/imgx/asf_logo.png | Bin 0 -> 21243 bytes
.../themes/mahout/imgx/available_interpreters.png | Bin 0 -> 251135 bytes
.../api/docs/assets/themes/mahout/imgx/body-bg.jpg | Bin 0 -> 3188 bytes
.../api/docs/assets/themes/mahout/imgx/graph1.png | Bin 0 -> 19462 bytes
.../api/docs/assets/themes/mahout/imgx/graph2.png | Bin 0 -> 22596 bytes
.../docs/assets/themes/mahout/imgx/header-bg.jpg | Bin 0 -> 10442 bytes
.../api/docs/assets/themes/mahout/imgx/helium.gif | Bin 0 -> 221381 bytes
.../assets/themes/mahout/imgx/highlight-bg.jpg | Bin 0 -> 34222 bytes
.../docs/assets/themes/mahout/imgx/interpreter.png | Bin 0 -> 36035 bytes
.../themes/mahout/imgx/maven_default_icon.png | Bin 0 -> 4248 bytes
.../docs/assets/themes/mahout/imgx/notebook.png | Bin 0 -> 194904 bytes
.../docs/assets/themes/mahout/imgx/personalize.gif | Bin 0 -> 508468 bytes
.../api/docs/assets/themes/mahout/imgx/scope.gif | Bin 0 -> 206852 bytes
.../mahout/imgx/screenshots/collaboration.png | Bin 0 -> 10901 bytes
.../mahout/imgx/screenshots/display_angular.png | Bin 0 -> 69621 bytes
.../mahout/imgx/screenshots/display_angular1.png | Bin 0 -> 97217 bytes
.../mahout/imgx/screenshots/display_angular2.png | Bin 0 -> 94041 bytes
.../mahout/imgx/screenshots/display_angular3.png | Bin 0 -> 76267 bytes
.../mahout/imgx/screenshots/display_html.png | Bin 0 -> 28401 bytes
.../mahout/imgx/screenshots/display_image.png | Bin 0 -> 25969 bytes
.../mahout/imgx/screenshots/display_table.png | Bin 0 -> 30912 bytes
.../mahout/imgx/screenshots/display_table1.png | Bin 0 -> 41254 bytes
.../mahout/imgx/screenshots/display_table_html.png | Bin 0 -> 33133 bytes
.../mahout/imgx/screenshots/display_text.png | Bin 0 -> 21517 bytes
.../mahout/imgx/screenshots/display_text1.png | Bin 0 -> 22977 bytes
.../themes/mahout/imgx/screenshots/dynamicform.png | Bin 0 -> 119566 bytes
.../themes/mahout/imgx/screenshots/form_input.png | Bin 0 -> 16832 bytes
.../mahout/imgx/screenshots/form_input_default.png | Bin 0 -> 15299 bytes
.../imgx/screenshots/form_input_default_prog.png | Bin 0 -> 16223 bytes
.../mahout/imgx/screenshots/form_input_prog.png | Bin 0 -> 17602 bytes
.../themes/mahout/imgx/screenshots/form_select.png | Bin 0 -> 19207 bytes
.../imgx/screenshots/form_select_displayname.png | Bin 0 -> 20191 bytes
.../mahout/imgx/screenshots/form_select_prog.png | Bin 0 -> 26119 bytes
.../imgx/screenshots/homepage_notebook_id.png | Bin 0 -> 18950 bytes
.../imgx/screenshots/homepage_notebook_list.png | Bin 0 -> 35186 bytes
.../mahout/imgx/screenshots/interpreter_create.png | Bin 0 -> 10591 bytes
.../imgx/screenshots/interpreter_setting.png | Bin 0 -> 14602 bytes
.../imgx/screenshots/interpreter_setting_spark.png | Bin 0 -> 40237 bytes
.../mahout/imgx/screenshots/link_paragraph.png | Bin 0 -> 17113 bytes
.../themes/mahout/imgx/screenshots/markdown.png | Bin 0 -> 22145 bytes
.../imgx/screenshots/multiple_language_backend.png | Bin 0 -> 16491 bytes
.../themes/mahout/imgx/screenshots/notebook.png | Bin 0 -> 64398 bytes
.../themes/mahout/imgx/screenshots/pivot.png | Bin 0 -> 76003 bytes
.../themes/mahout/imgx/screenshots/publish.png | Bin 0 -> 548918 bytes
.../themes/mahout/imgx/screenshots/spark.png | Bin 0 -> 48215 bytes
.../themes/mahout/imgx/screenshots/sparksql.png | Bin 0 -> 36004 bytes
.../mahout/imgx/screenshots/visualization.png | Bin 0 -> 39545 bytes
.../docs/assets/themes/mahout/imgx/sidebar-bg.jpg | Bin 0 -> 3200 bytes
.../docs/assets/themes/mahout/imgx/spark_logo.jpg | Bin 0 -> 25886 bytes
.../themes/mahout/imgx/user-impersonation.gif | Bin 0 -> 913552 bytes
.../themes/mahout/imgx/zeppelin_classic_logo.png | Bin 0 -> 24056 bytes
.../assets/themes/mahout/imgx/zeppelin_logo.png | Bin 0 -> 16393 bytes
.../themes/mahout/imgx/zeppelin_svg_logo.svg | 77 +
.../api/docs/assets/themes/mahout/js/anchor.min.js | 6 +
.../api/docs/assets/themes/mahout/js/docs.js | 42 +
.../assets/themes/mahout/js/helium.controller.js | 89 +
.../assets/themes/mahout/js/medium.controller.js | 47 +
.../api/docs/assets/themes/mahout/js/moment.min.js | 551 ++
.../themes/mahout2/css/bootstrap-theme.min.css | 7 +
.../assets/themes/mahout2/css/bootstrap.min.css | 7 +
.../assets/themes/mahout2/css/bs-sticky-footer.css | 29 +
.../api/docs/assets/themes/mahout2/css/style.css | 126 +
.../mahout2/fonts/glyphicons-halflings-regular.eot | Bin 0 -> 20290 bytes
.../mahout2/fonts/glyphicons-halflings-regular.svg | 229 +
.../mahout2/fonts/glyphicons-halflings-regular.ttf | Bin 0 -> 41236 bytes
.../fonts/glyphicons-halflings-regular.woff | Bin 0 -> 23292 bytes
.../docs/assets/themes/mahout2/js/bootstrap.min.js | 7 +
.../themes/mahout3/css/bootstrap-theme.min.css | 7 +
.../assets/themes/mahout3/css/bootstrap.min.css | 7 +
.../assets/themes/mahout3/css/bs-sticky-footer.css | 29 +
.../api/docs/assets/themes/mahout3/css/style.css | 161 +
.../mahout3/fonts/glyphicons-halflings-regular.eot | 0
.../mahout3/fonts/glyphicons-halflings-regular.svg | 229 +
.../mahout3/fonts/glyphicons-halflings-regular.ttf | 0
.../fonts/glyphicons-halflings-regular.woff | 0
.../docs/assets/themes/mahout3/js/bootstrap.min.js | 17 +
docs/0.13.1/api/docs/changelog.md | 70 +
.../api/docs/distributed/flink-bindings.html | 233 +
.../0.13.1/api/docs/distributed/h2o-internals.html | 232 +
.../MahoutScalaAndSparkBindings.pptx | Bin 0 -> 846177 bytes
.../spark-bindings/ScalaSparkBindings.pdf | 6215 +++++++++++++++++
.../api/docs/distributed/spark-bindings/faq.html | 237 +
.../api/docs/distributed/spark-bindings/index.html | 288 +
docs/0.13.1/api/docs/index.html | 304 +
docs/0.13.1/api/docs/mahout-samsara/faq.html | 237 +
.../api/docs/mahout-samsara/in-core-reference.html | 593 ++
.../docs/mahout-samsara/out-of-core-reference.html | 566 ++
docs/0.13.1/api/docs/native-solvers/cuda.html | 192 +
.../api/docs/native-solvers/viennacl-omp.html | 192 +
docs/0.13.1/api/docs/native-solvers/viennacl.html | 192 +
docs/0.13.1/api/docs/quickstart.html | 246 +
docs/0.13.1/api/docs/robots.txt | 1 +
docs/0.13.1/api/docs/sitemap.xml | 304 +
.../api/docs/tutorials/cco-lastfm/cco-lastfm.scala | 112 +
.../api/docs/tutorials/cco-lastfm/index.html | 344 +
.../api/docs/tutorials/eigenfaces/eigenfaces.png | Bin 0 -> 355453 bytes
.../api/docs/tutorials/eigenfaces/index.html | 317 +
.../tutorials/intro-cooccurrence-spark/index.html | 668 ++
.../classification/bankmarketing-example.html | 239 +
.../map-reduce/classification/breiman-example.html | 258 +
.../classification/twenty-newsgroups.html | 402 ++
.../wikipedia-classifier-example.html | 245 +
.../map-reduce/clustering/20newsgroups.html | 195 +
.../map-reduce/clustering/canopy-commandline.html | 263 +
.../clustering-of-synthetic-control-data.html | 253 +
.../clustering/clustering-seinfeld-episodes.html | 195 +
.../map-reduce/clustering/clusteringyourdata.html | 319 +
.../clustering/fuzzy-k-means-commandline.html | 289 +
.../map-reduce/clustering/k-means-commandline.html | 285 +
.../map-reduce/clustering/lda-commandline.html | 275 +
.../map-reduce/clustering/viewing-result.html | 204 +
.../map-reduce/clustering/viewing-results.html | 231 +
.../clustering/visualizing-sample-clusters.html | 237 +
.../api/docs/tutorials/map-reduce/index.html | 202 +
.../tutorials/map-reduce/misc/mr---map-reduce.html | 204 +
.../misc/parallel-frequent-pattern-mining.html | 372 +
.../map-reduce/misc/perceptron-and-winnow.html | 226 +
.../docs/tutorials/map-reduce/misc/testing.html | 235 +
.../misc/using-mahout-with-python-via-jpype.html | 404 ++
.../map-reduce/recommender/intro-als-hadoop.html | 288 +
.../recommender/intro-cooccurrence-spark.html | 659 ++
.../recommender/intro-itembased-hadoop.html | 245 +
.../recommender/matrix-factorization.html | 405 ++
.../map-reduce/recommender/quickstart.html | 218 +
.../recommender/recommender-documentation.html | 466 ++
.../recommender/recommender-first-timer-faq.html | 238 +
.../recommender/userbased-5-minutes.html | 316 +
.../tutorials/misc/contributing-algos/Foo.scala | 75 +
.../contributing-algos/RegressionSuiteBase.scala | 202 +
.../misc/contributing-algos/create-pr.png | Bin 0 -> 27568 bytes
.../misc/contributing-algos/github-branch.png | Bin 0 -> 87923 bytes
.../tutorials/misc/contributing-algos/index.html | 650 ++
.../tutorials/misc/contributing-algos/jira.png | Bin 0 -> 30685 bytes
.../tutorials/misc/contributing-algos/new-jira.png | Bin 0 -> 75111 bytes
.../docs/tutorials/misc/how-to-build-an-app.html | 470 ++
.../tutorials/misc/mahout-in-zeppelin/index.html | 468 ++
.../misc/mahout-in-zeppelin/zeppelin1.png | Bin 0 -> 50936 bytes
.../misc/mahout-in-zeppelin/zeppelin2.png | Bin 0 -> 46906 bytes
.../misc/mahout-in-zeppelin/zeppelin3.png | Bin 0 -> 68551 bytes
.../samsara/classify-a-doc-from-the-shell.html | 478 ++
.../docs/tutorials/samsara/play-with-shell.html | 462 ++
.../samsara/playing-with-samsara-flink-batch.html | 308 +
.../docs/tutorials/samsara/spark-naive-bayes.html | 349 +
292 files changed, 69065 insertions(+)
diff --git a/docs/0.13.1/api/docs/404.html b/docs/0.13.1/api/docs/404.html
new file mode 100755
index 0000000..6904bcd
--- /dev/null
+++ b/docs/0.13.1/api/docs/404.html
@@ -0,0 +1 @@
+Sorry this page does not exist =(
diff --git a/docs/0.13.1/api/docs/Gemfile b/docs/0.13.1/api/docs/Gemfile
new file mode 100755
index 0000000..301d29c
--- /dev/null
+++ b/docs/0.13.1/api/docs/Gemfile
@@ -0,0 +1,5 @@
+source "https://rubygems.org"
+
+gem "jekyll", "~> 3.1"
+gem "jekyll-sitemap"
+gem "pygments.rb"
diff --git a/docs/0.13.1/api/docs/Gemfile.lock b/docs/0.13.1/api/docs/Gemfile.lock
new file mode 100644
index 0000000..32b4e59
--- /dev/null
+++ b/docs/0.13.1/api/docs/Gemfile.lock
@@ -0,0 +1,60 @@
+GEM
+ remote: https://rubygems.org/
+ specs:
+ addressable (2.5.2)
+ public_suffix (>= 2.0.2, < 4.0)
+ colorator (1.1.0)
+ ffi (1.9.18)
+ forwardable-extended (2.6.0)
+ jekyll (3.6.2)
+ addressable (~> 2.4)
+ colorator (~> 1.0)
+ jekyll-sass-converter (~> 1.0)
+ jekyll-watch (~> 1.1)
+ kramdown (~> 1.14)
+ liquid (~> 4.0)
+ mercenary (~> 0.3.3)
+ pathutil (~> 0.9)
+ rouge (>= 1.7, < 3)
+ safe_yaml (~> 1.0)
+ jekyll-sass-converter (1.5.1)
+ sass (~> 3.4)
+ jekyll-sitemap (1.1.1)
+ jekyll (~> 3.3)
+ jekyll-watch (1.5.1)
+ listen (~> 3.0)
+ kramdown (1.16.2)
+ liquid (4.0.0)
+ listen (3.1.5)
+ rb-fsevent (~> 0.9, >= 0.9.4)
+ rb-inotify (~> 0.9, >= 0.9.7)
+ ruby_dep (~> 1.2)
+ mercenary (0.3.6)
+ multi_json (1.12.2)
+ pathutil (0.16.1)
+ forwardable-extended (~> 2.6)
+ public_suffix (3.0.1)
+ pygments.rb (1.2.1)
+ multi_json (>= 1.0.0)
+ rb-fsevent (0.10.2)
+ rb-inotify (0.9.10)
+ ffi (>= 0.5.0, < 2)
+ rouge (2.2.1)
+ ruby_dep (1.5.0)
+ safe_yaml (1.0.4)
+ sass (3.5.4)
+ sass-listen (~> 4.0.0)
+ sass-listen (4.0.0)
+ rb-fsevent (~> 0.9, >= 0.9.4)
+ rb-inotify (~> 0.9, >= 0.9.7)
+
+PLATFORMS
+ ruby
+
+DEPENDENCIES
+ jekyll (~> 3.1)
+ jekyll-sitemap
+ pygments.rb
+
+BUNDLED WITH
+ 1.16.0
diff --git a/docs/0.13.1/api/docs/algorithms/clustering/canopy/Canopy.png b/docs/0.13.1/api/docs/algorithms/clustering/canopy/Canopy.png
new file mode 100644
index 0000000..934efd7
Binary files /dev/null and b/docs/0.13.1/api/docs/algorithms/clustering/canopy/Canopy.png differ
diff --git a/docs/0.13.1/api/docs/algorithms/clustering/canopy/Canopy10.png b/docs/0.13.1/api/docs/algorithms/clustering/canopy/Canopy10.png
new file mode 100644
index 0000000..4bb291c
Binary files /dev/null and b/docs/0.13.1/api/docs/algorithms/clustering/canopy/Canopy10.png differ
diff --git a/docs/0.13.1/api/docs/algorithms/clustering/canopy/SampleData.png b/docs/0.13.1/api/docs/algorithms/clustering/canopy/SampleData.png
new file mode 100644
index 0000000..48c4690
Binary files /dev/null and b/docs/0.13.1/api/docs/algorithms/clustering/canopy/SampleData.png differ
diff --git a/docs/0.13.1/api/docs/algorithms/clustering/canopy/index.html b/docs/0.13.1/api/docs/algorithms/clustering/canopy/index.html
new file mode 100644
index 0000000..481a300
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/clustering/canopy/index.html
@@ -0,0 +1,318 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ Canopy Clustering
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/clustering/canopy/">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h3 id="about">About</h3>
+
+<p><a href="http://www.kamalnigam.com/papers/canopy-kdd00.pdf">Canopy Clustering</a>
+ is a very simple, fast and surprisingly accurate method for grouping
+objects into clusters. All objects are represented as a point in a
+multidimensional feature space. The algorithm uses a fast approximate
+distance metric and two distance thresholds T1 > T2 for processing. The
+basic algorithm is to begin with a set of points and remove one at random.
+Create a Canopy containing this point and iterate through the remainder of
+the point set. At each point, if its distance from the first point is < T1,
+then add the point to the cluster. If, in addition, the distance is < T2,
+then remove the point from the set. This way points that are very close to
+the original will avoid all further processing. The algorithm loops until
+the initial set is empty, accumulating a set of Canopies, each containing
+one or more points. A given point may occur in more than one Canopy.</p>
+
+<p>Canopy Clustering is often used as an initial step in more rigorous
+clustering techniques, such as <a href="k-means-clustering.html">K-Means Clustering</a>
+. By starting with an initial clustering the number of more expensive
+distance measurements can be significantly reduced by ignoring points
+outside of the initial canopies.</p>
+
+<h4 id="strategy-for-parallelization">Strategy for parallelization</h4>
+
+<p>Looking at the sample Hadoop implementation in <a href="http://code.google.com/p/canopy-clustering/">http://code.google.com/p/canopy-clustering/</a>
+ the processing is done in 3 steps:</p>
+<ol>
+ <li>The data is massaged into suitable input format</li>
+ <li>Each mapper performs canopy clustering on the points in its input set and
+outputs its canopies’ centers</li>
+ <li>The reducer clusters the canopy centers to produce the final canopy
+centers</li>
+</ol>
+
+<p>The points are then clustered into these final canopies when the <code class="highlighter-rouge">model.cluster(inputDRM)</code> is called.</p>
+
+<p>Some ideas can be found in <a href="https://www.youtube.com/watch?v=yjPBkvYh-ss&list=PLEFAB97242917704A">Cluster computing and MapReduce</a>
+ lecture video series [by Google(r)]; Canopy Clustering is discussed in <a href="https://www.youtube.com/watch?v=1ZDybXl212Q">lecture #4</a>
+. Finally here is the <a href="http://en.wikipedia.org/wiki/Canopy_clustering_algorithm">Wikipedia page</a>
+.</p>
+
+<h4 id="illustrations">Illustrations</h4>
+
+<p>The following images illustrate Canopy clustering applied to a set of
+randomly-generated 2-d data points. The points are generated using a normal
+distribution centered at a mean location and with a constant standard
+deviation. See the README file in the <a href="https://github.com/apache/mahout/blob/master/examples/src/main/java/org/apache/mahout/clustering/display/README.txt">/examples/src/main/java/org/apache/mahout/clustering/display/README.txt</a>
+ for details on running similar examples.</p>
+
+<p>The points are generated as follows:</p>
+
+<ul>
+ <li>500 samples m=[1.0, 1.0](1.0,-1.0.html)
+ sd=3.0</li>
+ <li>300 samples m=[1.0, 0.0](1.0,-0.0.html)
+ sd=0.5</li>
+ <li>300 samples m=[0.0, 2.0](0.0,-2.0.html)
+ sd=0.1</li>
+</ul>
+
+<p>In the first image, the points are plotted and the 3-sigma boundaries of
+their generator are superimposed.</p>
+
+<p><img src="SampleData.png" alt="sample data" /></p>
+
+<p>In the second image, the resulting canopies are shown superimposed upon the
+sample data. Each canopy is represented by two circles, with radius T1 and
+radius T2.</p>
+
+<p><img src="Canopy.png" alt="canopy" /></p>
+
+<p>The third image uses the same values of T1 and T2 but only superimposes
+canopies covering more than 10% of the population. This is a bit better
+representation of the data but it still has lots of room for improvement.
+The advantage of Canopy clustering is that it is single-pass and fast
+enough to iterate runs using different T1, T2 parameters and display
+thresholds.</p>
+
+<p><img src="Canopy10.png" alt="canopy" /></p>
+
+<h3 id="parameters">Parameters</h3>
+
+<div class="table-striped">
+ <table class="table">
+ <tr>
+ <th>Parameter</th>
+ <th>Description</th>
+ <th>Default Value</th>
+ </tr>
+ <tr>
+ <td><code>'distanceMeasure</code></td>
+ <td>The metric used for calculating distance, see <a href="../distance-metrics.html">Distance Metrics</a></td>
+ <td><code>'Cosine</code></td>
+ </tr>
+ <tr>
+ <td><code>'t1</code></td>
+ <td>The "loose" distance in the mapping phase</code></td>
+ <td>0.5</td>
+ </tr>
+ <tr>
+ <td><code>'t2</code></td>
+ <td>The "tight" distance in the mapping phase</code></td>
+ <td>0.1</td>
+ </tr>
+ <tr>
+ <td><code>'t3</code></td>
+ <td>The "loose" distance in the reducing phase</code></td>
+ <td><code>'t1</code></td>
+ </tr>
+ <tr>
+ <td><code>'t4</code></td>
+ <td>The "tight" distance in the reducing phase</code></td>
+ <td><code>'t2</code></td>
+ </tr>
+ </table>
+</div>
+
+<h3 id="example">Example</h3>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>val drmA = drmParallelize(dense((1.0, 1.2, 1.3, 1.4), (1.1, 1.5, 2.5, 1.0), (6.0, 5.2, -5.2, 5.3), (7.0,6.0, 5.0, 5.0), (10.0, 1.0, 20.0, -10.0)))
+
+import org.apache.mahout.math.algorithms.clustering.CanopyClustering
+
+val model = new CanopyClustering().fit(drmA, 't1 -> 6.5, 't2 -> 5.5, 'distanceMeasure -> 'Chebyshev)
+model.cluster(drmA).collect
+</code></pre>
+</div>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/clustering/distance-metrics.html b/docs/0.13.1/api/docs/algorithms/clustering/distance-metrics.html
new file mode 100644
index 0000000..1893dd7
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/clustering/distance-metrics.html
@@ -0,0 +1,269 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ Distance Metrics
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/clustering/distance-metrics.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h3 id="distance-metrics-supported-by-mahout">Distance Metrics Supported By Mahout</h3>
+
+<div class="table-striped">
+ <table class="table">
+ <tr>
+ <th>Name</th>
+ <th>Object</th>
+ <th>Symbol</th>
+ </tr>
+ <tr>
+ <td><a href="https://en.wikipedia.org/wiki/Chebyshev_distance">Chebyshev Distance</a></td>
+ <td><code>org.apache.mahout.math.algorithms.common.distance.Chebyshev</code></td>
+ <td><code>'Chebyshev</code></td>
+ </tr>
+ <tr>
+ <td><a href="https://en.wikipedia.org/wiki/Cosine_similarity">Cosine Similarity</a></td>
+ <td><code>org.apache.mahout.math.algorithms.common.distance.Cosine</code></td>
+ <td><code>'Cosine</code></td>
+ </tr>
+ </table>
+</div>
+
+<!--
+A beginner JIRA to port the rest of these
+[Euclidean](https://en.wikipedia.org/wiki/Euclidean_distance)
+
+[Mahalanobis](https://en.wikipedia.org/wiki/Mahalanobis_distance)
+
+[Manhattan](https://en.wiktionary.org/wiki/Manhattan_distance)
+
+[Minkowski](https://en.wikipedia.org/wiki/Minkowski_distance)
+
+[Squared Euclidian](https://en.wikipedia.org/wiki/Euclidean_distance#Squared_Euclidean_distance)
+
+[Tanimoto](https://en.wikipedia.org/wiki/Jaccard_index#Tanimoto_similarity_and_distance)
+
+Weighted Euclidean
+
+Weighted Manhattan-->
+
+<h3 id="using-distance-metrics">Using Distance Metrics</h3>
+
+<p>In Mahout one can access the distant metrics directly to measure the distance between two arbitrary vectors, or
+can specify which distance metric to use as part of an algorithm. In the latter case the distance metric is called
+by <code class="highlighter-rouge">Symbol</code>, we never pass Distance metrics directly to an algorithm. This design choice, in part has to do with
+serialization of object and keeping the engine bindings as simple as possible. Behind the scenes, the only thing
+that is serialized and sent to the workers is a number which specifies what distant metric to use- this is much more
+abstract and easier to maintain on the back end than making sure each function can be serialized by any arbitrary engine.
+We feel from the user perspective, it may seem quirky but causes no decrease in usability. If a user wishes to use a
+custom distance metric- simply add it to <a href="https://github.com/apache/mahout/blob/master/math-scala/src/main/scala/org/apache/mahout/math/algorithms/common/DistanceMetrics.scala">math-scala/src/main/org/apache/mahout/math/common/DistanceMetrics.scala</a>
+and recompile.</p>
+
+<h3 id="examples">Examples</h3>
+
+<p><strong>Meausring the distance between two vectors</strong></p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>import org.apache.mahout.math.algorithms.common.distance._
+
+val v1 = dvec(1.0, 1.5, -1.2, 3.5)
+val v2 = dvec(0.1, -1.4, 10.5, 3.2)
+
+Cosine.distance(v1, v2)
+</code></pre>
+</div>
+
+<p><strong>Using distance in clustering</strong></p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>import org.apache.mahout.math.algorithms.clustering.CanopyClustering
+
+val drmA = drmParallelize(dense((1.0, 1.2, 1.3, 1.4),
+ (1.1, 1.5, 2.5, 1.0),
+ (6.0, 5.2, -5.2, 5.3),
+ (7.0,6.0, 5.0, 5.0),
+ (10.0, 1.0, 20.0, -10.0)))
+
+val model = new CanopyClustering().fit(drmA, 'distanceMeasure -> 'Cosine)
+</code></pre>
+</div>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/clustering/index.html b/docs/0.13.1/api/docs/algorithms/clustering/index.html
new file mode 100644
index 0000000..c77c4cc
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/clustering/index.html
@@ -0,0 +1,194 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ Clustering Algorithms
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/clustering/">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a href="canopy/">Canopy Clustering</a></p>
+
+<p><a href="distance-metrics.html">Distance Metrics</a></p>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/linear-algebra/d-qr.html b/docs/0.13.1/api/docs/algorithms/linear-algebra/d-qr.html
new file mode 100644
index 0000000..6d4517e
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/linear-algebra/d-qr.html
@@ -0,0 +1,244 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ Distributed Cholesky QR
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/linear-algebra/d-qr.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h2 id="intro">Intro</h2>
+
+<p>Mahout has a distributed implementation of QR decomposition for tall thin matrices<a href="[Mahout Scala and Mahout Spark Bindings for Linear Algebra Subroutines](http://mahout.apache.org/users/sparkbindings/ScalaSparkBindings.pdf)">1</a>.</p>
+
+<h2 id="algorithm">Algorithm</h2>
+
+<p>For the classic QR decomposition of the form <script type="math/tex">\mathbf{A}=\mathbf{QR},\mathbf{A}\in\mathbb{R}^{m\times n}</script>
+a distributed version is fairly easily achieved if <script type="math/tex">\mathbf{A}</script> is tall and thin such that
+<script type="math/tex">\mathbf{A}^{\top}\mathbf{A}</script> fits in memory, i.e. <em>m</em> is large but <em>n</em> < ~5000 Under such circumstances,
+only <script type="math/tex">\mathbf{A}</script> and <script type="math/tex">\mathbf{Q}</script> are distributed matrices and <script type="math/tex">\mathbf{A^{\top}A}</script> and
+<script type="math/tex">\mathbf{R}</script> are in-core products. We just compute the in-core version of the Cholesky decomposition
+in the form of <script type="math/tex">\mathbf{LL}^{\top}= \mathbf{A}^{\top}\mathbf{A}</script>. After that we take <script type="math/tex">\mathbf{R}= \mathbf{L}^{\top}</script>
+ and <script type="math/tex">\mathbf{Q}=\mathbf{A}\left(\mathbf{L}^{\top}\right)^{-1}</script>. The latter is easily achieved by multiplying each
+ vertical block of <script type="math/tex">\mathbf{A}</script> by <script type="math/tex">\left(\mathbf{L}^{\top}\right)^{-1}</script>. (There is no actual matrix inversion
+ happening).</p>
+
+<h2 id="implementation">Implementation</h2>
+
+<p>Mahout <code class="highlighter-rouge">dqrThin(...)</code> is implemented in the mahout <code class="highlighter-rouge">math-scala</code> algebraic optimizer which translates Mahout’s R-like linear algebra operators into a physical plan for both Spark and H2O distributed engines.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>def dqrThin[K: ClassTag](A: DrmLike[K], checkRankDeficiency: Boolean = true): (DrmLike[K], Matrix) = {
+ if (drmA.ncol > 5000)
+ log.warn("A is too fat. A'A must fit in memory and easily broadcasted.")
+ implicit val ctx = drmA.context
+ val AtA = (drmA.t %*% drmA).checkpoint()
+ val inCoreAtA = AtA.collect
+ val ch = chol(inCoreAtA)
+ val inCoreR = (ch.getL cloned) t
+ if (checkRankDeficiency && !ch.isPositiveDefinite)
+ throw new IllegalArgumentException("R is rank-deficient.")
+ val bcastAtA = sc.broadcast(inCoreAtA)
+ val Q = A.mapBlock() {
+ case (keys, block) => keys -> chol(bcastAtA).solveRight(block)
+ }
+ Q -> inCoreR
+}
+</code></pre>
+</div>
+
+<h2 id="usage">Usage</h2>
+
+<p>The scala <code class="highlighter-rouge">dqrThin(...)</code> method can easily be called in any Spark or H2O application built with the <code class="highlighter-rouge">math-scala</code> library and the corresponding <code class="highlighter-rouge">Spark</code> or <code class="highlighter-rouge">H2O</code> engine module as follows:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>import org.apache.mahout.math._
+import decompositions._
+import drm._
+
+val(drmQ, inCoreR) = dqrThin(drma)
+</code></pre>
+</div>
+
+<h2 id="references">References</h2>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/linear-algebra/d-spca.html b/docs/0.13.1/api/docs/algorithms/linear-algebra/d-spca.html
new file mode 100644
index 0000000..3abe0ae
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/linear-algebra/d-spca.html
@@ -0,0 +1,365 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ Distributed Stochastic PCA
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/linear-algebra/d-spca.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h2 id="intro">Intro</h2>
+
+<p>Mahout has a distributed implementation of Stochastic PCA<a href="Lyubimov and Palumbo, ["Apache Mahout: Beyond MapReduce; Distributed Algorithm Design"](https://www.amazon.com/Apache-Mahout-MapReduce-Dmitriy-Lyubimov/dp/1523775785)">1</a>. This algorithm computes the exact equivalent of Mahout’s dssvd(<code class="highlighter-rouge">\(\mathbf{A-1\mu^\top}\)</code>) by modifying the <code class="highlighter-rouge">dssvd</code> algorithm so as to avoid forming <code class="hi [...]
+
+<h2 id="algorithm">Algorithm</h2>
+
+<p>Given an <em>m</em> <code class="highlighter-rouge">\(\times\)</code> <em>n</em> matrix <code class="highlighter-rouge">\(\mathbf{A}\)</code>, a target rank <em>k</em>, and an oversampling parameter <em>p</em>, this procedure computes a <em>k</em>-rank PCA by finding the unknowns in <code class="highlighter-rouge">\(\mathbf{A−1\mu^\top \approx U\Sigma V^\top}\)</code>:</p>
+
+<ol>
+ <li>Create seed for random <em>n</em> <code class="highlighter-rouge">\(\times\)</code> <em>(k+p)</em> matrix <code class="highlighter-rouge">\(\Omega\)</code>.</li>
+ <li><code class="highlighter-rouge">\(\mathbf{s_\Omega \leftarrow \Omega^\top \mu}\)</code>.</li>
+ <li><code class="highlighter-rouge">\(\mathbf{Y_0 \leftarrow A\Omega − 1 {s_\Omega}^\top, Y \in \mathbb{R}^{m\times(k+p)}}\)</code>.</li>
+ <li>Column-orthonormalize <code class="highlighter-rouge">\(\mathbf{Y_0} \rightarrow \mathbf{Q}\)</code> by computing thin decomposition <code class="highlighter-rouge">\(\mathbf{Y_0} = \mathbf{QR}\)</code>. Also, <code class="highlighter-rouge">\(\mathbf{Q}\in\mathbb{R}^{m\times(k+p)}, \mathbf{R}\in\mathbb{R}^{(k+p)\times(k+p)}\)</code>.</li>
+ <li><code class="highlighter-rouge">\(\mathbf{s_Q \leftarrow Q^\top 1}\)</code>.</li>
+ <li><code class="highlighter-rouge">\(\mathbf{B_0 \leftarrow Q^\top A: B \in \mathbb{R}^{(k+p)\times n}}\)</code>.</li>
+ <li><code class="highlighter-rouge">\(\mathbf{s_B \leftarrow {B_0}^\top \mu}\)</code>.</li>
+ <li>For <em>i</em> in 1..<em>q</em> repeat (power iterations):
+ <ul>
+ <li>For <em>j</em> in 1..<em>n</em> apply <code class="highlighter-rouge">\(\mathbf{(B_{i−1})_{∗j} \leftarrow (B_{i−1})_{∗j}−\mu_j s_Q}\)</code>.</li>
+ <li><code class="highlighter-rouge">\(\mathbf{Y_i \leftarrow A{B_{i−1}}^\top−1(s_B−\mu^\top \mu s_Q)^\top}\)</code>.</li>
+ <li>Column-orthonormalize <code class="highlighter-rouge">\(\mathbf{Y_i} \rightarrow \mathbf{Q}\)</code> by computing thin decomposition <code class="highlighter-rouge">\(\mathbf{Y_i = QR}\)</code>.</li>
+ <li><code class="highlighter-rouge">\(\mathbf{s_Q \leftarrow Q^\top 1}\)</code>.</li>
+ <li><code class="highlighter-rouge">\(\mathbf{B_i \leftarrow Q^\top A}\)</code>.</li>
+ <li><code class="highlighter-rouge">\(\mathbf{s_B \leftarrow {B_i}^\top \mu}\)</code>.</li>
+ </ul>
+ </li>
+ <li>Let <code class="highlighter-rouge">\(\mathbf{C \triangleq s_Q {s_B}^\top}\)</code>. <code class="highlighter-rouge">\(\mathbf{M \leftarrow B_q {B_q}^\top − C − C^\top + \mu^\top \mu s_Q {s_Q}^\top}\)</code>.</li>
+ <li>Compute an eigensolution of the small symmetric <code class="highlighter-rouge">\(\mathbf{M = \hat{U} \Lambda \hat{U}^\top: M \in \mathbb{R}^{(k+p)\times(k+p)}}\)</code>.</li>
+ <li>The singular values <code class="highlighter-rouge">\(\Sigma = \Lambda^{\circ 0.5}\)</code>, or, in other words, <code class="highlighter-rouge">\(\mathbf{\sigma_i= \sqrt{\lambda_i}}\)</code>.</li>
+ <li>If needed, compute <code class="highlighter-rouge">\(\mathbf{U = Q\hat{U}}\)</code>.</li>
+ <li>If needed, compute <code class="highlighter-rouge">\(\mathbf{V = B^\top \hat{U} \Sigma^{−1}}\)</code>.</li>
+ <li>If needed, items converted to the PCA space can be computed as <code class="highlighter-rouge">\(\mathbf{U\Sigma}\)</code>.</li>
+</ol>
+
+<h2 id="implementation">Implementation</h2>
+
+<p>Mahout <code class="highlighter-rouge">dspca(...)</code> is implemented in the mahout <code class="highlighter-rouge">math-scala</code> algebraic optimizer which translates Mahout’s R-like linear algebra operators into a physical plan for both Spark and H2O distributed engines.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>def dspca[K](drmA: DrmLike[K], k: Int, p: Int = 15, q: Int = 0):
+(DrmLike[K], DrmLike[Int], Vector) = {
+
+ // Some mapBlock() calls need it
+ implicit val ktag = drmA.keyClassTag
+
+ val drmAcp = drmA.checkpoint()
+ implicit val ctx = drmAcp.context
+
+ val m = drmAcp.nrow
+ val n = drmAcp.ncol
+ assert(k <= (m min n), "k cannot be greater than smaller of m, n.")
+ val pfxed = safeToNonNegInt((m min n) - k min p)
+
+ // Actual decomposition rank
+ val r = k + pfxed
+
+ // Dataset mean
+ val mu = drmAcp.colMeans
+
+ val mtm = mu dot mu
+
+ // We represent Omega by its seed.
+ val omegaSeed = RandomUtils.getRandom().nextInt()
+ val omega = Matrices.symmetricUniformView(n, r, omegaSeed)
+
+ // This done in front in a single-threaded fashion for now. Even though it doesn't require any
+ // memory beyond that is required to keep xi around, it still might be parallelized to backs
+ // for significantly big n and r. TODO
+ val s_o = omega.t %*% mu
+
+ val bcastS_o = drmBroadcast(s_o)
+ val bcastMu = drmBroadcast(mu)
+
+ var drmY = drmAcp.mapBlock(ncol = r) {
+ case (keys, blockA) ⇒
+ val s_o:Vector = bcastS_o
+ val blockY = blockA %*% Matrices.symmetricUniformView(n, r, omegaSeed)
+ for (row ← 0 until blockY.nrow) blockY(row, ::) -= s_o
+ keys → blockY
+ }
+ // Checkpoint Y
+ .checkpoint()
+
+ var drmQ = dqrThin(drmY, checkRankDeficiency = false)._1.checkpoint()
+
+ var s_q = drmQ.colSums()
+ var bcastVarS_q = drmBroadcast(s_q)
+
+ // This actually should be optimized as identically partitioned map-side A'B since A and Q should
+ // still be identically partitioned.
+ var drmBt = (drmAcp.t %*% drmQ).checkpoint()
+
+ var s_b = (drmBt.t %*% mu).collect(::, 0)
+ var bcastVarS_b = drmBroadcast(s_b)
+
+ for (i ← 0 until q) {
+
+ // These closures don't seem to live well with outside-scope vars. This doesn't record closure
+ // attributes correctly. So we create additional set of vals for broadcast vars to properly
+ // create readonly closure attributes in this very scope.
+ val bcastS_q = bcastVarS_q
+ val bcastMuInner = bcastMu
+
+ // Fix Bt as B' -= xi cross s_q
+ drmBt = drmBt.mapBlock() {
+ case (keys, block) ⇒
+ val s_q: Vector = bcastS_q
+ val mu: Vector = bcastMuInner
+ keys.zipWithIndex.foreach {
+ case (key, idx) ⇒ block(idx, ::) -= s_q * mu(key)
+ }
+ keys → block
+ }
+
+ drmY.uncache()
+ drmQ.uncache()
+
+ val bCastSt_b = drmBroadcast(s_b -=: mtm * s_q)
+
+ drmY = (drmAcp %*% drmBt)
+ // Fix Y by subtracting st_b from each row of the AB'
+ .mapBlock() {
+ case (keys, block) ⇒
+ val st_b: Vector = bCastSt_b
+ block := { (_, c, v) ⇒ v - st_b(c) }
+ keys → block
+ }
+ // Checkpoint Y
+ .checkpoint()
+
+ drmQ = dqrThin(drmY, checkRankDeficiency = false)._1.checkpoint()
+
+ s_q = drmQ.colSums()
+ bcastVarS_q = drmBroadcast(s_q)
+
+ // This on the other hand should be inner-join-and-map A'B optimization since A and Q_i are not
+ // identically partitioned anymore.
+ drmBt = (drmAcp.t %*% drmQ).checkpoint()
+
+ s_b = (drmBt.t %*% mu).collect(::, 0)
+ bcastVarS_b = drmBroadcast(s_b)
+ }
+
+ val c = s_q cross s_b
+ val inCoreBBt = (drmBt.t %*% drmBt).checkpoint(CacheHint.NONE).collect -=:
+ c -=: c.t +=: mtm *=: (s_q cross s_q)
+ val (inCoreUHat, d) = eigen(inCoreBBt)
+ val s = d.sqrt
+
+ // Since neither drmU nor drmV are actually computed until actually used, we don't need the flags
+ // instructing compute (or not compute) either of the U,V outputs anymore. Neat, isn't it?
+ val drmU = drmQ %*% inCoreUHat
+ val drmV = drmBt %*% (inCoreUHat %*% diagv(1 / s))
+
+ (drmU(::, 0 until k), drmV(::, 0 until k), s(0 until k))
+}
+</code></pre>
+</div>
+
+<h2 id="usage">Usage</h2>
+
+<p>The scala <code class="highlighter-rouge">dspca(...)</code> method can easily be called in any Spark, Flink, or H2O application built with the <code class="highlighter-rouge">math-scala</code> library and the corresponding <code class="highlighter-rouge">Spark</code>, <code class="highlighter-rouge">Flink</code>, or <code class="highlighter-rouge">H2O</code> engine module as follows:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>import org.apache.mahout.math._
+import decompositions._
+import drm._
+
+val (drmU, drmV, s) = dspca(drmA, k=200, q=1)
+</code></pre>
+</div>
+
+<p>Note the parameter is optional and its default value is zero.</p>
+
+<h2 id="references">References</h2>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/linear-algebra/d-ssvd.html b/docs/0.13.1/api/docs/algorithms/linear-algebra/d-ssvd.html
new file mode 100644
index 0000000..298b129
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/linear-algebra/d-ssvd.html
@@ -0,0 +1,330 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ Distributed Stochastic Singular Value Decomposition
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/linear-algebra/d-ssvd.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h2 id="intro">Intro</h2>
+
+<p>Mahout has a distributed implementation of Stochastic Singular Value Decomposition <a href="[Mahout Scala and Mahout Spark Bindings for Linear Algebra Subroutines](http://mahout.apache.org/users/sparkbindings/ScalaSparkBindings.pdf)">1</a> using the parallelization strategy comprehensively defined in Nathan Halko’s dissertation <a href="http://amath.colorado.edu/faculty/martinss/Pubs/2012_halko_dissertation.pdf">“Randomized methods for computing low-rank approximations of matrices”</a [...]
+
+<h2 id="modified-ssvd-algorithm">Modified SSVD Algorithm</h2>
+
+<p>Given an <foo>\(m\times n\)</foo>
+matrix <foo>\(\mathbf{A}\)</foo>, a target rank <foo>\(k\in\mathbb{N}_{1}\)</foo>
+, an oversampling parameter <foo>\(p\in\mathbb{N}_{1}\)</foo>,
+and the number of additional power iterations <foo>\(q\in\mathbb{N}_{0}\)</foo>,
+this procedure computes an <foo>\(m\times\left(k+p\right)\)</foo>
+SVD <foo>\(\mathbf{A\approx U}\boldsymbol{\Sigma}\mathbf{V}^{\top}\)</foo>:</p>
+
+<ol>
+ <li>
+ <p>Create seed for random <foo>\(n\times\left(k+p\right)\)</foo>
+ matrix <foo>\(\boldsymbol{\Omega}\)</foo>. The seed defines matrix <foo>\(\mathbf{\Omega}\)</foo>
+ using Gaussian unit vectors per one of suggestions in [Halko, Martinsson, Tropp].</p>
+ </li>
+ <li>
+ <foo>\(\mathbf{Y=A\boldsymbol{\Omega}},\,\mathbf{Y}\in\mathbb{R}^{m\times\left(k+p\right)}\)</foo>
+ </li>
+ <li>
+ <p>Column-orthonormalize <foo>\(\mathbf{Y}\rightarrow\mathbf{Q}\)</foo>
+ by computing thin decomposition <foo>\(\mathbf{Y}=\mathbf{Q}\mathbf{R}\)</foo>.
+ Also, <foo>\(\mathbf{Q}\in\mathbb{R}^{m\times\left(k+p\right)},\,\mathbf{R}\in\mathbb{R}^{\left(k+p\right)\times\left(k+p\right)}\)</foo>; denoted as <foo>\(\mathbf{Q}=\mbox{qr}\left(\mathbf{Y}\right).\mathbf{Q}\)</foo></p>
+ </li>
+ <li>
+ <foo>\(\mathbf{B}_{0}=\mathbf{Q}^{\top}\mathbf{A}:\,\,\mathbf{B}\in\mathbb{R}^{\left(k+p\right)\times n}\)</foo>
+ <p>.</p>
+ </li>
+ <li>If <foo>\(q>0\)</foo>
+ repeat: for <foo>\(i=1..q\)</foo>:</li>
+</ol>
+<foo>\(\mathbf{B}_{i}^{\top}=\mathbf{A}^{\top}\mbox{qr}\left(\mathbf{A}\mathbf{B}_{i-1}^{\top}\right).\mathbf{Q}\)</foo>
+<p>(power iterations step).</p>
+
+<ol>
+ <li>Compute Eigensolution of a small Hermitian <foo>\(\mathbf{B}_{q}\mathbf{B}_{q}^{\top}=\mathbf{\hat{U}}\boldsymbol{\Lambda}\mathbf{\hat{U}}^{\top}\)</foo>,</li>
+</ol>
+<foo>\(\mathbf{B}_{q}\mathbf{B}_{q}^{\top}\in\mathbb{R}^{\left(k+p\right)\times\left(k+p\right)}\)</foo>
+<p>.</p>
+
+<ol>
+ <li>
+ <p>Singular values <foo>\(\mathbf{\boldsymbol{\Sigma}}=\boldsymbol{\Lambda}^{0.5}\)</foo>,
+ or, in other words, <foo>\(s_{i}=\sqrt{\sigma_{i}}\)</foo>.</p>
+ </li>
+ <li>
+ <p>If needed, compute <foo>\(\mathbf{U}=\mathbf{Q}\hat{\mathbf{U}}\)</foo>.</p>
+ </li>
+ <li>
+ <p>If needed, compute <foo>\(\mathbf{V}=\mathbf{B}_{q}^{\top}\hat{\mathbf{U}}\boldsymbol{\Sigma}^{-1}\)</foo>.
+Another way is <foo>\(\mathbf{V}=\mathbf{A}^{\top}\mathbf{U}\boldsymbol{\Sigma}^{-1}\)</foo>.</p>
+ </li>
+</ol>
+
+<h2 id="implementation">Implementation</h2>
+
+<p>Mahout <code class="highlighter-rouge">dssvd(...)</code> is implemented in the mahout <code class="highlighter-rouge">math-scala</code> algebraic optimizer which translates Mahout’s R-like linear algebra operators into a physical plan for both Spark and H2O distributed engines.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>def dssvd[K: ClassTag](drmA: DrmLike[K], k: Int, p: Int = 15, q: Int = 0):
+ (DrmLike[K], DrmLike[Int], Vector) = {
+
+ val drmAcp = drmA.checkpoint()
+
+ val m = drmAcp.nrow
+ val n = drmAcp.ncol
+ assert(k <= (m min n), "k cannot be greater than smaller of m, n.")
+ val pfxed = safeToNonNegInt((m min n) - k min p)
+
+ // Actual decomposition rank
+ val r = k + pfxed
+
+ // We represent Omega by its seed.
+ val omegaSeed = RandomUtils.getRandom().nextInt()
+
+ // Compute Y = A*Omega.
+ var drmY = drmAcp.mapBlock(ncol = r) {
+ case (keys, blockA) =>
+ val blockY = blockA %*% Matrices.symmetricUniformView(n, r, omegaSeed)
+ keys -> blockY
+ }
+
+ var drmQ = dqrThin(drmY.checkpoint())._1
+
+ // Checkpoint Q if last iteration
+ if (q == 0) drmQ = drmQ.checkpoint()
+
+ var drmBt = drmAcp.t %*% drmQ
+
+ // Checkpoint B' if last iteration
+ if (q == 0) drmBt = drmBt.checkpoint()
+
+ for (i <- 0 until q) {
+ drmY = drmAcp %*% drmBt
+ drmQ = dqrThin(drmY.checkpoint())._1
+
+ // Checkpoint Q if last iteration
+ if (i == q - 1) drmQ = drmQ.checkpoint()
+
+ drmBt = drmAcp.t %*% drmQ
+
+ // Checkpoint B' if last iteration
+ if (i == q - 1) drmBt = drmBt.checkpoint()
+ }
+
+ val (inCoreUHat, d) = eigen(drmBt.t %*% drmBt)
+ val s = d.sqrt
+
+ // Since neither drmU nor drmV are actually computed until actually used
+ // we don't need the flags instructing compute (or not compute) either of the U,V outputs
+ val drmU = drmQ %*% inCoreUHat
+ val drmV = drmBt %*% (inCoreUHat %*%: diagv(1 /: s))
+
+ (drmU(::, 0 until k), drmV(::, 0 until k), s(0 until k))
+}
+</code></pre>
+</div>
+
+<p>Note: As a side effect of checkpointing, U and V values are returned as logical operators (i.e. they are neither checkpointed nor computed). Therefore there is no physical work actually done to compute <foo>\(\mathbf{U}\)</foo> or <foo>\(\mathbf{V}\)</foo> until they are used in a subsequent expression.</p>
+
+<h2 id="usage">Usage</h2>
+
+<p>The scala <code class="highlighter-rouge">dssvd(...)</code> method can easily be called in any Spark or H2O application built with the <code class="highlighter-rouge">math-scala</code> library and the corresponding <code class="highlighter-rouge">Spark</code> or <code class="highlighter-rouge">H2O</code> engine module as follows:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>import org.apache.mahout.math._
+import decompositions._
+import drm._
+
+
+val(drmU, drmV, s) = dssvd(drma, k = 40, q = 1)
+</code></pre>
+</div>
+
+<h2 id="references">References</h2>
+
+<p>approximations of matrices](http://amath.colorado.edu/faculty/martinss/Pubs/2012_halko_dissertation.pdf)</p>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/linear-algebra/index.html b/docs/0.13.1/api/docs/algorithms/linear-algebra/index.html
new file mode 100644
index 0000000..a1ac2da
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/linear-algebra/index.html
@@ -0,0 +1,199 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ Distributed Linear Algebra
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/linear-algebra/">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p>Mahout has a number of distributed linear algebra “algorithms” that, in concert with the mathematically expressive R-Like Scala DSL, make it possible for users to quickly “roll their own” distributed algorithms.</p>
+
+<p><a href="d-qr.html">Distributed QR Decomposition</a></p>
+
+<p><a href="d-spca.html">Distributed Stochastic Principal Component Analysis</a></p>
+
+<p><a href="d-ssvd.html">Distributed Stochastic Singular Value Decomposition</a></p>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/bayesian.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/bayesian.html
new file mode 100644
index 0000000..d6d2d80
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/bayesian.html
@@ -0,0 +1,365 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated)
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/bayesian.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h1 id="naive-bayes">Naive Bayes</h1>
+
+<h2 id="intro">Intro</h2>
+
+<p>Mahout currently has two Naive Bayes Map-Reduce implementations. The first is standard Multinomial Naive Bayes. The second is an implementation of Transformed Weight-normalized Complement Naive Bayes as introduced by Rennie et al. <a href="http://people.csail.mit.edu/jrennie/papers/icml03-nb.pdf">[1]</a>. We refer to the former as Bayes and the latter as CBayes.</p>
+
+<p>Where Bayes has long been a standard in text classification, CBayes is an extension of Bayes that performs particularly well on datasets with skewed classes and has been shown to be competitive with algorithms of higher complexity such as Support Vector Machines.</p>
+
+<h2 id="implementations">Implementations</h2>
+<p>Both Bayes and CBayes are currently trained via MapReduce Jobs. Testing and classification can be done via a MapReduce Job or sequentially. Mahout provides CLI drivers for preprocessing, training and testing. A Spark implementation is currently in the works (<a href="https://issues.apache.org/jira/browse/MAHOUT-1493">MAHOUT-1493</a>).</p>
+
+<h2 id="preprocessing-and-algorithm">Preprocessing and Algorithm</h2>
+
+<p>As described in <a href="http://people.csail.mit.edu/jrennie/papers/icml03-nb.pdf">[1]</a> Mahout Naive Bayes is broken down into the following steps (assignments are over all possible index values):</p>
+
+<ul>
+ <li>Let <code class="highlighter-rouge">\(\vec{d}=(\vec{d_1},...,\vec{d_n})\)</code> be a set of documents; <code class="highlighter-rouge">\(d_{ij}\)</code> is the count of word <code class="highlighter-rouge">\(i\)</code> in document <code class="highlighter-rouge">\(j\)</code>.</li>
+ <li>Let <code class="highlighter-rouge">\(\vec{y}=(y_1,...,y_n)\)</code> be their labels.</li>
+ <li>Let <code class="highlighter-rouge">\(\alpha_i\)</code> be a smoothing parameter for all words in the vocabulary; let <code class="highlighter-rouge">\(\alpha=\sum_i{\alpha_i}\)</code>.</li>
+ <li><strong>Preprocessing</strong>(via seq2Sparse) TF-IDF transformation and L2 length normalization of <code class="highlighter-rouge">\(\vec{d}\)</code>
+ <ol>
+ <li><code class="highlighter-rouge">\(d_{ij} = \sqrt{d_{ij}}\)</code></li>
+ <li><code class="highlighter-rouge">\(d_{ij} = d_{ij}\left(\log{\frac{\sum_k1}{\sum_k\delta_{ik}+1}}+1\right)\)</code></li>
+ <li><code class="highlighter-rouge">\(d_{ij} =\frac{d_{ij}}{\sqrt{\sum_k{d_{kj}^2}}}\)</code></li>
+ </ol>
+ </li>
+ <li><strong>Training: Bayes</strong><code class="highlighter-rouge">\((\vec{d},\vec{y})\)</code> calculate term weights <code class="highlighter-rouge">\(w_{ci}\)</code> as:
+ <ol>
+ <li><code class="highlighter-rouge">\(\hat\theta_{ci}=\frac{d_{ic}+\alpha_i}{\sum_k{d_{kc}}+\alpha}\)</code></li>
+ <li><code class="highlighter-rouge">\(w_{ci}=\log{\hat\theta_{ci}}\)</code></li>
+ </ol>
+ </li>
+ <li><strong>Training: CBayes</strong><code class="highlighter-rouge">\((\vec{d},\vec{y})\)</code> calculate term weights <code class="highlighter-rouge">\(w_{ci}\)</code> as:
+ <ol>
+ <li><code class="highlighter-rouge">\(\hat\theta_{ci} = \frac{\sum_{j:y_j\neq c}d_{ij}+\alpha_i}{\sum_{j:y_j\neq c}{\sum_k{d_{kj}}}+\alpha}\)</code></li>
+ <li><code class="highlighter-rouge">\(w_{ci}=-\log{\hat\theta_{ci}}\)</code></li>
+ <li><code class="highlighter-rouge">\(w_{ci}=\frac{w_{ci}}{\sum_i \lvert w_{ci}\rvert}\)</code></li>
+ </ol>
+ </li>
+ <li><strong>Label Assignment/Testing:</strong>
+ <ol>
+ <li>Let <code class="highlighter-rouge">\(\vec{t}= (t_1,...,t_n)\)</code> be a test document; let <code class="highlighter-rouge">\(t_i\)</code> be the count of the word <code class="highlighter-rouge">\(t\)</code>.</li>
+ <li>Label the document according to <code class="highlighter-rouge">\(l(t)=\arg\max_c \sum\limits_{i} t_i w_{ci}\)</code></li>
+ </ol>
+ </li>
+</ul>
+
+<p>As we can see, the main difference between Bayes and CBayes is the weight calculation step. Where Bayes weighs terms more heavily based on the likelihood that they belong to class <code class="highlighter-rouge">\(c\)</code>, CBayes seeks to maximize term weights on the likelihood that they do not belong to any other class.</p>
+
+<h2 id="running-from-the-command-line">Running from the command line</h2>
+
+<p>Mahout provides CLI drivers for all above steps. Here we will give a simple overview of Mahout CLI commands used to preprocess the data, train the model and assign labels to the training set. An <a href="https://github.com/apache/mahout/blob/master/examples/bin/classify-20newsgroups.sh">example script</a> is given for the full process from data acquisition through classification of the classic <a href="https://mahout.apache.org/users/classification/twenty-newsgroups.html">20 Newsgrou [...]
+
+<ul>
+ <li>
+ <p><strong>Preprocessing:</strong>
+For a set of Sequence File Formatted documents in PATH_TO_SEQUENCE_FILES the <a href="https://mahout.apache.org/users/basics/creating-vectors-from-text.html">mahout seq2sparse</a> command performs the TF-IDF transformations (-wt tfidf option) and L2 length normalization (-n 2 option) as follows:</p>
+
+ <div class="highlighter-rouge"><pre class="highlight"><code> mahout seq2sparse
+ -i ${PATH_TO_SEQUENCE_FILES}
+ -o ${PATH_TO_TFIDF_VECTORS}
+ -nv
+ -n 2
+ -wt tfidf
+</code></pre>
+ </div>
+ </li>
+ <li>
+ <p><strong>Training:</strong>
+The model is then trained using <code class="highlighter-rouge">mahout trainnb</code> . The default is to train a Bayes model. The -c option is given to train a CBayes model:</p>
+
+ <div class="highlighter-rouge"><pre class="highlight"><code> mahout trainnb
+ -i ${PATH_TO_TFIDF_VECTORS}
+ -o ${PATH_TO_MODEL}/model
+ -li ${PATH_TO_MODEL}/labelindex
+ -ow
+ -c
+</code></pre>
+ </div>
+ </li>
+ <li>
+ <p><strong>Label Assignment/Testing:</strong>
+Classification and testing on a holdout set can then be performed via <code class="highlighter-rouge">mahout testnb</code>. Again, the -c option indicates that the model is CBayes. The -seq option tells <code class="highlighter-rouge">mahout testnb</code> to run sequentially:</p>
+
+ <div class="highlighter-rouge"><pre class="highlight"><code> mahout testnb
+ -i ${PATH_TO_TFIDF_TEST_VECTORS}
+ -m ${PATH_TO_MODEL}/model
+ -l ${PATH_TO_MODEL}/labelindex
+ -ow
+ -o ${PATH_TO_OUTPUT}
+ -c
+ -seq
+</code></pre>
+ </div>
+ </li>
+</ul>
+
+<h2 id="command-line-options">Command line options</h2>
+
+<ul>
+ <li>
+ <p><strong>Preprocessing:</strong></p>
+
+ <p>Only relevant parameters used for Bayes/CBayes as detailed above are shown. Several other transformations can be performed by <code class="highlighter-rouge">mahout seq2sparse</code> and used as input to Bayes/CBayes. For a full list of <code class="highlighter-rouge">mahout seq2Sparse</code> options see the <a href="https://mahout.apache.org/users/basics/creating-vectors-from-text.html">Creating vectors from text</a> page.</p>
+
+ <div class="highlighter-rouge"><pre class="highlight"><code> mahout seq2sparse
+ --output (-o) output The directory pathname for output.
+ --input (-i) input Path to job input directory.
+ --weight (-wt) weight The kind of weight to use. Currently TF
+ or TFIDF. Default: TFIDF
+ --norm (-n) norm The norm to use, expressed as either a
+ float or "INF" if you want to use the
+ Infinite norm. Must be greater or equal
+ to 0. The default is not to normalize
+ --overwrite (-ow) If set, overwrite the output directory
+ --sequentialAccessVector (-seq) (Optional) Whether output vectors should
+ be SequentialAccessVectors. If set true
+ else false
+ --namedVector (-nv) (Optional) Whether output vectors should
+ be NamedVectors. If set true else false
+</code></pre>
+ </div>
+ </li>
+ <li>
+ <p><strong>Training:</strong></p>
+
+ <div class="highlighter-rouge"><pre class="highlight"><code> mahout trainnb
+ --input (-i) input Path to job input directory.
+ --output (-o) output The directory pathname for output.
+ --alphaI (-a) alphaI Smoothing parameter. Default is 1.0
+ --trainComplementary (-c) Train complementary? Default is false.
+ --labelIndex (-li) labelIndex The path to store the label index in
+ --overwrite (-ow) If present, overwrite the output directory
+ before running job
+ --help (-h) Print out help
+ --tempDir tempDir Intermediate output directory
+ --startPhase startPhase First phase to run
+ --endPhase endPhase Last phase to run
+</code></pre>
+ </div>
+ </li>
+ <li>
+ <p><strong>Testing:</strong></p>
+
+ <div class="highlighter-rouge"><pre class="highlight"><code> mahout testnb
+ --input (-i) input Path to job input directory.
+ --output (-o) output The directory pathname for output.
+ --overwrite (-ow) If present, overwrite the output directory
+ before running job
+
+
+ --model (-m) model The path to the model built during training
+ --testComplementary (-c) Test complementary? Default is false.
+ --runSequential (-seq) Run sequential?
+ --labelIndex (-l) labelIndex The path to the location of the label index
+ --help (-h) Print out help
+ --tempDir tempDir Intermediate output directory
+ --startPhase startPhase First phase to run
+ --endPhase endPhase Last phase to run
+</code></pre>
+ </div>
+ </li>
+</ul>
+
+<h2 id="examples">Examples</h2>
+
+<p>Mahout provides an example for Naive Bayes classification:</p>
+
+<ol>
+ <li><a href="twenty-newsgroups.html">Classify 20 Newsgroups</a></li>
+</ol>
+
+<h2 id="references">References</h2>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/class-discovery.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/class-discovery.html
new file mode 100644
index 0000000..2224f78
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/class-discovery.html
@@ -0,0 +1,343 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Class Discovery
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/class-discovery.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="ClassDiscovery-ClassDiscovery"></a></p>
+<h1 id="class-discovery">Class Discovery</h1>
+
+<p>See http://www.cs.bham.ac.uk/~wbl/biblio/gecco1999/GP-417.pdf</p>
+
+<p>CDGA uses a Genetic Algorithm to discover a classification rule for a given
+dataset.
+A dataset can be seen as a table:</p>
+
+<table>
+<tr><th> </th><th>attribute 1</th><th>attribute 2</th><th>...</th><th>attribute N</th></tr>
+<tr><td>row 1</td><td>value1</td><td>value2</td><td>...</td><td>valueN</td></tr>
+<tr><td>row 2</td><td>value1</td><td>value2</td><td>...</td><td>valueN</td></tr>
+<tr><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td></tr>
+<tr><td>row M</td><td>value1</td><td>value2</td><td>...</td><td>valueN</td></tr>
+</table>
+
+<p>An attribute can be numerical, for example a “temperature” attribute, or
+categorical, for example a “color” attribute. For classification purposes,
+one of the categorical attributes is designated as a <em>label</em>, which means
+that its value defines the <em>class</em> of the rows.
+A classification rule can be represented as follows:</p>
+<table>
+<tr><th> </th><th>attribute 1</th><th>attribute 2</th><th>...</th><th>attribute N</th></tr>
+<tr><td>weight</td><td>w1</td><td>w2</td><td>...</td><td>wN</td></tr>
+<tr><td>operator</td><td>op1</td><td>op2</td><td>...</td><td>opN</td></tr>
+<tr><td>value</td><td>value1</td><td>value2</td><td>...</td><td>valueN</td></tr>
+</table>
+
+<p>For a given <em>target</em> class and a weight <em>threshold</em>, the classification
+rule can be read :</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>for each row of the dataset
+ if (rule.w1 < threshold || (rule.w1 >= threshold && row.value1 rule.op1 rule.value1)) &&
+ (rule.w2 < threshold || (rule.w2 >= threshold && row.value2 rule.op2 rule.value2)) &&
+ ...
+ (rule.wN < threshold || (rule.wN >= threshold && row.valueN rule.opN rule.valueN)) then
+ row is part of the target class
+</code></pre>
+</div>
+
+<p><em>Important:</em> The label attribute is not evaluated by the rule.</p>
+
+<p>The threshold parameter allows some conditions of the rule to be skipped if
+their weight is too small. The operators available depend on the attribute
+types:</p>
+<ul>
+ <li>for a numerical attributes, the available operators are ‘<’ and ‘>=’</li>
+ <li>for categorical attributes, the available operators are ‘!=’ and ‘==’</li>
+</ul>
+
+<p>The “threshold” and “target” are user defined parameters, and because the
+label is always a categorical attribute, the target is the (zero based)
+index of the class label value in all the possible values of the label. For
+example, if the label attribute can have the following values (blue, brown,
+green), then a target of 1 means the “blue” class.</p>
+
+<p>For example, we have the following dataset (the label attribute is “Eyes
+Color”):</p>
+<table>
+<tr><th> </th><th>Age</th><th>Eyes Color</th><th>Hair Color</th></tr>
+<tr><td>row 1</td><td>16</td><td>brown</td><td>dark</td></tr>
+<tr><td>row 2</td><td>25</td><td>green</td><td>light</td></tr>
+<tr><td>row 3</td><td>12</td><td>blue</td><td>light</td></tr>
+and a classification rule:
+<tr><td>weight</td><td>0</td><td>1</td></tr>
+<tr><td>operator</td><td><</td><td>!=</td></tr>
+<tr><td>value</td><td>20</td><td>light</td></tr>
+and the following parameters: threshold = 1 and target = 0 (brown).
+</table>
+
+<p>This rule can be read as follows:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>for each row of the dataset
+ if (0 < 1 || (0 >= 1 && row.value1 < 20)) &&
+ (1 < 1 || (1 >= 1 && row.value2 != light)) then
+ row is part of the "brown Eye Color" class
+</code></pre>
+</div>
+
+<p>Please note how the rule skipped the label attribute (Eye Color), and how
+the first condition is ignored because its weight is < threshold.</p>
+
+<p><a name="ClassDiscovery-Runningtheexample:"></a></p>
+<h1 id="running-the-example">Running the example:</h1>
+<p>NOTE: Substitute in the appropriate version for the Mahout JOB jar</p>
+
+<ol>
+ <li>cd <MAHOUT_HOME>/examples</MAHOUT_HOME></li>
+ <li>ant job</li>
+ <li>{code}<HADOOP_HOME>/bin/hadoop dfs -put</HADOOP_HOME></li>
+</ol>
+<MAHOUT_HOME>/examples/src/test/resources/wdbc wdbc{code}
+1. {code}<HADOOP_HOME>/bin/hadoop dfs -put
+<MAHOUT_HOME>/examples/src/test/resources/wdbc.infos wdbc.infos{code}
+1. {code}<HADOOP_HOME>/bin/hadoop jar
+<MAHOUT_HOME>/examples/build/apache-mahout-examples-0.1-dev.job
+org.apache.mahout.ga.watchmaker.cd.CDGA
+<MAHOUT_HOME>/examples/src/test/resources/wdbc 1 0.9 1 0.033 0.1 0 100 10
+
+ CDGA needs 9 parameters:
+ * param 1 : path of the directory that contains the dataset and its infos
+file
+ * param 2 : target class
+ * param 3 : threshold
+ * param 4 : number of crossover points for the multi-point crossover
+ * param 5 : mutation rate
+ * param 6 : mutation range
+ * param 7 : mutation precision
+ * param 8 : population size
+ * param 9 : number of generations before the program stops
+
+ For more information about 4th parameter, please see [Multi-point Crossover|http://www.geatbx.com/docu/algindex-03.html#P616_36571]
+.
+ For a detailed explanation about the 5th, 6th and 7th parameters, please
+see [Real Valued Mutation|http://www.geatbx.com/docu/algindex-04.html#P659_42386]
+.
+
+ *TODO*: Fill in where to find the output and what it means.
+
+ h1. The info file:
+ To run properly, CDGA needs some informations about the dataset. Each
+dataset should be accompanied by an .infos file that contains the needed
+informations. for each attribute a corresponding line in the info file
+describes it, it can be one of the following:
+ * IGNORED
+ if the attribute is ignored
+ * LABEL, val1, val2,...
+ if the attribute is the label (class), and its possible values
+ * CATEGORICAL, val1, val2,...
+ if the attribute is categorial (nominal), and its possible values
+ * NUMERICAL, min, max
+ if the attribute is numerical, and its min and max values
+
+ This file can be generated automaticaly using a special tool available with
+CDGA.
+
+
+
+* the tool searches for an existing infos file (*must be filled by the
+user*), in the same directory of the dataset with the same name and with
+the ".infos" extension, that contain the type of the attributes:
+ ** 'N' numerical attribute
+ ** 'C' categorical attribute
+ ** 'L' label (this also a categorical attribute)
+ ** 'I' to ignore the attribute
+ each attribute is in a separate
+* A Hadoop job is used to parse the dataset and collect the informations.
+This means that *the dataset can be distributed over HDFS*.
+* the results are written back in the same .info file, with the correct
+format needed by CDGA.
+</MAHOUT_HOME></MAHOUT_HOME></HADOOP_HOME></MAHOUT_HOME></HADOOP_HOME></MAHOUT_HOME>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/classifyingyourdata.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/classifyingyourdata.html
new file mode 100644
index 0000000..9bdd58d
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/classifyingyourdata.html
@@ -0,0 +1,212 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) ClassifyingYourData
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/classifyingyourdata.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h1 id="classifying-data-from-the-command-line">Classifying data from the command line</h1>
+
+<p>After you’ve done the <a href="../basics/quickstart.html">Quickstart</a> and are familiar with the basics of Mahout, it is time to build a
+classifier from your own data. The following pieces <em>may</em> be useful for in getting started:</p>
+
+<p><a name="ClassifyingYourData-Input"></a></p>
+<h1 id="input">Input</h1>
+
+<p>For starters, you will need your data in an appropriate Vector format: See <a href="../basics/creating-vectors.html">Creating Vectors</a> as well as <a href="../basics/creating-vectors-from-text.html">Creating Vectors from Text</a>.</p>
+
+<p><a name="ClassifyingYourData-RunningtheProcess"></a></p>
+<h1 id="running-the-process">Running the Process</h1>
+
+<ul>
+ <li>Logistic regression <a href="logistic-regression.html">background</a></li>
+ <li><a href="naivebayes.html">Naive Bayes background</a> and <a href="bayesian-commandline.html">commandline</a> options.</li>
+ <li><a href="complementary-naive-bayes.html">Complementary naive bayes background</a>, <a href="https://issues.apache.org/jira/browse/mahout-60.html">design</a>, and <a href="c-bayes-commandline.html">c-bayes-commandline</a></li>
+ <li><a href="https://cwiki.apache.org/confluence/display/MAHOUT/Random+Forests">Random Forests Classification</a> comes with a <a href="breiman-example.html">Breiman example</a>. There is some really great documentation
+over at <a href="http://www.markhneedham.com/blog/2012/10/27/kaggle-digit-recognizer-mahout-random-forest-attempt/">Mark Needham’s blog</a>. Also checkout the description on <a href="http://shawnwan.wordpress.com/2012/06/01/mahout-0-7-random-forest-examples/">Xiaomeng Shawn Wan
+s</a> blog.</li>
+</ul>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/collocations.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/collocations.html
new file mode 100644
index 0000000..1028dd8
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/collocations.html
@@ -0,0 +1,567 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Collocations
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/collocations.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="Collocations-CollocationsinMahout"></a></p>
+<h1 id="collocations-in-mahout">Collocations in Mahout</h1>
+
+<p>A collocation is defined as a sequence of words or terms which co-occur
+more often than would be expected by chance. Statistically relevant
+combinations of terms identify additional lexical units which can be
+treated as features in a vector-based representation of a text. A detailed
+discussion of collocations can be found on <a href="http://en.wikipedia.org/wiki/Collocation">Wikipedia</a>.</p>
+
+<p>See there for a more detailed discussion of collocations in the <a href="http://comments.gmane.org/gmane.comp.apache.mahout.user/5685">Reuters example</a>.</p>
+
+<p><a name="Collocations-Log-LikelihoodbasedCollocationIdentification"></a></p>
+<h2 id="theory-behind-implementation-log-likelihood-based-collocation-identification">Theory behind implementation: Log-Likelihood based Collocation Identification</h2>
+
+<p>Mahout provides an implementation of a collocation identification algorithm
+which scores collocations using log-likelihood ratio. The log-likelihood
+score indicates the relative usefulness of a collocation with regards other
+term combinations in the text. Collocations with the highest scores in a
+particular corpus will generally be more useful as features.</p>
+
+<p>Calculating the LLR is very straightforward and is described concisely in
+<a href="http://tdunning.blogspot.com/2008/03/surprise-and-coincidence.html">Ted Dunning’s blog post</a>
+. Ted describes the series of counts reqired to calculate the LLR for two
+events A and B in order to determine if they co-occur more often than pure
+chance. These counts include the number of times the events co-occur (k11),
+the number of times the events occur without each other (k12 and k21), and
+the number of times anything occurs. These counts are summarized in the
+following table:</p>
+
+<table>
+<tr><td> </td><td> Event A </td><td> Everything but Event A </td></tr>
+<tr><td> Event B </td><td> A and B together (k11) </td><td> B but not A (k12) </td></tr>
+<tr><td> Everything but Event B </td><td> A but not B (k21) </td><td> Neither B nor A (k22) </td></tr>
+</table>
+
+<p>For the purposes of collocation identification, it is useful to begin by
+thinking in word pairs, bigrams. In this case the leading or head term from
+the pair corresponds to A from the table above, B corresponds to the
+trailing or tail term, while neither B nor A is the total number of word
+pairs in the corpus less those containing B, A or both B and A.</p>
+
+<p>Given the word pair of ‘oscillation overthruster’, the Log-Likelihood ratio
+is computed by looking at the number of occurences of that word pair in the
+corpus, the number of word pairs that begin with ‘oscillation’ but end with
+something other than ‘overthruster’, the number of word pairs that end with
+‘overthruster’ begin with something other than ‘oscillation’ and the number
+of word pairs in the corpus that contain neither ‘oscillation’ and
+overthruster.</p>
+
+<p>This can be extended from bigrams to trigrams, 4-grams and beyond. In these
+cases, the current algorithm uses the first token of the ngram as the head
+of the ngram and the remaining n-1 tokens from the ngram, the n-1gram as it
+were, as the tail. Given the trigram ‘hong kong cavaliers’, ‘hong’ is
+treated as the head while ‘kong cavaliers’ is treated as the tail. Future
+versions of this algorithm will allow for variations in which tokens of the
+ngram are treated as the head and tail.</p>
+
+<p>Beyond ngrams, it is often useful to inspect cases where individual words
+occur around other interesting features of the text such as sentence
+boundaries.</p>
+
+<p><a name="Collocations-GeneratingNGrams"></a></p>
+<h2 id="generating-ngrams">Generating NGrams</h2>
+
+<p>The tools that the collocation identification algorithm are embeeded within
+either consume tokenized text as input or provide the ability to specify an
+implementation of the Lucene Analyzer class perform tokenization in order
+to form ngrams. The tokens are passed through a Lucene ShingleFilter to
+produce NGrams of the desired length.</p>
+
+<p>Given the text “Alice was beginning to get very tired” as an example,
+Lucene’s StandardAnalyzer produces the tokens ‘alice’, ‘beginning’, ‘get’,
+‘very’ and ‘tired’, while the ShingleFilter with a max NGram size set to 3
+produces the shingles ‘alice beginning’, ‘alice beginning get’, ‘beginning
+get’, ‘beginning get very’, ‘get very’, ‘get very tired’ and ‘very tired’.
+Note that both bigrams and trigrams are produced here. A future enhancement
+to the existing algorithm would involve limiting the output to a particular
+gram size as opposed to solely specifiying a max ngram size.</p>
+
+<p><a name="Collocations-RunningtheCollocationIdentificationAlgorithm."></a></p>
+<h2 id="running-the-collocation-identification-algorithm">Running the Collocation Identification Algorithm.</h2>
+
+<p>There are a couple ways to run the llr-based collocation algorithm in
+mahout</p>
+
+<p><a name="Collocations-Whencreatingvectorsfromasequencefile"></a></p>
+<h3 id="when-creating-vectors-from-a-sequence-file">When creating vectors from a sequence file</h3>
+
+<p>The llr collocation identifier is integrated into the process that is used
+to create vectors from sequence files of text keys and values. Collocations
+are generated when the –maxNGramSize (-ng) option is not specified and
+defaults to 2 or is set to a number of 2 or greater. The –minLLR option
+can be used to control the cutoff that prevents collocations below the
+specified LLR score from being emitted, and the –minSupport argument can
+be used to filter out collocations that appear below a certain number of
+times.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>bin/mahout seq2sparse
+
+Usage:
+ [--minSupport <minSupport> --analyzerName <analyzerName> --chunkSize <chunkSize>
+ --output <output> --input <input> --minDF <minDF>
+ --maxDFPercent<maxDFPercent> --weight <weight> --norm <norm> --minLLR <minLLR>
+ --numReducers <numReducers> --maxNGramSize <ngramSize> --overwrite --help
+ --sequentialAccessVector]
+Options
+
+ --minSupport (-s) minSupport (Optional) Minimum Support. Default Value: 2
+
+ --analyzerName (-a) analyzerName The class name of the analyzer
+
+ --chunkSize (-chunk) chunkSize The chunkSize in MegaBytes. 100-10000MB
+
+ --output (-o) output The output directory
+
+ --input (-i) input Input dir containing the documents in sequence file format
+
+ --minDF (-md) minDF The minimum document frequency. Default is 1
+
+ --maxDFPercent (-x) maxDFPercent The max percentage of docs for the DF. Can be used to remove
+ really high frequency terms. Expressed as an
+ integer between 0 and 100. Default is 99.
+
+ --weight (-wt) weight The kind of weight to use. Currently TF
+ or TFIDF
+
+ --norm (-n) norm The norm to use, expressed as either a
+ float or "INF" if you want to use the
+ Infinite norm. Must be greater orequal
+ to 0. The default is not to normalize
+
+ --minLLR (-ml) minLLR (Optional)The minimum Log Likelihood
+ Ratio(Float) Default is 1.0
+
+ --numReducers (-nr) numReducers (Optional) Number of reduce tasks.
+ Default Value: 1
+
+ --maxNGramSize (-ng) ngramSize (Optional) The maximum size of ngrams to
+ create (2 = bigrams, 3 = trigrams, etc)
+ Default Value:2
+
+ --overwrite (-w) If set, overwrite the output directory
+ --help (-h) Print out help
+ --sequentialAccessVector (-seq) (Optional) Whether output vectors should
+ be SequentialAccessVectors If set true
+ else false
+</code></pre>
+</div>
+
+<p><a name="Collocations-CollocDriver"></a></p>
+<h3 id="collocdriver">CollocDriver</h3>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>bin/mahout org.apache.mahout.vectorizer.collocations.llr.CollocDriver
+
+Usage:
+ [--input <input> --output <output> --maxNGramSize <ngramSize> --overwrite
+--minSupport <minSupport> --minLLR <minLLR> --numReducers <numReducers>
+--analyzerName <analyzerName> --preprocess --unigram --help]
+
+Options
+
+ --input (-i) input The Path for input files.
+
+ --output (-o) output The Path write output to
+
+ --maxNGramSize (-ng) ngramSize (Optional) The maximum size of ngramsto
+ create (2 = bigrams, 3 = trigrams,etc)
+ Default Value:2
+
+ --overwrite (-w) If set, overwrite the outputdirectory
+
+ --minSupport (-s) minSupport (Optional) Minimum Support. Default
+ Value: 2
+
+ --minLLR (-ml) minLLR (Optional)The minimum Log Likelihood
+ Ratio(Float) Default is 1.0
+
+ --numReducers (-nr) numReducers (Optional) Number of reduce tasks.
+ Default Value: 1
+
+ --analyzerName (-a) analyzerName The class name of the analyzer
+
+ --preprocess (-p) If set, input is SequenceFile<Text,Text>
+ where the value is the document, which
+ will be tokenized using the specified
+ analyzer.
+
+ --unigram (-u) If set, unigrams will be emitted inthe
+ final output alongside collocations
+
+ --help (-h) Print out help
+</code></pre>
+</div>
+
+<p><a name="Collocations-Algorithmdetails"></a></p>
+<h2 id="algorithm-details">Algorithm details</h2>
+
+<p>This section describes the implementation of the collocation identification
+algorithm in terms of the map-reduce phases that are used to generate
+ngrams and count the frequencies required to perform the log-likelihood
+calculation. Unless otherwise noted, classes that are indicated in
+CamelCase can be found in the mahout-utils module under the package
+org.apache.mahout.utils.nlp.collocations.llr</p>
+
+<p>The algorithm is implemented in two map-reduce passes:</p>
+
+<p><a name="Collocations-Pass1:CollocDriver.generateCollocations(...)"></a></p>
+<h3 id="pass-1-collocdrivergeneratecollocations">Pass 1: CollocDriver.generateCollocations(…)</h3>
+
+<p>Generates NGrams and counts frequencies for ngrams, head and tail subgrams.</p>
+
+<p><a name="Collocations-Map:CollocMapper"></a></p>
+<h4 id="map-collocmapper">Map: CollocMapper</h4>
+
+<p>Input k: Text (documentId), v: StringTuple (tokens)</p>
+
+<p>Each call to the mapper passes in the full set of tokens for the
+corresponding document using a StringTuple. The ShingleFilter is run across
+these tokens to produce ngrams of the desired length. ngrams and
+frequencies are collected across the entire document.</p>
+
+<p>Once this is done, ngrams are split into head and tail portions. A key of type GramKey is generated which is used later to join ngrams with their heads and tails in the reducer phase. The GramKey is a composite key made up of a string n-gram fragement as the primary key and a secondary key used for grouping and sorting in the reduce phase. The secondary key will either be EMPTY in the case where we are collecting either the head or tail of an ngram as the value or it will contain the [...]
+ form of the ngram when collecting an ngram as the value.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>head_key(EMPTY) -> (head subgram, head frequency)
+
+head_key(ngram) -> (ngram, ngram frequency)
+
+tail_key(EMPTY) -> (tail subgram, tail frequency)
+
+tail_key(ngram) -> (ngram, ngram frequency)
+</code></pre>
+</div>
+
+<p>subgram and ngram values are packaged in Gram objects.</p>
+
+<p>For each ngram found, the Count.NGRAM_TOTAL counter is incremented. When
+the pass is complete, this counter will hold the total number of ngrams
+encountered in the input which is used as a part of the LLR calculation.</p>
+
+<p>Output k: GramKey (head or tail subgram), v: Gram (head, tail or ngram with
+frequency)</p>
+
+<p><a name="Collocations-Combiner:CollocCombiner"></a></p>
+<h4 id="combiner-colloccombiner">Combiner: CollocCombiner</h4>
+
+<p>Input k: GramKey, v:Gram (as above)</p>
+
+<p>This phase merges the counts for unique ngrams or ngram fragments across
+multiple documents. The combiner treats the entire GramKey as the key and
+as such, identical tuples from separate documents are passed into a single
+call to the combiner’s reduce method, their frequencies are summed and a
+single tuple is passed out via the collector.</p>
+
+<p>Output k: GramKey, v:Gram</p>
+
+<p><a name="Collocations-Reduce:CollocReducer"></a></p>
+<h4 id="reduce-collocreducer">Reduce: CollocReducer</h4>
+
+<p>Input k: GramKey, v: Gram (as above)</p>
+
+<p>The CollocReducer employs the Hadoop secondary sort strategy to avoid
+caching ngram tuples in memory in order to calculate total ngram and
+subgram frequencies. The GramKeyPartitioner ensures that tuples with the
+same primary key are sent to the same reducer while the
+GramKeyGroupComparator ensures that iterator provided by the reduce method
+first returns the subgram and then returns ngram values grouped by ngram.
+This eliminates the need to cache the values returned by the iterator in
+order to calculate total frequencies for both subgrams and ngrams. There
+input will consist of multiple frequencies for each (subgram_key, subgram)
+or (subgram_key, ngram) tuple; one from each map task executed in which the
+particular subgram was found.
+The input will be traversed in the following order:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>(head subgram, frequency 1)
+(head subgram, frequency 2)
+...
+(head subgram, frequency N)
+(ngram 1, frequency 1)
+(ngram 1, frequency 2)
+...
+(ngram 1, frequency N)
+(ngram 2, frequency 1)
+(ngram 2, frequency 2)
+...
+(ngram 2, frequency N)
+...
+(ngram N, frequency 1)
+(ngram N, frequency 2)
+...
+(ngram N, frequency N)
+</code></pre>
+</div>
+
+<p>Where all of the ngrams above share the same head. Data is presented in the
+same manner for the tail subgrams.</p>
+
+<p>As the values for a subgram or ngram are traversed, frequencies are
+accumulated. Once all values for a subgram or ngram are processed the
+resulting key/value pairs are passed to the collector as long as the ngram
+frequency is equal to or greater than the specified minSupport. When an
+ngram is skipped in this way the Skipped.LESS_THAN_MIN_SUPPORT counter to
+be incremented.</p>
+
+<p>Pairs are passed to the collector in the following format:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>ngram, ngram frequency -> subgram subgram frequency
+</code></pre>
+</div>
+
+<p>In this manner, the output becomes an unsorted version of the following:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>ngram 1, frequency -> ngram 1 head, head frequency
+ngram 1, frequency -> ngram 1 tail, tail frequency
+ngram 2, frequency -> ngram 2 head, head frequency
+ngram 2, frequency -> ngram 2 tail, tail frequency
+ngram N, frequency -> ngram N head, head frequency
+ngram N, frequency -> ngram N tail, tail frequency
+</code></pre>
+</div>
+
+<p>Output is in the format k:Gram (ngram, frequency), v:Gram (subgram,
+frequency)</p>
+
+<p><a name="Collocations-Pass2:CollocDriver.computeNGramsPruneByLLR(...)"></a></p>
+<h3 id="pass-2-collocdrivercomputengramsprunebyllr">Pass 2: CollocDriver.computeNGramsPruneByLLR(…)</h3>
+
+<p>Pass 1 has calculated full frequencies for ngrams and subgrams, Pass 2
+performs the LLR calculation.</p>
+
+<p><a name="Collocations-MapPhase:IdentityMapper(org.apache.hadoop.mapred.lib.IdentityMapper)"></a></p>
+<h4 id="map-phase-identitymapper-orgapachehadoopmapredlibidentitymapper">Map Phase: IdentityMapper (org.apache.hadoop.mapred.lib.IdentityMapper)</h4>
+
+<p>This phase is a no-op. The data is passed through unchanged. The rest of
+the work for llr calculation is done in the reduce phase.</p>
+
+<p><a name="Collocations-ReducePhase:LLRReducer"></a></p>
+<h4 id="reduce-phase-llrreducer">Reduce Phase: LLRReducer</h4>
+
+<p>Input is k:Gram, v:Gram (as above)</p>
+
+<p>This phase receives the head and tail subgrams and their frequencies for
+each ngram (with frequency) produced for the input:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>ngram 1, frequency -> ngram 1 head, frequency; ngram 1 tail, frequency
+ngram 2, frequency -> ngram 2 head, frequency; ngram 2 tail, frequency
+...
+ngram 1, frequency -> ngram N head, frequency; ngram N tail, frequency
+</code></pre>
+</div>
+
+<p>It also reads the full ngram count obtained from the first pass, passed in
+as a configuration option. The parameters to the llr calculation are
+calculated as follows:</p>
+
+<p>k11 = f_n
+k12 = f_h - f_n
+k21 = f_t - f_n
+k22 = N - ((f_h + f_t) - f_n)</p>
+
+<p>Where f_n is the ngram frequency, f_h and f_t the frequency of head and
+tail and N is the total number of ngrams.</p>
+
+<p>Tokens with a llr below that of the specified minimum llr are dropped and
+the Skipped.LESS_THAN_MIN_LLR counter is incremented.</p>
+
+<p>Output is k: Text (ngram), v: DoubleWritable (llr score)</p>
+
+<p><a name="Collocations-Unigrampass-through."></a></p>
+<h3 id="unigram-pass-through">Unigram pass-through.</h3>
+
+<p>By default in seq2sparse, or if the -u option is provided to the
+CollocDriver, unigrams (single tokens) will be passed through the job and
+each token’s frequency will be calculated. As with ngrams, unigrams are
+subject to filtering with minSupport and minLLR.</p>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/gaussian-discriminative-analysis.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/gaussian-discriminative-analysis.html
new file mode 100644
index 0000000..e4fbfaa
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/gaussian-discriminative-analysis.html
@@ -0,0 +1,204 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Gaussian Discriminative Analysis
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/gaussian-discriminative-analysis.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="GaussianDiscriminativeAnalysis-GaussianDiscriminativeAnalysis"></a></p>
+<h1 id="gaussian-discriminative-analysis">Gaussian Discriminative Analysis</h1>
+
+<p>Gaussian Discriminative Analysis is a tool for multigroup classification
+based on extending linear discriminant analysis. The paper on the approach
+is located at http://citeseer.ist.psu.edu/4617.html (note, for some reason
+the paper is backwards, in that page 1 is at the end)</p>
+
+<p><a name="GaussianDiscriminativeAnalysis-Parallelizationstrategy"></a></p>
+<h2 id="parallelization-strategy">Parallelization strategy</h2>
+
+<p><a name="GaussianDiscriminativeAnalysis-Designofpackages"></a></p>
+<h2 id="design-of-packages">Design of packages</h2>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/hidden-markov-models.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/hidden-markov-models.html
new file mode 100644
index 0000000..54a3580
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/hidden-markov-models.html
@@ -0,0 +1,299 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Hidden Markov Models
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/hidden-markov-models.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h1 id="hidden-markov-models">Hidden Markov Models</h1>
+
+<p><a name="HiddenMarkovModels-IntroductionandUsage"></a></p>
+<h2 id="introduction-and-usage">Introduction and Usage</h2>
+
+<p>Hidden Markov Models are used in multiple areas of Machine Learning, such
+as speech recognition, handwritten letter recognition or natural language
+processing.</p>
+
+<p><a name="HiddenMarkovModels-FormalDefinition"></a></p>
+<h2 id="formal-definition">Formal Definition</h2>
+
+<p>A Hidden Markov Model (HMM) is a statistical model of a process consisting
+of two (in our case discrete) random variables O and Y, which change their
+state sequentially. The variable Y with states {y_1, … , y_n} is called
+the “hidden variable”, since its state is not directly observable. The
+state of Y changes sequentially with a so called - in our case first-order</p>
+<ul>
+ <li>Markov Property. This means, that the state change probability of Y only
+depends on its current state and does not change in time. Formally we
+write: P(Y(t+1)=y_i|Y(0)…Y(t)) = P(Y(t+1)=y_i|Y(t)) = P(Y(2)=y_i|Y(1)).
+The variable O with states {o_1, … , o_m} is called the “observable
+variable”, since its state can be directly observed. O does not have a
+Markov Property, but its state probability depends statically on the
+current state of Y.</li>
+</ul>
+
+<p>Formally, an HMM is defined as a tuple M=(n,m,P,A,B), where n is the number of hidden states, m is the number of observable states, P is an n-dimensional vector containing initial hidden state probabilities, A is the nxn-dimensional “transition matrix” containing the transition probabilities such that A[i,j](i,j.html)
+=P(Y(t)=y_i|Y(t-1)=y_j) and B is the mxn-dimensional “emission matrix”
+containing the observation probabilities such that B[i,j]=
+P(O=o_i|Y=y_j).</p>
+
+<p><a name="HiddenMarkovModels-Problems"></a></p>
+<h2 id="problems">Problems</h2>
+
+<p>Rabiner [1](1.html)
+ defined three main problems for HMM models:</p>
+
+<ol>
+ <li>Evaluation: Given a sequence O of observations and a model M, what is
+the probability P(O|M) that sequence O was generated by model M. The
+Evaluation problem can be efficiently solved using the Forward algorithm</li>
+ <li>Decoding: Given a sequence O of observations and a model M, what is
+the most likely sequence Y*=argmax(Y) P(O|M,Y) of hidden variables to
+generate this sequence. The Decoding problem can be efficiently solved
+using the Viterbi algorithm.</li>
+ <li>Learning: Given a sequence O of observations, what is the most likely
+model M*=argmax(M)P(O|M) to generate this sequence. The Learning problem
+can be efficiently solved using the Baum-Welch algorithm.</li>
+</ol>
+
+<p><a name="HiddenMarkovModels-Example"></a></p>
+<h2 id="example">Example</h2>
+
+<p>To build a Hidden Markov Model and use it to build some predictions, try a simple example like this:</p>
+
+<p>Create an input file to train the model. Here we have a sequence drawn from the set of states 0, 1, 2, and 3, separated by space characters.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>$ echo "0 1 2 2 2 1 1 0 0 3 3 3 2 1 2 1 1 1 1 2 2 2 0 0 0 0 0 0 2 2 2 0 0 0 0 0 0 2 2 2 3 3 3 3 3 3 2 3 2 3 2 3 2 1 3 0 0 0 1 0 1 0 2 1 2 1 2 1 2 3 3 3 3 2 2 3 2 1 1 0" > hmm-input
+</code></pre>
+</div>
+
+<p>Now run the baumwelch job to train your model, after first setting MAHOUT_LOCAL to true, to use your local file system.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>$ export MAHOUT_LOCAL=true
+$ $MAHOUT_HOME/bin/mahout baumwelch -i hmm-input -o hmm-model -nh 3 -no 4 -e .0001 -m 1000
+</code></pre>
+</div>
+
+<p>Output like the following should appear in the console.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>Initial probabilities:
+0 1 2
+1.0 0.0 3.5659361683006626E-251
+Transition matrix:
+ 0 1 2
+0 6.098919959130616E-5 0.9997275322964165 2.1147850399214744E-4
+1 7.404648706054873E-37 0.9086408633885092 0.09135913661149081
+2 0.2284374545687356 7.01786289571088E-11 0.7715625453610858
+Emission matrix:
+ 0 1 2 3
+0 0.9999997858591223 2.0536163836449762E-39 2.1414087769942127E-7 1.052441093535389E-27
+1 7.495656581383351E-34 0.2241269055449904 0.4510889999455847 0.32478409450942497
+2 0.815051477991782 0.18494852200821799 8.465660634827592E-33 2.8603899591778015E-36
+14/03/22 09:52:21 INFO driver.MahoutDriver: Program took 180 ms (Minutes: 0.003)
+</code></pre>
+</div>
+
+<p>The model trained with the input set now is in the file ‘hmm-model’, which we can use to build a predicted sequence.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>$ $MAHOUT_HOME/bin/mahout hmmpredict -m hmm-model -o hmm-predictions -l 10
+</code></pre>
+</div>
+
+<p>To see the predictions:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>$ cat hmm-predictions
+0 1 3 3 2 2 2 2 1 2
+</code></pre>
+</div>
+
+<p><a name="HiddenMarkovModels-Resources"></a></p>
+<h2 id="resources">Resources</h2>
+
+<p>[1]
+ Lawrence R. Rabiner (February 1989). “A tutorial on Hidden Markov Models
+and selected applications in speech recognition”. Proceedings of the IEEE
+77 (2): 257-286. doi:10.1109/5.18626.</p>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/independent-component-analysis.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/independent-component-analysis.html
new file mode 100644
index 0000000..8423565
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/independent-component-analysis.html
@@ -0,0 +1,201 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Independent Component Analysis
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/independent-component-analysis.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="IndependentComponentAnalysis-IndependentComponentAnalysis"></a></p>
+<h1 id="independent-component-analysis">Independent Component Analysis</h1>
+
+<p>See also: Principal Component Analysis.</p>
+
+<p><a name="IndependentComponentAnalysis-Parallelizationstrategy"></a></p>
+<h2 id="parallelization-strategy">Parallelization strategy</h2>
+
+<p><a name="IndependentComponentAnalysis-Designofpackages"></a></p>
+<h2 id="design-of-packages">Design of packages</h2>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/locally-weighted-linear-regression.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/locally-weighted-linear-regression.html
new file mode 100644
index 0000000..3e0a6d6
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/locally-weighted-linear-regression.html
@@ -0,0 +1,209 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Locally Weighted Linear Regression
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/locally-weighted-linear-regression.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="LocallyWeightedLinearRegression-LocallyWeightedLinearRegression"></a></p>
+<h1 id="locally-weighted-linear-regression">Locally Weighted Linear Regression</h1>
+
+<p>Model-based methods, such as SVM, Naive Bayes and the mixture of Gaussians,
+use the data to build a parameterized model. After training, the model is
+used for predictions and the data are generally discarded. In contrast,
+“memory-based” methods are non-parametric approaches that explicitly retain
+the training data, and use it each time a prediction needs to be made.
+Locally weighted regression (LWR) is a memory-based method that performs a
+regression around a point of interest using only training data that are
+“local” to that point. Source:
+http://www.cs.cmu.edu/afs/cs/project/jair/pub/volume4/cohn96a-html/node7.html</p>
+
+<p><a name="LocallyWeightedLinearRegression-Strategyforparallelregression"></a></p>
+<h2 id="strategy-for-parallel-regression">Strategy for parallel regression</h2>
+
+<p><a name="LocallyWeightedLinearRegression-Designofpackages"></a></p>
+<h2 id="design-of-packages">Design of packages</h2>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/logistic-regression.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/logistic-regression.html
new file mode 100644
index 0000000..0631e93
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/logistic-regression.html
@@ -0,0 +1,319 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Logistic Regression
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/logistic-regression.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="LogisticRegression-LogisticRegression(SGD)"></a></p>
+<h1 id="logistic-regression-sgd">Logistic Regression (SGD)</h1>
+
+<p>Logistic regression is a model used for prediction of the probability of
+occurrence of an event. It makes use of several predictor variables that
+may be either numerical or categories.</p>
+
+<p>Logistic regression is the standard industry workhorse that underlies many
+production fraud detection and advertising quality and targeting products.
+The Mahout implementation uses Stochastic Gradient Descent (SGD) to all
+large training sets to be used.</p>
+
+<p>For a more detailed analysis of the approach, have a look at the <a href="http://repository.cmu.edu/cgi/viewcontent.cgi?article=1221&context=robotics">thesis of
+Paul Komarek</a> [1].</p>
+
+<p>See MAHOUT-228 for the main JIRA issue for SGD.</p>
+
+<p>A more detailed overview of the Mahout Linear Regression classifier and <a href="http://blog.trifork.com/2014/02/04/an-introduction-to-mahouts-logistic-regression-sgd-classifier/">detailed discription of building a Logistic Regression classifier</a> for the classic <a href="http://en.wikipedia.org/wiki/Iris_flower_data_set">Iris flower dataset</a> is also available [2].</p>
+
+<p>An example of training a Logistic Regression classifier for the <a href="http://mlr.cs.umass.edu/ml/datasets/Bank+Marketing">UCI Bank Marketing Dataset</a> can be found <a href="http://mahout.apache.org/users/classification/bankmarketing-example.html">on the Mahout website</a> [3].</p>
+
+<p>An example of training and testing a Logistic Regression document classifier for the classic <a href="https://github.com/apache/mahout/blob/master/examples/bin/classify-20newsgroups.sh">20 newsgroups corpus</a> [4] is also available.</p>
+
+<p><a name="LogisticRegression-Parallelizationstrategy"></a></p>
+<h2 id="parallelization-strategy">Parallelization strategy</h2>
+
+<p>The bad news is that SGD is an inherently sequential algorithm. The good
+news is that it is blazingly fast and thus it is not a problem for Mahout’s
+implementation to handle training sets of tens of millions of examples.
+With the down-sampling typical in many data-sets, this is equivalent to a
+dataset with billions of raw training examples.</p>
+
+<p>The SGD system in Mahout is an online learning algorithm which means that
+you can learn models in an incremental fashion and that you can do
+performance testing as your system runs. Often this means that you can
+stop training when a model reaches a target level of performance. The SGD
+framework includes classes to do on-line evaluation using cross validation
+(the CrossFoldLearner) and an evolutionary system to do learning
+hyper-parameter optimization on the fly (the AdaptiveLogisticRegression).
+The AdaptiveLogisticRegression system makes heavy use of threads to
+increase machine utilization. The way it works is that it runs 20
+CrossFoldLearners in separate threads, each with slightly different
+learning parameters. As better settings are found, these new settings are
+propagating to the other learners.</p>
+
+<p><a name="LogisticRegression-Designofpackages"></a></p>
+<h2 id="design-of-packages">Design of packages</h2>
+
+<p>There are three packages that are used in Mahout’s SGD system. These
+include</p>
+
+<ul>
+ <li>
+ <p>The vector encoding package (found in org.apache.mahout.vectorizer.encoders)</p>
+ </li>
+ <li>
+ <p>The SGD learning package (found in org.apache.mahout.classifier.sgd)</p>
+ </li>
+ <li>
+ <p>The evolutionary optimization system (found in org.apache.mahout.ep)</p>
+ </li>
+</ul>
+
+<p><a name="LogisticRegression-Featurevectorencoding"></a></p>
+<h2 id="feature-vector-encoding">Feature vector encoding</h2>
+
+<p>Because the SGD algorithms need to have fixed length feature vectors and
+because it is a pain to build a dictionary ahead of time, most SGD
+applications use the hashed feature vector encoding system that is rooted
+at FeatureVectorEncoder.</p>
+
+<p>The basic idea is that you create a vector, typically a
+RandomAccessSparseVector, and then you use various feature encoders to
+progressively add features to that vector. The size of the vector should
+be large enough to avoid feature collisions as features are hashed.</p>
+
+<p>There are specialized encoders for a variety of data types. You can
+normally encode either a string representation of the value you want to
+encode or you can encode a byte level representation to avoid string
+conversion. In the case of ContinuousValueEncoder and
+ConstantValueEncoder, it is also possible to encode a null value and pass
+the real value in as a weight. This avoids numerical parsing entirely in
+case you are getting your training data from a system like Avro.</p>
+
+<p>Here is a class diagram for the encoders package:</p>
+
+<p><img src="../../images/vector-class-hierarchy.png" alt="class diagram" /></p>
+
+<p><a name="LogisticRegression-SGDLearning"></a></p>
+<h2 id="sgd-learning">SGD Learning</h2>
+
+<p>For the simplest applications, you can construct an
+OnlineLogisticRegression and be off and running. Typically, though, it is
+nice to have running estimates of performance on held out data. To do
+that, you should use a CrossFoldLearner which keeps a stable of five (by
+default) OnlineLogisticRegression objects. Each time you pass a training
+example to a CrossFoldLearner, it passes this example to all but one of its
+children as training and passes the example to the last child to evaluate
+current performance. The children are used for evaluation in a round-robin
+fashion so, if you are using the default 5 way split, all of the children
+get 80% of the training data for training and get 20% of the data for
+evaluation.</p>
+
+<p>To avoid the pesky need to configure learning rates, regularization
+parameters and annealing schedules, you can use the
+AdaptiveLogisticRegression. This class maintains a pool of
+CrossFoldLearners and adapts learning rates and regularization on the fly
+so that you don’t have to.</p>
+
+<p>Here is a class diagram for the classifiers.sgd package. As you can see,
+the number of twiddlable knobs is pretty large. For some examples, see the
+<a href="https://github.com/apache/mahout/blob/master/examples/src/main/java/org/apache/mahout/classifier/sgd/TrainNewsGroups.java">TrainNewsGroups</a> example code.</p>
+
+<p><img src="../../images/sgd-class-hierarchy.png" alt="sgd class diagram" /></p>
+
+<h2 id="references">References</h2>
+
+<p>[1] <a href="http://repository.cmu.edu/cgi/viewcontent.cgi?article=1221&context=robotics">Thesis of
+Paul Komarek</a></p>
+
+<p>[2] <a href="http://blog.trifork.com/2014/02/04/an-introduction-to-mahouts-logistic-regression-sgd-classifier/">An Introduction To Mahout’s Logistic Regression SGD Classifier</a></p>
+
+<h2 id="examples">Examples</h2>
+
+<p>[3] <a href="http://mahout.apache.org/users/classification/bankmarketing-example.html">SGD Bank Marketing Example</a></p>
+
+<p>[4] <a href="https://github.com/apache/mahout/blob/master/examples/bin/classify-20newsgroups.sh">SGD 20 newsgroups classification</a></p>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/mahout-collections.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/mahout-collections.html
new file mode 100644
index 0000000..ea95bb5
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/mahout-collections.html
@@ -0,0 +1,239 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) mahout-collections
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/mahout-collections.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h1 id="mahout-collections">Mahout collections</h1>
+
+<p><a name="mahout-collections-Introduction"></a></p>
+<h2 id="introduction">Introduction</h2>
+
+<p>The Mahout Collections library is a set of container classes that address
+some limitations of the standard collections in Java. <a href="http://domino.research.ibm.com/comm/research_people.nsf/pages/sevitsky.pubs.html/$FILE/oopsla08%20memory-efficient%20java%20slides.pdf">This presentation</a>
+ describes a number of performance problems with the standard collections.</p>
+
+<p>Mahout collections addresses two of the more glaring: the lack of support
+for primitive types and the lack of open hashing.</p>
+
+<p><a name="mahout-collections-PrimitiveTypes"></a></p>
+<h2 id="primitive-types">Primitive Types</h2>
+
+<p>The most visible feature of Mahout Collections is the large collection of
+primitive type collections. Given Java’s asymmetrical support for the
+primitive types, the only efficient way to handle them is with many
+classes. So, there are ArrayList-like containers for all of the primitive
+types, and hash maps for all the useful combinations of primitive type and
+object keys and values.</p>
+
+<p>These classes do not, in general, implement interfaces from <em>java.util</em>.
+Even when the <em>java.util</em> interfaces could be type-compatible, they tend
+to include requirements that are not consistent with efficient use of
+primitive types.</p>
+
+<p><a name="mahout-collections-OpenAddressing"></a></p>
+<h1 id="open-addressing">Open Addressing</h1>
+
+<p>All of the sets and maps in Mahout Collections are open-addressed hash
+tables. Open addressing has a much smaller memory footprint than chaining.
+Since the purpose of these collections is to avoid the memory cost of
+autoboxing, open addressing is a consistent design choice.</p>
+
+<p><a name="mahout-collections-Sets"></a></p>
+<h2 id="sets">Sets</h2>
+
+<p>Mahout Collections includes open hash sets. Unlike <em>java.util</em>, a set is
+not a recycled hash table; the sets are separately implemented and do not
+have any additional storage usage for unused keys.</p>
+
+<p><a name="mahout-collections-CreditwhereCreditisdue"></a></p>
+<h1 id="credit-where-credit-is-due">Credit where Credit is due</h1>
+
+<p>The implementation of Mahout Collections is derived from <a href="http://acs.lbl.gov/~hoschek/colt/">Cern Colt</a>
+.</p>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/mlp.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/mlp.html
new file mode 100644
index 0000000..e031f5d
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/mlp.html
@@ -0,0 +1,430 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Multilayer Perceptron
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/mlp.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h1 id="multilayer-perceptron">Multilayer Perceptron</h1>
+
+<p>A multilayer perceptron is a biologically inspired feed-forward network that can
+be trained to represent a nonlinear mapping between input and output data. It
+consists of multiple layers, each containing multiple artificial neuron units and
+can be used for classification and regression tasks in a supervised learning approach.</p>
+
+<h2 id="command-line-usage">Command line usage</h2>
+
+<p>The MLP implementation is currently located in the MapReduce-Legacy package. It
+can be used with the following commands:</p>
+
+<h1 id="model-training">model training</h1>
+<div class="highlighter-rouge"><pre class="highlight"><code>$ bin/mahout org.apache.mahout.classifier.mlp.TrainMultilayerPerceptron # model usage
+$ bin/mahout org.apache.mahout.classifier.mlp.RunMultilayerPerceptron
+</code></pre>
+</div>
+
+<p>To train and use the model, a number of parameters can be specified. Parameters without default values have to be specified by the user. Consider that not all parameters can be used both for training and running the model. We give an example of the usage below.</p>
+
+<h3 id="parameters">Parameters</h3>
+
+<table>
+ <thead>
+ <tr>
+ <th style="text-align: left">Command</th>
+ <th style="text-align: right">Default</th>
+ <th style="text-align: left">Description</th>
+ <th style="text-align: left">Type</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td style="text-align: left">–input -i</td>
+ <td style="text-align: right"> </td>
+ <td style="text-align: left">Path to the input data (currently, only .csv-files are allowed)</td>
+ <td style="text-align: left"> </td>
+ </tr>
+ <tr>
+ <td style="text-align: left">–skipHeader -sh</td>
+ <td style="text-align: right">false</td>
+ <td style="text-align: left">Skip first row of the input file (corresponds to the csv headers)</td>
+ <td style="text-align: left"> </td>
+ </tr>
+ <tr>
+ <td style="text-align: left">–update -u</td>
+ <td style="text-align: right">false</td>
+ <td style="text-align: left">Whether the model should be updated incrementally with every new training instance. If this parameter is not given, the model is trained from scratch.</td>
+ <td style="text-align: left">training</td>
+ </tr>
+ <tr>
+ <td style="text-align: left">–labels -labels</td>
+ <td style="text-align: right"> </td>
+ <td style="text-align: left">Instance labels separated by whitespaces.</td>
+ <td style="text-align: left">training</td>
+ </tr>
+ <tr>
+ <td style="text-align: left">–model -mo</td>
+ <td style="text-align: right"> </td>
+ <td style="text-align: left">Location where the model will be stored / is stored (if the specified location has an existing model, it will update the model through incremental learning).</td>
+ <td style="text-align: left"> </td>
+ </tr>
+ <tr>
+ <td style="text-align: left">–layerSize -ls</td>
+ <td style="text-align: right"> </td>
+ <td style="text-align: left">Number of units per layer, including input, hidden and ouput layers. This parameter specifies the topology of the network (see <a href="mlperceptron_structure.png" title="Architecture of a three-layer MLP">this image</a> for an example specified by <code class="highlighter-rouge">-ls 4 8 3</code>).</td>
+ <td style="text-align: left">training</td>
+ </tr>
+ <tr>
+ <td style="text-align: left">–squashingFunction -sf</td>
+ <td style="text-align: right">Sigmoid</td>
+ <td style="text-align: left">The squashing function to use for the units. Currently only the sigmoid fucntion is available.</td>
+ <td style="text-align: left">training</td>
+ </tr>
+ <tr>
+ <td style="text-align: left">–learningRate -l</td>
+ <td style="text-align: right">0.5</td>
+ <td style="text-align: left">The learning rate that is used for weight updates.</td>
+ <td style="text-align: left">training</td>
+ </tr>
+ <tr>
+ <td style="text-align: left">–momemtumWeight -m</td>
+ <td style="text-align: right">0.1</td>
+ <td style="text-align: left">The momentum weight that is used for gradient descent. Must be in the range between 0 … 1.0</td>
+ <td style="text-align: left">training</td>
+ </tr>
+ <tr>
+ <td style="text-align: left">–regularizationWeight -r</td>
+ <td style="text-align: right">0</td>
+ <td style="text-align: left">Regularization value for the weight vector. Must be in the range between 0 … 0.1</td>
+ <td style="text-align: left">training</td>
+ </tr>
+ <tr>
+ <td style="text-align: left">–format -f</td>
+ <td style="text-align: right">csv</td>
+ <td style="text-align: left">Input file format. Currently only csv is supported.</td>
+ <td style="text-align: left"> </td>
+ </tr>
+ <tr>
+ <td style="text-align: left">–columnRange -cr</td>
+ <td style="text-align: right"> </td>
+ <td style="text-align: left">Range of the columns to use from the input file, starting with 0 (i.e. <code class="highlighter-rouge">-cr 0 5</code> for including the first six columns only)</td>
+ <td style="text-align: left">testing</td>
+ </tr>
+ <tr>
+ <td style="text-align: left">–output -o</td>
+ <td style="text-align: right"> </td>
+ <td style="text-align: left">Path to store the labeled results from running the model.</td>
+ <td style="text-align: left">testing</td>
+ </tr>
+ </tbody>
+</table>
+
+<h2 id="example-usage">Example usage</h2>
+
+<p>In this example, we will train a multilayer perceptron for classification on the iris data set. The iris flower data set contains data of three flower species where each datapoint consists of four features.
+The dimensions of the data set are given through some flower parameters (sepal length, sepal width, …). All samples contain a label that indicates the flower species they belong to.</p>
+
+<h3 id="training">Training</h3>
+
+<p>To train our multilayer perceptron model from the command line, we call the following command</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>$ bin/mahout org.apache.mahout.classifier.mlp.TrainMultilayerPerceptron \
+ -i ./mrlegacy/src/test/resources/iris.csv -sh \
+ -labels setosa versicolor virginica \
+ -mo /tmp/model.model -ls 4 8 3 -l 0.2 -m 0.35 -r 0.0001
+</code></pre>
+</div>
+
+<p>The individual parameters are explained in the following.</p>
+
+<ul>
+ <li><code class="highlighter-rouge">-i ./mrlegacy/src/test/resources/iris.csv</code> use the iris data set as input data</li>
+ <li><code class="highlighter-rouge">-sh</code> since the file <code class="highlighter-rouge">iris.csv</code> contains a header row, this row needs to be skipped</li>
+ <li><code class="highlighter-rouge">-labels setosa versicolor virginica</code> we specify, which class labels should be learnt (which are the flower species in this case)</li>
+ <li><code class="highlighter-rouge">-mo /tmp/model.model</code> specify where to store the model file</li>
+ <li><code class="highlighter-rouge">-ls 4 8 3</code> we specify the structure and depth of our layers. The actual network structure can be seen in the figure below.</li>
+ <li><code class="highlighter-rouge">-l 0.2</code> we set the learning rate to <code class="highlighter-rouge">0.2</code></li>
+ <li><code class="highlighter-rouge">-m 0.35</code> momemtum weight is set to <code class="highlighter-rouge">0.35</code></li>
+ <li><code class="highlighter-rouge">-r 0.0001</code> regularization weight is set to <code class="highlighter-rouge">0.0001</code></li>
+</ul>
+
+<table>
+ <thead>
+ <tr>
+ <th> </th>
+ <th> </th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td>The picture shows the architecture defined by the above command. The topolgy of the network is completely defined through the number of layers and units because in this implementation of the MLP every unit is fully connected to the units of the next and previous layer. Bias units are added automatically.</td>
+ <td><img src="mlperceptron_structure.png" alt="Multilayer perceptron network" title="Architecture of a three-layer MLP" /></td>
+ </tr>
+ </tbody>
+</table>
+
+<h3 id="testing">Testing</h3>
+
+<p>To test / run the multilayer perceptron classification on the trained model, we can use the following command</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>$ bin/mahout org.apache.mahout.classifier.mlp.RunMultilayerPerceptron \
+ -i ./mrlegacy/src/test/resources/iris.csv -sh -cr 0 3 \
+ -mo /tmp/model.model -o /tmp/labelResult.txt
+</code></pre>
+</div>
+
+<p>The individual parameters are explained in the following.</p>
+
+<ul>
+ <li><code class="highlighter-rouge">-i ./mrlegacy/src/test/resources/iris.csv</code> use the iris data set as input data</li>
+ <li><code class="highlighter-rouge">-sh</code> since the file <code class="highlighter-rouge">iris.csv</code> contains a header row, this row needs to be skipped</li>
+ <li><code class="highlighter-rouge">-cr 0 3</code> we specify the column range of the input file</li>
+ <li><code class="highlighter-rouge">-mo /tmp/model.model</code> specify where the model file is stored</li>
+ <li><code class="highlighter-rouge">-o /tmp/labelResult.txt</code> specify where the labeled output file will be stored</li>
+</ul>
+
+<h2 id="implementation">Implementation</h2>
+
+<p>The Multilayer Perceptron implementation is based on a more general Neural Network class. Command line support was added later on and provides a simple usage of the MLP as shown in the example. It is implemented to run on a single machine using stochastic gradient descent where the weights are updated using one datapoint at a time, resulting in a weight update of the form:
+<script type="math/tex">\vec{w}^{(t + 1)} = \vec{w}^{(t)} - n \Delta E_n(\vec{w}^{(t)})</script></p>
+
+<p>where <em>a</em> is the activation of the unit. It is not yet possible to change the learning to more advanced methods using adaptive learning rates yet.</p>
+
+<p>The number of layers and units per layer can be specified manually and determines the whole topology with each unit being fully connected to the previous layer. A bias unit is automatically added to the input of every layer.
+Currently, the logistic sigmoid is used as a squashing function in every hidden and output layer. It is of the form:</p>
+
+<script type="math/tex; mode=display">\frac{1}{1 + exp(-a)}</script>
+
+<p>The command line version <strong>does not perform iterations</strong> which leads to bad results on small datasets. Another restriction is, that the CLI version of the MLP only supports classification, since the labels have to be given explicitly when executing on the command line.</p>
+
+<p>A learned model can be stored and updated with new training instanced using the <code class="highlighter-rouge">--update</code> flag. Output of classification reults is saved as a .txt-file and only consists of the assigned labels. Apart from the command-line interface, it is possible to construct and compile more specialized neural networks using the API and interfaces in the mrlegacy package.</p>
+
+<h2 id="theoretical-background">Theoretical Background</h2>
+
+<p>The <em>multilayer perceptron</em> was inspired by the biological structure of the brain where multiple neurons are connected and form columns and layers. Perceptual input enters this network through our sensory organs and is then further processed into higher levels.
+The term multilayer perceptron is a little misleading since the <em>perceptron</em> is a special case of a single <em>artificial neuron</em> that can be used for simple computations <a href="http://en.wikipedia.org/wiki/Perceptron" title="The perceptron in wikipedia">[1]</a>. The difference is that the perceptron uses a discontinous nonlinearity while for the MLP neurons that are implemented in mahout it is important to use continous nonlinearities. This is necessary for the implemented [...]
+
+<p>The first layer of the MLP represents the input and has no other purpose than routing the input to every connected unit in a feed-forward fashion. Following layers are called hidden layers and the last layer serves the special purpose to determine the output. The activation of a unit <em>u</em> in a hidden layer is computed through a weighted sum of all inputs, resulting in
+<script type="math/tex">a_j = \sum_{i=1}^{D} w_{ji}^{(l)} x_i + w_{j0}^{(l)}</script>
+This computes the activation <em>a</em> for neuron <em>j</em> where <em>w</em> is the weight from neuron <em>i</em> to neuron <em>j</em> in layer <em>l</em>. The last part, where <em>i = 0</em> is called the bias and can be used as an offset, independent from the input.</p>
+
+<p>The activation is then transformed by the aforementioned differentiable, nonlinear <em>activation function</em> and serves as the input to the next layer. The activation function is usually chosen from the family of sigmoidal functions such as <em>tanh</em> or <em>logistic sigmoidal</em> <a href="http://en.wikipedia.org/wiki/Sigmoid_function" title="Sigmoid function on wikipedia">[2]</a>. Often sigmoidal and logistic sigmoidal are used synonymous. Another word for the activation funct [...]
+
+<p>For different units or layers, different activation functions can be used to obtain different behaviors. Especially in the output layer, the activation function can be chosen to obtain the output value <em>y</em>, depending on the learning problem:
+<script type="math/tex">y_k = \sigma (a_k)</script></p>
+
+<p>If the learning problem is a linear regression task, sigma can be chosen to be the identity function. In case of classification problems, the choice of the squashing functions depends on the exact task at hand and often softmax activation functions are used.</p>
+
+<p>The equation for a MLP with three layers (one input, one hidden and one output) is then given by</p>
+
+<script type="math/tex; mode=display">y_k(\vec{x}, \vec{w}) = h \left( \sum_{j=1}^{M} w_{kj}^{(2)} h \left( \sum_{i=1}^{D} w_{ji}^{(1)} x_i + w_{j0}^{(1)} \right) + w_{k0}^{(2)} \right)</script>
+
+<p>where <em>h</em> indicates the respective squashing function that is used in the units of a layer. <em>M</em> and <em>D</em> specify the number of incoming connections to a unit and we can see that the input to the first layer (hidden layer) is just the original input <em>x</em> whereas the input into the second layer (output layer) is the transformed output of layer one. The output <em>y</em> of unit <em>k</em> is therefore given by the above equation and depends on the input <em>x</ [...]
+
+<script type="math/tex; mode=display">y_k(\vec{x}, \vec{w}) = h \left( \sum_{j=0}^{M} w_{kj}^{(2)} h \left( \sum_{i=0}^{D} w_{ji}^{(1)} x_i \right) \right)</script>
+
+<p>The previous paragraphs described how the MLP transforms a given input into some output using a combination of different nonlinear functions. Of course what we really want is to learn the structure of our data so that we can feed data with unknown labels into the network and get the estimated target labels <em>t</em>. To achieve this, we have to train our network. In this context, training means optimizing some function such that the error between the real labels <em>y</em> and the ne [...]
+
+<script type="math/tex; mode=display">E(\vec{w}) = \frac{1}{2} \sum_{n=1}^N \left( y(\vec{x}_n, \vec{w}) - t_n \right)^2</script>
+
+<p>The goal is to minimize this function and thereby increase the performance of our model. A common method to achieve this is to use gradient descent and the so called technique of <em>backpropagation</em> where the goal is to compute the contribution of every unit to the overall error and changing the weight according to this contribution and into the direction of the gradient of the error function at this particular unit. In the following we try to give a short overview of the model t [...]
+
+<p>The problem with minimizing the error function is that the error can only be computed at the output layers where we get <em>t</em>, but we want to update all the weights of all the units. Therefore we use the technique of backpropagation to propagate the error, that we first compute at the output layer, back to the units of the previous layers. For this approach we also need to compute the gradients of the activation function.</p>
+
+<p>Weights are then updated with a small step in the direction of the negative gradient, regulated by the learning rate <em>n</em> such that we arrive at the formula for weight update:</p>
+
+<script type="math/tex; mode=display">\vec{w}^{(t + 1)} = \vec{w}^{(t)} - n \Delta E(\vec{w}^{(t)})</script>
+
+<p>A momentum weight can be set as a parameter of the gradient descent method to increase the probability of finding better local or global optima of the error function.</p>
+
+<p>References</p>
+
+<p>[1] http://en.wikipedia.org/wiki/Perceptron</p>
+
+<p>[2] http://en.wikipedia.org/wiki/Sigmoid_function</p>
+
+<p>[3] <a href="http://research.microsoft.com/en-us/um/people/cmbishop/prml/">Christopher M. Bishop: Pattern Recognition and Machine Learning, Springer 2009</a></p>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/naivebayes.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/naivebayes.html
new file mode 100644
index 0000000..c83a0a4
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/naivebayes.html
@@ -0,0 +1,230 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) NaiveBayes
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/naivebayes.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="NaiveBayes-NaiveBayes"></a></p>
+<h1 id="naive-bayes">Naive Bayes</h1>
+
+<p>Naive Bayes is an algorithm that can be used to classify objects into
+usually binary categories. It is one of the most common learning algorithms
+in spam filters. Despite its simplicity and rather naive assumptions it has
+proven to work surprisingly well in practice.</p>
+
+<p>Before applying the algorithm, the objects to be classified need to be
+represented by numerical features. In the case of e-mail spam each feature
+might indicate whether some specific word is present or absent in the mail
+to classify. The algorithm comes in two phases: Learning and application.
+During learning, a set of feature vectors is given to the algorithm, each
+vector labeled with the class the object it represents, belongs to. From
+that it is deduced which combination of features appears with high
+probability in spam messages. Given this information, during application
+one can easily compute the probability of a new message being either spam
+or not.</p>
+
+<p>The algorithm does make several assumptions, that are not true for most
+datasets, but make computations easier. The worst probably being, that all
+features of an objects are considered independent. In practice, that means,
+given the phrase “Statue of Liberty” was already found in a text, does not
+influence the probability of seeing the phrase “New York” as well.</p>
+
+<p><a name="NaiveBayes-StrategyforaparallelNaiveBayes"></a></p>
+<h2 id="strategy-for-a-parallel-naive-bayes">Strategy for a parallel Naive Bayes</h2>
+
+<p>See <a href="https://issues.apache.org/jira/browse/MAHOUT-9">https://issues.apache.org/jira/browse/MAHOUT-9</a>
+.</p>
+
+<p><a name="NaiveBayes-Examples"></a></p>
+<h2 id="examples">Examples</h2>
+
+<p><a href="20newsgroups.html">20Newsgroups</a></p>
+<ul>
+ <li>Example code showing how to train and use the Naive Bayes classifier
+using the 20 Newsgroups data available at [http://people.csail.mit.edu/jrennie/20Newsgroups/]</li>
+</ul>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/neural-network.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/neural-network.html
new file mode 100644
index 0000000..52e030f
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/neural-network.html
@@ -0,0 +1,205 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Neural Network
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/neural-network.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="NeuralNetwork-NeuralNetworks"></a></p>
+<h1 id="neural-networks">Neural Networks</h1>
+
+<p>Neural Networks are a means for classifying multi dimensional objects. We
+concentrate on implementing back propagation networks with one hidden layer
+as these networks have been covered by the <a href="http://www.cs.stanford.edu/people/ang/papers/nips06-mapreducemulticore.pdf">2006 NIPS map reduce paper</a>
+. Those networks are capable of learning not only linear separating hyper
+planes but arbitrary decision boundaries.</p>
+
+<p><a name="NeuralNetwork-Strategyforparallelbackpropagationnetwork"></a></p>
+<h2 id="strategy-for-parallel-backpropagation-network">Strategy for parallel backpropagation network</h2>
+
+<p><a name="NeuralNetwork-Designofimplementation"></a></p>
+<h2 id="design-of-implementation">Design of implementation</h2>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/partial-implementation.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/partial-implementation.html
new file mode 100644
index 0000000..0aaf103
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/partial-implementation.html
@@ -0,0 +1,333 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Partial Implementation
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/partial-implementation.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h1 id="classifying-with-random-forests">Classifying with random forests</h1>
+
+<p><a name="PartialImplementation-Introduction"></a></p>
+<h1 id="introduction">Introduction</h1>
+
+<p>This quick start page shows how to build a decision forest using the
+partial implementation. This tutorial also explains how to use the decision
+forest to classify new data.
+Partial Decision Forests is a mapreduce implementation where each mapper
+builds a subset of the forest using only the data available in its
+partition. This allows building forests using large datasets as long as
+each partition can be loaded in-memory.</p>
+
+<p><a name="PartialImplementation-Steps"></a></p>
+<h1 id="steps">Steps</h1>
+<p><a name="PartialImplementation-Downloadthedata"></a></p>
+<h2 id="download-the-data">Download the data</h2>
+<ul>
+ <li>The current implementation is compatible with the UCI repository file
+format. In this example we’ll use the NSL-KDD dataset because its large
+enough to show the performances of the partial implementation.
+You can download the dataset here http://nsl.cs.unb.ca/NSL-KDD/
+You can either download the full training set “KDDTrain+.ARFF”, or a 20%
+subset “KDDTrain+_20Percent.ARFF” (we’ll use the full dataset in this
+tutorial) and the test set “KDDTest+.ARFF”.</li>
+ <li>Open the train and test files and remove all the lines that begin with
+‘@’. All those lines are at the top of the files. Actually you can keep
+those lines somewhere, because they’ll help us describe the dataset to
+Mahout</li>
+ <li>Put the data in HDFS: {code}
+$HADOOP_HOME/bin/hadoop fs -mkdir testdata
+$HADOOP_HOME/bin/hadoop fs -put <PATH TO="" DATA=""> testdata{code}</PATH></li>
+</ul>
+
+<p><a name="PartialImplementation-BuildtheJobfiles"></a></p>
+<h2 id="build-the-job-files">Build the Job files</h2>
+<ul>
+ <li>In $MAHOUT_HOME/ run: {code}mvn clean install -DskipTests{code}</li>
+</ul>
+
+<p><a name="PartialImplementation-Generateafiledescriptorforthedataset:"></a></p>
+<h2 id="generate-a-file-descriptor-for-the-dataset">Generate a file descriptor for the dataset:</h2>
+<p>run the following command:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>$HADOOP_HOME/bin/hadoop jar $MAHOUT_HOME/core/target/mahout-core-<VERSION>-job.jar org.apache.mahout.classifier.df.tools.Describe -p testdata/KDDTrain+.arff -f testdata/KDDTrain+.info -d N 3 C 2 N C 4 N C 8 N 2 C 19 N L
+</code></pre>
+</div>
+
+<p>The “N 3 C 2 N C 4 N C 8 N 2 C 19 N L” string describes all the attributes
+of the data. In this cases, it means 1 numerical(N) attribute, followed by
+3 Categorical(C) attributes, …L indicates the label. You can also use ‘I’
+to ignore some attributes</p>
+
+<p><a name="PartialImplementation-Runtheexample"></a></p>
+<h2 id="run-the-example">Run the example</h2>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>$HADOOP_HOME/bin/hadoop jar $MAHOUT_HOME/examples/target/mahout-examples-<version>-job.jar org.apache.mahout.classifier.df.mapreduce.BuildForest -Dmapred.max.split.size=1874231 -d testdata/KDDTrain+.arff -ds testdata/KDDTrain+.info -sl 5 -p -t 100 -o nsl-forest
+</code></pre>
+</div>
+
+<p>which builds 100 trees (-t argument) using the partial implementation (-p).
+Each tree is built using 5 random selected attribute per node (-sl
+argument) and the example outputs the decision tree in the “nsl-forest”
+directory (-o).
+The number of partitions is controlled by the -Dmapred.max.split.size
+argument that indicates to Hadoop the max. size of each partition, in this
+case 1/10 of the size of the dataset. Thus 10 partitions will be used.
+IMPORTANT: using less partitions should give better classification results,
+but needs a lot of memory. So if the Jobs are failing, try increasing the
+number of partitions.</p>
+<ul>
+ <li>
+ <p>The example outputs the Build Time and the oob error estimation</p>
+
+ <p>10/03/13 17:57:29 INFO mapreduce.BuildForest: Build Time: 0h 7m 43s 582
+ 10/03/13 17:57:33 INFO mapreduce.BuildForest: oob error estimate :
+0.002325895231517865
+ 10/03/13 17:57:33 INFO mapreduce.BuildForest: Storing the forest in:
+nsl-forest/forest.seq</p>
+ </li>
+</ul>
+
+<p><a name="PartialImplementation-UsingtheDecisionForesttoClassifynewdata"></a></p>
+<h2 id="using-the-decision-forest-to-classify-new-data">Using the Decision Forest to Classify new data</h2>
+<p>run the following command:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>$HADOOP_HOME/bin/hadoop jar $MAHOUT_HOME/examples/target/mahout-examples-<version>-job.jar org.apache.mahout.classifier.df.mapreduce.TestForest -i nsl-kdd/KDDTest+.arff -ds nsl-kdd/KDDTrain+.info -m nsl-forest -a -mr -o predictions
+</code></pre>
+</div>
+
+<p>This will compute the predictions of “KDDTest+.arff” dataset (-i argument)
+using the same data descriptor generated for the training dataset (-ds) and
+the decision forest built previously (-m). Optionally (if the test dataset
+contains the labels of the tuples) run the analyzer to compute the
+confusion matrix (-a), and you can also store the predictions in a text
+file or a directory of text files(-o). Passing the (-mr) parameter will use
+Hadoop to distribute the classification.</p>
+
+<ul>
+ <li>
+ <p>The example should output the classification time and the confusion
+matrix</p>
+
+ <p>10/03/13 18:08:56 INFO mapreduce.TestForest: Classification Time: 0h 0m 6s
+355
+ 10/03/13 18:08:56 INFO mapreduce.TestForest:
+=======================================================
+ Summary
+ ——————————————————-
+ Correctly Classified Instances : 17657 78.3224%
+ Incorrectly Classified Instances : 4887 21.6776%
+ Total Classified Instances : 22544</p>
+
+ <p>=======================================================
+ Confusion Matrix
+ ——————————————————-
+ a b <–Classified as
+ 9459 252 | 9711 a = normal
+ 4635 8198 | 12833 b = anomaly
+ Default Category: unknown: 2</p>
+ </li>
+</ul>
+
+<p>If the input is a single file then the output will be a single text file,
+in the above example ‘predictions’ would be one single file. If the input
+if a directory containing for example two files ‘a.data’ and ‘b.data’, then
+the output will be a directory ‘predictions’ containing two files
+‘a.data.out’ and ‘b.data.out’</p>
+
+<p><a name="PartialImplementation-KnownIssuesandlimitations"></a></p>
+<h2 id="known-issues-and-limitations">Known Issues and limitations</h2>
+<p>The “Decision Forest” code is still “a work in progress”, many features are
+still missing. Here is a list of some known issues:</p>
+<ul>
+ <li>For now, the training does not support multiple input files. The input
+dataset must be one single file (this support will be available with the upcoming release).
+Classifying new data does support multiple
+input files.</li>
+ <li>The tree building is done when each mapper.close() method is called.
+Because the mappers don’t refresh their state, the job can fail when the
+dataset is big and you try to build a large number of trees.</li>
+</ul>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/random-forests.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/random-forests.html
new file mode 100644
index 0000000..f213c43
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/random-forests.html
@@ -0,0 +1,427 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Random Forests
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/random-forests.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="RandomForests-HowtogrowaDecisionTree"></a></p>
+<h3 id="how-to-grow-a-decision-tree">How to grow a Decision Tree</h3>
+
+<p>source : [3](3.html)</p>
+
+<p>LearnUnprunedTree(<em>X</em>,<em>Y</em>)</p>
+
+<p>Input: <em>X</em> a matrix of <em>R</em> rows and <em>M</em> columns where <em>X{</em>}{<em>}{~}ij{~}</em> =
+the value of the <em>j</em>‘th attribute in the <em>i</em>‘th input datapoint. Each
+column consists of either all real values or all categorical values.
+Input: <em>Y</em> a vector of <em>R</em> elements, where <em>Y{</em>}{<em>}{~}i{~}</em> = the output
+class of the <em>i</em>‘th datapoint. The <em>Y{</em>}{<em>}{~}i{~}</em> values are categorical.
+Output: An Unpruned decision tree</p>
+
+<p>If all records in <em>X</em> have identical values in all their attributes (this
+includes the case where <em>R<2</em>), return a Leaf Node predicting the majority
+output, breaking ties randomly. This case also includes
+If all values in <em>Y</em> are the same, return a Leaf Node predicting this value
+as the output
+Else
+ select <em>m</em> variables at random out of the <em>M</em> variables
+ For <em>j</em> = 1 .. <em>m</em>
+ If <em>j</em>‘th attribute is
+categorical
+<em>
+IG{</em>}{<em>}{~}j{~}</em> = IG(<em>Y</em>|<em>X{</em>}{<em>}{~}j{~}</em>) (see Information
+Gain)
+ Else (<em>j</em>‘th attribute is
+real-valued)
+<em>
+IG{</em>}{<em>}{~}j{~}</em> = IG<em>(</em>Y<em>|</em>X{<em>}{</em>}{~}j{~}<em>) (see Information Gain)
+ Let *j*</em> = argmax{~}j~ <em>IG{</em>}{<em>}{~}j{~}</em> (this is the
+splitting attribute we’ll use)
+ If <em>j*</em> is categorical then
+ For each value <em>v</em> of the <em>j</em>‘th
+attribute
+ Let
+<em>X{</em>}{<em>}{^}v{^}</em> = subset of rows of <em>X</em> in which <em>X{</em>}{<em>}{~}ij{~}</em> = <em>v</em>.
+Let <em>Y{</em>}{<em>}{^}v{^}</em> = corresponding subset of <em>Y</em>
+ Let <em>Child{</em>}{<em>}{^}v{^}</em> =
+LearnUnprunedTree(<em>X{</em>}{<em>}{^}v{^}</em>,<em>Y{</em>}{<em>}{^}v{^}</em>)
+ Return a decision tree node,
+splitting on <em>j</em>‘th attribute. The number of children equals the number of
+values of the <em>j</em>‘th attribute, and the <em>v</em>‘th child is
+<em>Child{</em>}{<em>}{^}v{^}</em>
+ Else <em>j*</em> is real-valued and let <em>t</em> be the best split
+threshold
+ Let <em>X{</em>}{<em>}{^}LO{^}</em> = subset
+of rows of <em>X</em> in which <em>X{</em>}{<em>}{~}ij{~}</em> <em><= t</em>. Let <em>Y{</em>}{<em>}{^}LO{^}</em> =
+corresponding subset of <em>Y</em>
+ Let <em>Child{</em>}{<em>}{^}LO{^}</em> =
+LearnUnprunedTree(<em>X{</em>}{<em>}{^}LO{^}</em>,<em>Y{</em>}{<em>}{^}LO{^}</em>)
+ Let <em>X{</em>}{<em>}{^}HI{^}</em> = subset of rows of <em>X</em>
+in which <em>X{</em>}{<em>}{~}ij{~}</em> <em>> t</em>. Let <em>Y{</em>}{<em>}{^}HI{^}</em> = corresponding
+subset of <em>Y</em>
+ Let <em>Child{</em>}{<em>}{^}HI{^}</em> =
+LearnUnprunedTree(<em>X{</em>}{<em>}{^}HI{^}</em>,<em>Y{</em>}{<em>}{^}HI{^}</em>)
+ Return a decision tree node, splitting on
+<em>j</em>‘th attribute. It has two children corresponding to whether the <em>j</em>‘th
+attribute is above or below the given threshold.</p>
+
+<p><em>Note</em>: There are alternatives to Information Gain for splitting nodes
+ </p>
+
+<p><a name="RandomForests-Informationgain"></a></p>
+<h3 id="information-gain">Information gain</h3>
+
+<p>source : [3](3.html)</p>
+<ol>
+ <li>h4. nominal attributes</li>
+</ol>
+
+<p>suppose X can have one of m values V{~}1~,V{~}2~,…,V{~}m~
+P(X=V{~}1~)=p{~}1~, P(X=V{~}2~)=p{~}2~,…,P(X=V{~}m~)=p{~}m~
+
+H(X)= -sum{~}j=1{~}{^}m^ p{~}j~ log{~}2~ p{~}j~ (The entropy of X)
+H(Y|X=v) = the entropy of Y among only those records in which X has value
+v
+H(Y|X) = sum{~}j~ p{~}j~ H(Y|X=v{~}j~)
+IG(Y|X) = H(Y) - H(Y|X)</p>
+<ol>
+ <li>h4. real-valued attributes</li>
+</ol>
+
+<p>suppose X is real valued
+define IG(Y|X:t) as H(Y) - H(Y|X:t)
+define H(Y|X:t) = H(Y|X<t) P(X<t) + H(Y|X>=t) P(X>=t)
+define IG*(Y|X) = max{~}t~ IG(Y|X:t)</p>
+
+<p><a name="RandomForests-HowtogrowaRandomForest"></a></p>
+<h3 id="how-to-grow-a-random-forest">How to grow a Random Forest</h3>
+
+<p>source : [1](1.html)</p>
+
+<p>Each tree is grown as follows:</p>
+<ol>
+ <li>if the number of cases in the training set is <em>N</em>, sample <em>N</em> cases at
+random -but with replacement, from the original data. This sample will be
+the training set for the growing tree.</li>
+ <li>if there are <em>M</em> input variables, a number <em>m « M</em> is specified such
+that at each node, <em>m</em> variables are selected at random out of the <em>M</em> and
+the best split on these <em>m</em> is used to split the node. The value of <em>m</em> is
+held constant during the forest growing.</li>
+ <li>each tree is grown to its large extent possible. There is no pruning.</li>
+</ol>
+
+<p><a name="RandomForests-RandomForestparameters"></a></p>
+<h3 id="random-forest-parameters">Random Forest parameters</h3>
+
+<p>source : [2](2.html)
+Random Forests are easy to use, the only 2 parameters a user of the
+technique has to determine are the number of trees to be used and the
+number of variables (<em>m</em>) to be randomly selected from the available set of
+variables.
+Breinman’s recommendations are to pick a large number of trees, as well as
+the square root of the number of variables for <em>m</em>.
+ </p>
+
+<p><a name="RandomForests-Howtopredictthelabelofacase"></a></p>
+<h3 id="how-to-predict-the-label-of-a-case">How to predict the label of a case</h3>
+
+<p>Classify(<em>node</em>,<em>V</em>)
+ Input: <em>node</em> from the decision tree, if <em>node.attribute
+= j</em> then the split is done on the <em>j</em>‘th attribute</p>
+
+<p> Input: <em>V</em> a vector of <em>M</em> columns where
+<em>V{</em>}{<em>}{~}j{~}</em> = the value of the <em>j</em>‘th attribute.
+ Output: label of <em>V</em></p>
+
+<p> If <em>node</em> is a Leaf then
+ Return the value predicted
+by <em>node</em></p>
+
+<p> Else
+ Let <em>j =
+node.attribute</em>
+ If <em>j</em> is
+categorical then
+
+
+Let <em>v</em> = <em>V{</em>}{<em>}{~}j{~}</em>
+
+
+Let <em>child{</em>}{<em>}{^}v{^}</em> = child node corresponding to the attribute’s
+value <em>v</em>
+
+ Return Classify(<em>child{</em>}{<em>}{^}v{^}</em>,<em>V</em>)</p>
+
+<p> Else <em>j</em> is
+real-valued
+
+
+Let <em>t = node.threshold</em> (split threshold)
+
+ If Vj < t then
+
+ Let <em>child{</em>}{<em>}{^}LO{^}</em> = child
+node corresponding to (<em><t</em>)
+
+ Return
+Classify(<em>child{</em>}{<em>}{^}LO{^}</em>,<em>V</em>)
+
+
+Else
+
+ Let <em>child{</em>}{<em>}{^}HI{^}</em> =
+child node corresponding to (<em>>=t</em>)
+
+ Return
+Classify(<em>child{</em>}{<em>}{^}HI{^}</em>,<em>V</em>)
+ </p>
+
+<p><a name="RandomForests-Theoutofbag(oob)errorestimation"></a></p>
+<h3 id="the-out-of-bag-oob-error-estimation">The out of bag (oob) error estimation</h3>
+
+<p>source : [1](1.html)</p>
+
+<p>in random forests, there is no need for cross-validation or a separate test
+set to get an unbiased estimate of the test set error. It is estimated
+internally, during the run, as follows:</p>
+<ul>
+ <li>each tree is constructed using a different bootstrap sample from the
+original data. About one-third of the cases left of the bootstrap sample
+and not used in the construction of the <em>kth</em> tree.</li>
+ <li>put each case left out in the construction of the <em>kth</em> tree down the
+<em>kth{</em>}tree to get a classification. In this way, a test set classification
+is obtained for each case in about one-thrid of the trees. At the end of
+the run, take <em>j</em> to be the class that got most of the the votes every time
+case <em>n</em> was <em>oob</em>. The proportion of times that <em>j</em> is not equal to the
+true class of <em>n</em> averaged over all cases is the <em>oob error estimate</em>. This
+has proven to be unbiased in many tests.</li>
+</ul>
+
+<p><a name="RandomForests-OtherRFuses"></a></p>
+<h3 id="other-rf-uses">Other RF uses</h3>
+
+<p>source : [1](1.html)</p>
+<ul>
+ <li>variable importance</li>
+ <li>gini importance</li>
+ <li>proximities</li>
+ <li>scaling</li>
+ <li>prototypes</li>
+ <li>missing values replacement for the training set</li>
+ <li>missing values replacement for the test set</li>
+ <li>detecting mislabeled cases</li>
+ <li>detecting outliers</li>
+ <li>detecting novelties</li>
+ <li>unsupervised learning</li>
+ <li>balancing prediction error
+Please refer to [1](1.html)
+ for a detailed description</li>
+</ul>
+
+<p><a name="RandomForests-References"></a></p>
+<h3 id="references">References</h3>
+
+<p>[1](1.html)
+ Random Forests - Classification Description
+ <a href="http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm">http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm</a>
+[2](2.html)
+ B. Larivi�re & D. Van Den Poel, 2004. “Predicting Customer Retention
+and Profitability by Using Random Forests and Regression Forests
+Techniques,”
+ Working Papers of Faculty of
+Economics and Business Administration, Ghent University, Belgium 04/282,
+Ghent University,
+ Faculty of Economics and
+Business Administration.
+ Available online : <a href="http://ideas.repec.org/p/rug/rugwps/04-282.html">http://ideas.repec.org/p/rug/rugwps/04-282.html</a>
+[3](3.html)
+ Decision Trees - Andrew W. Moore[4]
+ http://www.cs.cmu.edu/~awm/tutorials[1](1.html)
+[4](4.html)
+ Information Gain - Andrew W. Moore
+ <a href="http://www.cs.cmu.edu/~awm/tutorials">http://www.cs.cmu.edu/~awm/tutorials</a></p>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/restricted-boltzmann-machines.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/restricted-boltzmann-machines.html
new file mode 100644
index 0000000..9e674f8
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/restricted-boltzmann-machines.html
@@ -0,0 +1,237 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Restricted Boltzmann Machines
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/restricted-boltzmann-machines.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p>NOTE: This implementation is a Work-In-Progress, at least till September,</p>
+<ol>
+ <li></li>
+</ol>
+
+<p>The JIRA issue is <a href="https://issues.apache.org/jira/browse/MAHOUT-375">here</a>
+.</p>
+
+<p><a name="RestrictedBoltzmannMachines-BoltzmannMachines"></a></p>
+<h3 id="boltzmann-machines">Boltzmann Machines</h3>
+<p>Boltzmann Machines are a type of stochastic neural networks that closely
+resemble physical processes. They define a network of units with an overall
+energy that is evolved over a period of time, until it reaches thermal
+equilibrium.</p>
+
+<p>However, the convergence speed of Boltzmann machines that have
+unconstrained connectivity is low.</p>
+
+<p><a name="RestrictedBoltzmannMachines-RestrictedBoltzmannMachines"></a></p>
+<h3 id="restricted-boltzmann-machines">Restricted Boltzmann Machines</h3>
+<p>Restricted Boltzmann Machines are a variant, that are ‘restricted’ in the
+sense that connections between hidden units of a single layer are <em>not</em>
+allowed. In addition, stacking multiple RBM’s is also feasible, with the
+activities of the hidden units forming the base for a higher-level RBM. The
+combination of these two features renders RBM’s highly usable for
+parallelization.</p>
+
+<p>In the Netflix Prize, RBM’s offered distinctly orthogonal predictions to
+SVD and k-NN approaches, and contributed immensely to the final solution.</p>
+
+<p><a name="RestrictedBoltzmannMachines-RBM'sinApacheMahout"></a></p>
+<h3 id="rbms-in-apache-mahout">RBM’s in Apache Mahout</h3>
+<p>An implementation of Restricted Boltzmann Machines is being developed for
+Apache Mahout as a Google Summer of Code 2010 project. A recommender
+interface will also be provided. The key aims of the implementation are:</p>
+<ol>
+ <li>Accurate - should replicate known results, including those of the Netflix
+Prize</li>
+ <li>Fast - The implementation uses Map-Reduce, hence, it should be fast</li>
+ <li>Scale - Should scale to large datasets, with a design whose critical
+parts don’t need a dependency between the amount of memory on your cluster
+systems and the size of your dataset</li>
+</ol>
+
+<p>You can view the patch as it develops <a href="http://github.com/sisirkoppaka/mahout-rbm/compare/trunk...rbm">here</a>
+.</p>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/classification/support-vector-machines.html b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/support-vector-machines.html
new file mode 100644
index 0000000..55afd7f
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/classification/support-vector-machines.html
@@ -0,0 +1,227 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Support Vector Machines
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/classification/support-vector-machines.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="SupportVectorMachines-SupportVectorMachines"></a></p>
+<h1 id="support-vector-machines">Support Vector Machines</h1>
+
+<p>As with Naive Bayes, Support Vector Machines (or SVMs in short) can be used
+to solve the task of assigning objects to classes. However, the way this
+task is solved is completely different to the setting in Naive Bayes.</p>
+
+<p>Each object is considered to be a point in <em>n</em> dimensional feature space,
+<em>n</em> being the number of features used to describe the objects numerically.
+In addition each object is assigned a binary label, let us assume the
+labels are “positive” and “negative”. During learning, the algorithm tries
+to find a hyperplane in that space, that perfectly separates positive from
+negative objects.
+It is trivial to think of settings where this might very well be
+impossible. To remedy this situation, objects can be assigned so called
+slack terms, that punish mistakes made during learning appropriately. That
+way, the algorithm is forced to find the hyperplane that causes the least
+number of mistakes.</p>
+
+<p>Another way to overcome the problem of there being no linear hyperplane to
+separate positive from negative objects is to simply project each feature
+vector into an higher dimensional feature space and search for a linear
+separating hyperplane in that new space. Usually the main problem with
+learning in high dimensional feature spaces is the so called curse of
+dimensionality. That is, there are fewer learning examples available than
+free parameters to tune. In the case of SVMs this problem is less
+detrimental, as SVMs impose additional structural constraints on their
+solutions. Each separating hyperplane needs to have a maximal margin to all
+training examples. In addition, that way, the solution may be based on the
+information encoded in only very few examples.</p>
+
+<p><a name="SupportVectorMachines-Strategyforparallelization"></a></p>
+<h2 id="strategy-for-parallelization">Strategy for parallelization</h2>
+
+<p><a name="SupportVectorMachines-Designofpackages"></a></p>
+<h2 id="design-of-packages">Design of packages</h2>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/canopy-clustering.html b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/canopy-clustering.html
new file mode 100644
index 0000000..d58768a
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/canopy-clustering.html
@@ -0,0 +1,382 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Canopy Clustering
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/clustering/canopy-clustering.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><strong><em>This Document refers to the deprecated Map-Reduce Version of this algorithm, please see <a href="/algorithms/clustering/canopy.html">the new implementation</a></em></strong></p>
+
+<p><a name="CanopyClustering-CanopyClustering"></a></p>
+<h1 id="canopy-clustering">Canopy Clustering</h1>
+
+<p><a href="http://www.kamalnigam.com/papers/canopy-kdd00.pdf">Canopy Clustering</a>
+ is a very simple, fast and surprisingly accurate method for grouping
+objects into clusters. All objects are represented as a point in a
+multidimensional feature space. The algorithm uses a fast approximate
+distance metric and two distance thresholds T1 > T2 for processing. The
+basic algorithm is to begin with a set of points and remove one at random.
+Create a Canopy containing this point and iterate through the remainder of
+the point set. At each point, if its distance from the first point is < T1,
+then add the point to the cluster. If, in addition, the distance is < T2,
+then remove the point from the set. This way points that are very close to
+the original will avoid all further processing. The algorithm loops until
+the initial set is empty, accumulating a set of Canopies, each containing
+one or more points. A given point may occur in more than one Canopy.</p>
+
+<p>Canopy Clustering is often used as an initial step in more rigorous
+clustering techniques, such as <a href="k-means-clustering.html">K-Means Clustering</a>
+. By starting with an initial clustering the number of more expensive
+distance measurements can be significantly reduced by ignoring points
+outside of the initial canopies.</p>
+
+<p><strong>WARNING</strong>: Canopy is deprecated in the latest release and will be removed once streaming k-means becomes stable enough.</p>
+
+<p><a name="CanopyClustering-Strategyforparallelization"></a></p>
+<h2 id="strategy-for-parallelization">Strategy for parallelization</h2>
+
+<p>Looking at the sample Hadoop implementation in <a href="http://code.google.com/p/canopy-clustering/">http://code.google.com/p/canopy-clustering/</a>
+ the processing is done in 3 M/R steps:</p>
+<ol>
+ <li>The data is massaged into suitable input format</li>
+ <li>Each mapper performs canopy clustering on the points in its input set and
+outputs its canopies’ centers</li>
+ <li>The reducer clusters the canopy centers to produce the final canopy
+centers</li>
+ <li>The points are then clustered into these final canopies</li>
+</ol>
+
+<p>Some ideas can be found in <a href="https://www.youtube.com/watch?v=yjPBkvYh-ss&list=PLEFAB97242917704A">Cluster computing and MapReduce</a>
+ lecture video series [by Google(r)]; Canopy Clustering is discussed in <a href="https://www.youtube.com/watch?v=1ZDybXl212Q">lecture #4</a>
+. Finally here is the <a href="http://en.wikipedia.org/wiki/Canopy_clustering_algorithm">Wikipedia page</a>
+.</p>
+
+<p><a name="CanopyClustering-Designofimplementation"></a></p>
+<h2 id="design-of-implementation">Design of implementation</h2>
+
+<p>The implementation accepts as input Hadoop SequenceFiles containing
+multidimensional points (VectorWritable). Points may be expressed either as
+dense or sparse Vectors and processing is done in two phases: Canopy
+generation and, optionally, Clustering.</p>
+
+<p><a name="CanopyClustering-Canopygenerationphase"></a></p>
+<h3 id="canopy-generation-phase">Canopy generation phase</h3>
+
+<p>During the map step, each mapper processes a subset of the total points and
+applies the chosen distance measure and thresholds to generate canopies. In
+the mapper, each point which is found to be within an existing canopy will
+be added to an internal list of Canopies. After observing all its input
+vectors, the mapper updates all of its Canopies and normalizes their totals
+to produce canopy centroids which are output, using a constant key
+(“centroid”) to a single reducer. The reducer receives all of the initial
+centroids and again applies the canopy measure and thresholds to produce a
+final set of canopy centroids which is output (i.e. clustering the cluster
+centroids). The reducer output format is: SequenceFile(Text, Canopy) with
+the <em>key</em> encoding the canopy identifier.</p>
+
+<p><a name="CanopyClustering-Clusteringphase"></a></p>
+<h3 id="clustering-phase">Clustering phase</h3>
+
+<p>During the clustering phase, each mapper reads the Canopies produced by the
+first phase. Since all mappers have the same canopy definitions, their
+outputs will be combined during the shuffle so that each reducer (many are
+allowed here) will see all of the points assigned to one or more canopies.
+The output format will then be: SequenceFile(IntWritable,
+WeightedVectorWritable) with the <em>key</em> encoding the canopyId. The
+WeightedVectorWritable has two fields: a double weight and a VectorWritable
+vector. Together they encode the probability that each vector is a member
+of the given canopy.</p>
+
+<p><a name="CanopyClustering-RunningCanopyClustering"></a></p>
+<h2 id="running-canopy-clustering">Running Canopy Clustering</h2>
+
+<p>The canopy clustering algorithm may be run using a command-line invocation
+on CanopyDriver.main or by making a Java call to CanopyDriver.run(…).
+Both require several arguments:</p>
+
+<p>Invocation using the command line takes the form:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>bin/mahout canopy \
+ -i <input vectors directory> \
+ -o <output working directory> \
+ -dm <DistanceMeasure> \
+ -t1 <T1 threshold> \
+ -t2 <T2 threshold> \
+ -t3 <optional reducer T1 threshold> \
+ -t4 <optional reducer T2 threshold> \
+ -cf <optional cluster filter size (default: 0)> \
+ -ow <overwrite output directory if present>
+ -cl <run input vector clustering after computing Canopies>
+ -xm <execution method: sequential or mapreduce>
+</code></pre>
+</div>
+
+<p>Invocation using Java involves supplying the following arguments:</p>
+
+<ol>
+ <li>input: a file path string to a directory containing the input data set a
+SequenceFile(WritableComparable, VectorWritable). The sequence file <em>key</em>
+is not used.</li>
+ <li>output: a file path string to an empty directory which is used for all
+output from the algorithm.</li>
+ <li>measure: the fully-qualified class name of an instance of DistanceMeasure
+which will be used for the clustering.</li>
+ <li>t1: the T1 distance threshold used for clustering.</li>
+ <li>t2: the T2 distance threshold used for clustering.</li>
+ <li>t3: the optional T1 distance threshold used by the reducer for
+clustering. If not specified, T1 is used by the reducer.</li>
+ <li>t4: the optional T2 distance threshold used by the reducer for
+clustering. If not specified, T2 is used by the reducer.</li>
+ <li>clusterFilter: the minimum size for canopies to be output by the
+algorithm. Affects both sequential and mapreduce execution modes, and
+mapper and reducer outputs.</li>
+ <li>runClustering: a boolean indicating, if true, that the clustering step is
+to be executed after clusters have been determined.</li>
+ <li>runSequential: a boolean indicating, if true, that the computation is to
+be run in memory using the reference Canopy implementation. Note: that the
+sequential implementation performs a single pass through the input vectors
+whereas the MapReduce implementation performs two passes (once in the
+mapper and again in the reducer). The MapReduce implementation will
+typically produce less clusters than the sequential implementation as a
+result.</li>
+</ol>
+
+<p>After running the algorithm, the output directory will contain:</p>
+<ol>
+ <li>clusters-0: a directory containing SequenceFiles(Text, Canopy) produced
+by the algorithm. The Text <em>key</em> contains the cluster identifier of the
+Canopy.</li>
+ <li>clusteredPoints: (if runClustering enabled) a directory containing
+SequenceFile(IntWritable, WeightedVectorWritable). The IntWritable <em>key</em> is
+the canopyId. The WeightedVectorWritable <em>value</em> is a bean containing a
+double <em>weight</em> and a VectorWritable <em>vector</em> where the weight indicates
+the probability that the vector is a member of the canopy. For canopy
+clustering, the weights are computed as 1/(1+distance) where the distance
+is between the cluster center and the vector using the chosen
+DistanceMeasure.</li>
+</ol>
+
+<p><a name="CanopyClustering-Examples"></a></p>
+<h1 id="examples">Examples</h1>
+
+<p>The following images illustrate Canopy clustering applied to a set of
+randomly-generated 2-d data points. The points are generated using a normal
+distribution centered at a mean location and with a constant standard
+deviation. See the README file in the <a href="https://github.com/apache/mahout/blob/master/examples/src/main/java/org/apache/mahout/clustering/display/README.txt">/examples/src/main/java/org/apache/mahout/clustering/display/README.txt</a>
+ for details on running similar examples.</p>
+
+<p>The points are generated as follows:</p>
+
+<ul>
+ <li>500 samples m=[1.0, 1.0](1.0,-1.0.html)
+ sd=3.0</li>
+ <li>300 samples m=[1.0, 0.0](1.0,-0.0.html)
+ sd=0.5</li>
+ <li>300 samples m=[0.0, 2.0](0.0,-2.0.html)
+ sd=0.1</li>
+</ul>
+
+<p>In the first image, the points are plotted and the 3-sigma boundaries of
+their generator are superimposed.</p>
+
+<p><img src="../../images/SampleData.png" alt="sample data" /></p>
+
+<p>In the second image, the resulting canopies are shown superimposed upon the
+sample data. Each canopy is represented by two circles, with radius T1 and
+radius T2.</p>
+
+<p><img src="../../images/Canopy.png" alt="canopy" /></p>
+
+<p>The third image uses the same values of T1 and T2 but only superimposes
+canopies covering more than 10% of the population. This is a bit better
+representation of the data but it still has lots of room for improvement.
+The advantage of Canopy clustering is that it is single-pass and fast
+enough to iterate runs using different T1, T2 parameters and display
+thresholds.</p>
+
+<p><img src="../../images/Canopy10.png" alt="canopy" /></p>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/cluster-dumper.html b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/cluster-dumper.html
new file mode 100644
index 0000000..c8cd743
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/cluster-dumper.html
@@ -0,0 +1,299 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Cluster Dumper
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/clustering/cluster-dumper.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="ClusterDumper-Introduction"></a></p>
+<h2 id="cluster-dumper---introduction">Cluster Dumper - Introduction</h2>
+
+<p>Clustering tasks in Mahout will output data in the format of a SequenceFile
+(Text, Cluster) and the Text is a cluster identifier string. To analyze
+this output we need to convert the sequence files to a human readable
+format and this is achieved using the clusterdump utility.</p>
+
+<p><a name="ClusterDumper-Stepsforanalyzingclusteroutputusingclusterdumputility"></a></p>
+<h2 id="steps-for-analyzing-cluster-output-using-clusterdump-utility">Steps for analyzing cluster output using clusterdump utility</h2>
+
+<p>After you’ve executed a clustering tasks (either examples or real-world),
+you can run clusterdumper in 2 modes:</p>
+
+<ol>
+ <li>Hadoop Environment</li>
+ <li>Standalone Java Program</li>
+</ol>
+
+<p><a name="ClusterDumper-HadoopEnvironment{anchor:HadoopEnvironment}"></a></p>
+<h3 id="hadoop-environment">Hadoop Environment</h3>
+
+<p>If you have setup your HADOOP_HOME environment variable, you can use the
+command line utility <code class="highlighter-rouge">mahout</code> to execute the ClusterDumper on Hadoop. In
+this case we wont need to get the output clusters to our local machines.
+The utility will read the output clusters present in HDFS and output the
+human-readable cluster values into our local file system. Say you’ve just
+executed the <a href="clustering-of-synthetic-control-data.html">synthetic control example </a>
+ and want to analyze the output, you can execute the <code class="highlighter-rouge">mahout clusterdumper</code> utility from the command line.</p>
+
+<h4 id="cli-options">CLI options:</h4>
+<div class="highlighter-rouge"><pre class="highlight"><code>--help Print out help
+--input (-i) input The directory containing Sequence
+ Files for the Clusters
+--output (-o) output The output file. If not specified,
+ dumps to the console.
+--outputFormat (-of) outputFormat The optional output format to write
+ the results as. Options: TEXT, CSV, or GRAPH_ML
+--substring (-b) substring The number of chars of the
+ asFormatString() to print
+--pointsDir (-p) pointsDir The directory containing points
+ sequence files mapping input vectors
+ to their cluster. If specified,
+ then the program will output the
+ points associated with a cluster
+--dictionary (-d) dictionary The dictionary file.
+--dictionaryType (-dt) dictionaryType The dictionary file type
+ (text|sequencefile)
+--distanceMeasure (-dm) distanceMeasure The classname of the DistanceMeasure.
+ Default is SquaredEuclidean.
+--numWords (-n) numWords The number of top terms to print
+--tempDir tempDir Intermediate output directory
+--startPhase startPhase First phase to run
+--endPhase endPhase Last phase to run
+--evaluate (-e) Run ClusterEvaluator and CDbwEvaluator over the
+ input. The output will be appended to the rest of
+ the output at the end.
+</code></pre>
+</div>
+
+<h3 id="standalone-java-program">Standalone Java Program</h3>
+
+<p>Run the clusterdump utility as follows as a standalone Java Program through Eclipse. <!-- - if you are using eclipse, setup mahout-utils as a project as specified in [Working with Maven in Eclipse](../../developers/buildingmahout.html). -->
+ To execute ClusterDumper.java,</p>
+
+<ul>
+ <li>Under mahout-utils, Right-Click on ClusterDumper.java</li>
+ <li>Choose Run-As, Run Configurations</li>
+ <li>On the left menu, click on Java Application</li>
+ <li>On the top-bar click on “New Launch Configuration”</li>
+ <li>
+ <p>A new launch should be automatically created with project as</p>
+
+ <p>“mahout-utils” and Main Class as “org.apache.mahout.utils.clustering.ClusterDumper”</p>
+ </li>
+</ul>
+
+<p>In the arguments tab, specify the below arguments</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>--seqFileDir <MAHOUT_HOME>/examples/output/clusters-10
+--pointsDir <MAHOUT_HOME>/examples/output/clusteredPoints
+--output <MAHOUT_HOME>/examples/output/clusteranalyze.txt
+replace <MAHOUT_HOME> with the actual path of your $MAHOUT_HOME
+</code></pre>
+</div>
+
+<ul>
+ <li>Hit run to execute the ClusterDumper using Eclipse. Setting breakpoints etc should just work fine.</li>
+</ul>
+
+<p>Reading the output file</p>
+
+<p>This will output the clusters into a file called clusteranalyze.txt inside $MAHOUT_HOME/examples/output
+Sample data will look like</p>
+
+<p>CL-0 { n=116 c=<a href="29.922,-30.407,-30.373,-30.094,-29.886,-29.937,-29.751,-30.054,-30.039,-30.126,-29.764,-29.835,-30.503,-29.876,-29.990,-29.605,-29.379,-30.120,-29.882,-30.161,-29.825,-30.074,-30.001,-30.421,-29.867,-29.736,-29.760,-30.192,-30.134,-30.082,-29.962,-29.512,-29.736,-29.594,-29.493,-29.761,-29.183,-29.517,-29.273,-29.161,-29.215,-29.731,-29.154,-29.113,-29.348,-28.981,-29.543,-29.192,-29.479,-29.406,-29.715,-29.344,-29.628,-29.074,-29.347,-29.812,-29.058,-29.177,-2 [...]
+ r=[3.463, 3.351, 3.452, 3.438, 3.371, 3.569, 3.253, 3.531, 3.439, 3.472,
+3.402, 3.459, 3.320, 3.260, 3.430, 3.452, 3.320, 3.499, 3.302, 3.511,
+3.520, 3.447, 3.516, 3.485, 3.345, 3.178, 3.492, 3.434, 3.619, 3.483,
+3.651, 3.833, 3.812, 3.433, 4.133, 3.855, 4.123, 3.999, 4.467, 4.731,
+4.539, 4.956, 4.644, 4.382, 4.277, 4.918, 4.784, 4.582, 4.915, 4.607,
+4.672, 4.577, 5.035, 5.241, 4.731, 4.688, 4.685, 4.657, 4.912, 4.300] }</p>
+
+<p>and on…</p>
+
+<p>where CL-0 is the Cluster 0 and n=116 refers to the number of points observed by this cluster and c = [29.922 …]
+ refers to the center of Cluster as a vector and r = [3.463 ..] refers to
+the radius of the cluster as a vector.</p>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/expectation-maximization.html b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/expectation-maximization.html
new file mode 100644
index 0000000..3052877
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/expectation-maximization.html
@@ -0,0 +1,247 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Expectation Maximization
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/clustering/expectation-maximization.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="ExpectationMaximization-ExpectationMaximization"></a></p>
+<h1 id="expectation-maximization">Expectation Maximization</h1>
+
+<p>The principle of EM can be applied to several learning settings, but is
+most commonly associated with clustering. The main principle of the
+algorithm is comparable to k-Means. Yet in contrast to hard cluster
+assignments, each object is given some probability to belong to a cluster.
+Accordingly cluster centers are recomputed based on the average of all
+objects weighted by their probability of belonging to the cluster at hand.</p>
+
+<p><a name="ExpectationMaximization-Canopy-modifiedEM"></a></p>
+<h2 id="canopy-modified-em">Canopy-modified EM</h2>
+
+<p>One can also use the canopies idea to speed up prototypebased clustering
+methods like K-means and Expectation-Maximization (EM). In general, neither
+K-means nor EMspecify how many clusters to use. The canopies technique does
+not help this choice.</p>
+
+<p>Prototypes (our estimates of the cluster centroids) are associated with the
+canopies that contain them, and the prototypes are only influenced by data
+that are inside their associated canopies. After creating the canopies, we
+decide how many prototypes will be created for each canopy. This could be
+done, for example, using the number of data points in a canopy and AIC or
+BIC where points that occur in more than one canopy are counted
+fractionally. Then we place prototypesinto each canopy. This initial
+placement can be random, as long as it is within the canopy in question, as
+determined by the inexpensive distance metric.</p>
+
+<p>Then, instead of calculating the distance from each prototype to every
+point (as is traditional, a O(nk) operation), theE-step instead calculates
+the distance from each prototype to a much smaller number of points. For
+each prototype, we find the canopies that contain it (using the cheap
+distance metric), and only calculate distances (using the expensive
+distance metric) from that prototype to points within those canopies.</p>
+
+<p>Note that by this procedure prototypes may move across canopy boundaries
+when canopies overlap. Prototypes may move to cover the data in the
+overlapping region, and then move entirely into another canopy in order to
+cover data there.</p>
+
+<p>The canopy-modified EM algorithm behaves very similarly to traditional EM,
+with the slight difference that points outside the canopy have no influence
+on points in the canopy, rather than a minute influence. If the canopy
+property holds, and points in the same cluster fall in the same canopy,
+then the canopy-modified EM will almost always converge to the same maximum
+in likelihood as the traditional EM. In fact, the difference in each
+iterative step (apart from the enormous computational savings of computing
+fewer terms) will be negligible since points outside the canopy will have
+exponentially small influence.</p>
+
+<p><a name="ExpectationMaximization-StrategyforParallelization"></a></p>
+<h2 id="strategy-for-parallelization">Strategy for Parallelization</h2>
+
+<p><a name="ExpectationMaximization-Map/ReduceImplementation"></a></p>
+<h2 id="mapreduce-implementation">Map/Reduce Implementation</h2>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/fuzzy-k-means.html b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/fuzzy-k-means.html
new file mode 100644
index 0000000..afc9a2e
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/fuzzy-k-means.html
@@ -0,0 +1,380 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Fuzzy K-Means
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/clustering/fuzzy-k-means.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p>Fuzzy K-Means (also called Fuzzy C-Means) is an extension of <a href="http://mahout.apache.org/users/clustering/k-means-clustering.html">K-Means</a>
+, the popular simple clustering technique. While K-Means discovers hard
+clusters (a point belong to only one cluster), Fuzzy K-Means is a more
+statistically formalized method and discovers soft clusters where a
+particular point can belong to more than one cluster with certain
+probability.</p>
+
+<p><a name="FuzzyK-Means-Algorithm"></a></p>
+<h4 id="algorithm">Algorithm</h4>
+
+<p>Like K-Means, Fuzzy K-Means works on those objects which can be represented
+in n-dimensional vector space and a distance measure is defined.
+The algorithm is similar to k-means.</p>
+
+<ul>
+ <li>Initialize k clusters</li>
+ <li>Until converged
+ <ul>
+ <li>Compute the probability of a point belong to a cluster for every <point,cluster> pair</li>
+ <li>Recompute the cluster centers using above probability membership values of points to clusters</li>
+ </ul>
+ </li>
+</ul>
+
+<p><a name="FuzzyK-Means-DesignImplementation"></a></p>
+<h4 id="design-implementation">Design Implementation</h4>
+
+<p>The design is similar to K-Means present in Mahout. It accepts an input
+file containing vector points. User can either provide the cluster centers
+as input or can allow canopy algorithm to run and create initial clusters.</p>
+
+<p>Similar to K-Means, the program doesn’t modify the input directories. And
+for every iteration, the cluster output is stored in a directory cluster-N.
+The code has set number of reduce tasks equal to number of map tasks. So,
+those many part-0</p>
+
+<p>Files are created in clusterN directory. The code uses
+driver/mapper/combiner/reducer as follows:</p>
+
+<p>FuzzyKMeansDriver - This is similar to KMeansDriver. It iterates over
+input points and cluster points for specified number of iterations or until
+it is converged.During every iteration i, a new cluster-i directory is
+created which contains the modified cluster centers obtained during
+FuzzyKMeans iteration. This will be feeded as input clusters in the next
+iteration. Once Fuzzy KMeans is run for specified number of
+iterations or until it is converged, a map task is run to output “the point
+and the cluster membership to each cluster” pair as final output to a
+directory named “points”.</p>
+
+<p>FuzzyKMeansMapper - reads the input cluster during its configure() method,
+then computes cluster membership probability of a point to each
+cluster.Cluster membership is inversely propotional to the distance.
+Distance is computed using user supplied distance measure. Output key
+is encoded clusterId. Output values are ClusterObservations containing
+observation statistics.</p>
+
+<p>FuzzyKMeansCombiner - receives all key:value pairs from the mapper and
+produces partial sums of the cluster membership probability times input
+vectors for each cluster. Output key is: encoded cluster identifier. Output
+values are ClusterObservations containing observation statistics.</p>
+
+<p>FuzzyKMeansReducer - Multiple reducers receives certain keys and all values
+associated with those keys. The reducer sums the values to produce a new
+centroid for the cluster which is output. Output key is: encoded cluster
+identifier (e.g. “C14”. Output value is: formatted cluster identifier (e.g.
+“C14”). The reducer encodes unconverged clusters with a ‘Cn’ cluster Id and
+converged clusters with ‘Vn’ clusterId.</p>
+
+<p><a name="FuzzyK-Means-RunningFuzzyk-MeansClustering"></a></p>
+<h2 id="running-fuzzy-k-means-clustering">Running Fuzzy k-Means Clustering</h2>
+
+<p>The Fuzzy k-Means clustering algorithm may be run using a command-line
+invocation on FuzzyKMeansDriver.main or by making a Java call to
+FuzzyKMeansDriver.run().</p>
+
+<p>Invocation using the command line takes the form:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>bin/mahout fkmeans \
+ -i <input vectors directory> \
+ -c <input clusters directory> \
+ -o <output working directory> \
+ -dm <DistanceMeasure> \
+ -m <fuzziness argument >1> \
+ -x <maximum number of iterations> \
+ -k <optional number of initial clusters to sample from input vectors> \
+ -cd <optional convergence delta. Default is 0.5> \
+ -ow <overwrite output directory if present>
+ -cl <run input vector clustering after computing Clusters>
+ -e <emit vectors to most likely cluster during clustering>
+ -t <threshold to use for clustering if -e is false>
+ -xm <execution method: sequential or mapreduce>
+</code></pre>
+</div>
+
+<p><em>Note:</em> if the -k argument is supplied, any clusters in the -c directory
+will be overwritten and -k random points will be sampled from the input
+vectors to become the initial cluster centers.</p>
+
+<p>Invocation using Java involves supplying the following arguments:</p>
+
+<ol>
+ <li>input: a file path string to a directory containing the input data set a
+SequenceFile(WritableComparable, VectorWritable). The sequence file <em>key</em>
+is not used.</li>
+ <li>clustersIn: a file path string to a directory containing the initial
+clusters, a SequenceFile(key, SoftCluster | Cluster | Canopy). Fuzzy
+k-Means SoftClusters, k-Means Clusters and Canopy Canopies may be used for
+the initial clusters.</li>
+ <li>output: a file path string to an empty directory which is used for all
+output from the algorithm.</li>
+ <li>measure: the fully-qualified class name of an instance of DistanceMeasure
+which will be used for the clustering.</li>
+ <li>convergence: a double value used to determine if the algorithm has
+converged (clusters have not moved more than the value in the last
+iteration)</li>
+ <li>max-iterations: the maximum number of iterations to run, independent of
+the convergence specified</li>
+ <li>m: the “fuzzyness” argument, a double > 1. For m equal to 2, this is
+equivalent to normalising the coefficient linearly to make their sum 1.
+When m is close to 1, then the cluster center closest to the point is given
+much more weight than the others, and the algorithm is similar to k-means.</li>
+ <li>runClustering: a boolean indicating, if true, that the clustering step is
+to be executed after clusters have been determined.</li>
+ <li>emitMostLikely: a boolean indicating, if true, that the clustering step
+should only emit the most likely cluster for each clustered point.</li>
+ <li>threshold: a double indicating, if emitMostLikely is false, the cluster
+probability threshold used for emitting multiple clusters for each point. A
+value of 0 will emit all clusters with their associated probabilities for
+each vector.</li>
+ <li>runSequential: a boolean indicating, if true, that the algorithm is to
+use the sequential reference implementation running in memory.</li>
+</ol>
+
+<p>After running the algorithm, the output directory will contain:</p>
+<ol>
+ <li>clusters-N: directories containing SequenceFiles(Text, SoftCluster)
+produced by the algorithm for each iteration. The Text <em>key</em> is a cluster
+identifier string.</li>
+ <li>clusteredPoints: (if runClustering enabled) a directory containing
+SequenceFile(IntWritable, WeightedVectorWritable). The IntWritable <em>key</em> is
+the clusterId. The WeightedVectorWritable <em>value</em> is a bean containing a
+double <em>weight</em> and a VectorWritable <em>vector</em> where the weights are
+computed as 1/(1+distance) where the distance is between the cluster center
+and the vector using the chosen DistanceMeasure.</li>
+</ol>
+
+<p><a name="FuzzyK-Means-Examples"></a></p>
+<h1 id="examples">Examples</h1>
+
+<p>The following images illustrate Fuzzy k-Means clustering applied to a set
+of randomly-generated 2-d data points. The points are generated using a
+normal distribution centered at a mean location and with a constant
+standard deviation. See the README file in the <a href="https://github.com/apache/mahout/blob/master/examples/src/main/java/org/apache/mahout/clustering/display/README.txt">/examples/src/main/java/org/apache/mahout/clustering/display/README.txt</a>
+ for details on running similar examples.</p>
+
+<p>The points are generated as follows:</p>
+
+<ul>
+ <li>500 samples m=[1.0, 1.0](1.0,-1.0.html)
+ sd=3.0</li>
+ <li>300 samples m=[1.0, 0.0](1.0,-0.0.html)
+ sd=0.5</li>
+ <li>300 samples m=[0.0, 2.0](0.0,-2.0.html)
+ sd=0.1</li>
+</ul>
+
+<p>In the first image, the points are plotted and the 3-sigma boundaries of
+their generator are superimposed.</p>
+
+<p><img src="/assets/img/SampleData.png" alt="fuzzy" /></p>
+
+<p>In the second image, the resulting clusters (k=3) are shown superimposed upon the sample data. As Fuzzy k-Means is an iterative algorithm, the centers of the clusters in each recent iteration are shown using different colors. Bold red is the final clustering and previous iterations are shown in [orange, yellow, green, blue, violet and gray](orange,-yellow,-green,-blue,-violet-and-gray.html)
+. Although it misses a lot of the points and cannot capture the original,
+superimposed cluster centers, it does a decent job of clustering this data.</p>
+
+<p><img src="/assets/img/FuzzyKMeans.png" alt="fuzzy" /></p>
+
+<p>The third image shows the results of running Fuzzy k-Means on a different
+data set which is generated using asymmetrical standard deviations.
+Fuzzy k-Means does a fair job handling this data set as well.</p>
+
+<p><img src="/assets/img/2dFuzzyKMeans.png" alt="fuzzy" /></p>
+
+<p><a name="FuzzyK-Means-References "></a></p>
+<h4 id="references">References </h4>
+
+<ul>
+ <li><a href="http://en.wikipedia.org/wiki/Fuzzy_clustering">http://en.wikipedia.org/wiki/Fuzzy_clustering</a></li>
+</ul>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/hierarchical-clustering.html b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/hierarchical-clustering.html
new file mode 100644
index 0000000..7377566
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/hierarchical-clustering.html
@@ -0,0 +1,200 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Hierarchical Clustering
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/clustering/hierarchical-clustering.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p>Hierarchical clustering is the process or finding bigger clusters, and also
+the smaller clusters inside the bigger clusters.</p>
+
+<p>In Apache Mahout, separate algorithms can be used for finding clusters at
+different levels.</p>
+
+<p>See <a href="https://cwiki.apache.org/confluence/display/MAHOUT/Top+Down+Clustering">Top Down Clustering</a>
+.</p>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/k-means-clustering.html b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/k-means-clustering.html
new file mode 100644
index 0000000..23f8f19
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/k-means-clustering.html
@@ -0,0 +1,371 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) K-Means Clustering
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/clustering/k-means-clustering.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h1 id="k-means-clustering---basics">k-Means clustering - basics</h1>
+
+<p><a href="http://en.wikipedia.org/wiki/Kmeans">k-Means</a> is a simple but well-known algorithm for grouping objects, clustering. All objects need to be represented
+as a set of numerical features. In addition, the user has to specify the
+number of groups (referred to as <em>k</em>) she wishes to identify.</p>
+
+<p>Each object can be thought of as being represented by some feature vector
+in an <em>n</em> dimensional space, <em>n</em> being the number of all features used to
+describe the objects to cluster. The algorithm then randomly chooses <em>k</em>
+points in that vector space, these point serve as the initial centers of
+the clusters. Afterwards all objects are each assigned to the center they
+are closest to. Usually the distance measure is chosen by the user and
+determined by the learning task.</p>
+
+<p>After that, for each cluster a new center is computed by averaging the
+feature vectors of all objects assigned to it. The process of assigning
+objects and recomputing centers is repeated until the process converges.
+The algorithm can be proven to converge after a finite number of
+iterations.</p>
+
+<p>Several tweaks concerning distance measure, initial center choice and
+computation of new average centers have been explored, as well as the
+estimation of the number of clusters <em>k</em>. Yet the main principle always
+remains the same.</p>
+
+<p><a name="K-MeansClustering-Quickstart"></a></p>
+<h2 id="quickstart">Quickstart</h2>
+
+<p><a href="https://github.com/apache/mahout/blob/master/examples/bin/cluster-reuters.sh">Here</a>
+ is a short shell script outline that will get you started quickly with
+k-means. This does the following:</p>
+
+<ul>
+ <li>Accepts clustering type: <em>kmeans</em>, <em>fuzzykmeans</em>, <em>lda</em>, or <em>streamingkmeans</em></li>
+ <li>Gets the Reuters dataset</li>
+ <li>Runs org.apache.lucene.benchmark.utils.ExtractReuters to generate
+reuters-out from reuters-sgm (the downloaded archive)</li>
+ <li>Runs seqdirectory to convert reuters-out to SequenceFile format</li>
+ <li>Runs seq2sparse to convert SequenceFiles to sparse vector format</li>
+ <li>Runs k-means with 20 clusters</li>
+ <li>Runs clusterdump to show results</li>
+</ul>
+
+<p>After following through the output that scrolls past, reading the code will
+offer you a better understanding.</p>
+
+<p><a name="K-MeansClustering-Designofimplementation"></a></p>
+<h2 id="implementation">Implementation</h2>
+
+<p>The implementation accepts two input directories: one for the data points
+and one for the initial clusters. The data directory contains multiple
+input files of SequenceFile(Key, VectorWritable), while the clusters
+directory contains one or more SequenceFiles(Text, Cluster)
+containing <em>k</em> initial clusters or canopies. None of the input directories
+are modified by the implementation, allowing experimentation with initial
+clustering and convergence values.</p>
+
+<p>Canopy clustering can be used to compute the initial clusters for k-KMeans:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>// run the CanopyDriver job
+CanopyDriver.runJob("testdata", "output"
+ManhattanDistanceMeasure.class.getName(), (float) 3.1, (float) 2.1, false);
+
+// now run the KMeansDriver job
+KMeansDriver.runJob("testdata", "output/clusters-0", "output",
+EuclideanDistanceMeasure.class.getName(), "0.001", "10", true);
+</code></pre>
+</div>
+
+<p>In the above example, the input data points are stored in ‘testdata’ and
+the CanopyDriver is configured to output to the ‘output/clusters-0’
+directory. Once the driver executes it will contain the canopy definition
+files. Upon running the KMeansDriver the output directory will have two or
+more new directories: ‘clusters-N’’ containining the clusters for each
+iteration and ‘clusteredPoints’ will contain the clustered data points.</p>
+
+<p>This diagram shows the examplary dataflow of the k-Means example
+implementation provided by Mahout:
+<img src="../../images/Example implementation of k-Means provided with Mahout.png" /></p>
+
+<p><a name="K-MeansClustering-Runningk-MeansClustering"></a></p>
+<h2 id="running-k-means-clustering">Running k-Means Clustering</h2>
+
+<p>The k-Means clustering algorithm may be run using a command-line invocation
+on KMeansDriver.main or by making a Java call to KMeansDriver.runJob().</p>
+
+<p>Invocation using the command line takes the form:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>bin/mahout kmeans \
+ -i <input vectors directory> \
+ -c <input clusters directory> \
+ -o <output working directory> \
+ -k <optional number of initial clusters to sample from input vectors> \
+ -dm <DistanceMeasure> \
+ -x <maximum number of iterations> \
+ -cd <optional convergence delta. Default is 0.5> \
+ -ow <overwrite output directory if present>
+ -cl <run input vector clustering after computing Canopies>
+ -xm <execution method: sequential or mapreduce>
+</code></pre>
+</div>
+
+<p>Note: if the -k argument is supplied, any clusters in the -c directory
+will be overwritten and -k random points will be sampled from the input
+vectors to become the initial cluster centers.</p>
+
+<p>Invocation using Java involves supplying the following arguments:</p>
+
+<ol>
+ <li>input: a file path string to a directory containing the input data set a
+SequenceFile(WritableComparable, VectorWritable). The sequence file <em>key</em>
+is not used.</li>
+ <li>clusters: a file path string to a directory containing the initial
+clusters, a SequenceFile(key, Cluster | Canopy). Both KMeans clusters and
+Canopy canopies may be used for the initial clusters.</li>
+ <li>output: a file path string to an empty directory which is used for all
+output from the algorithm.</li>
+ <li>distanceMeasure: the fully-qualified class name of an instance of
+DistanceMeasure which will be used for the clustering.</li>
+ <li>convergenceDelta: a double value used to determine if the algorithm has
+converged (clusters have not moved more than the value in the last
+iteration)</li>
+ <li>maxIter: the maximum number of iterations to run, independent of the
+convergence specified</li>
+ <li>runClustering: a boolean indicating, if true, that the clustering step is
+to be executed after clusters have been determined.</li>
+ <li>runSequential: a boolean indicating, if true, that the k-means sequential
+implementation is to be used to process the input data.</li>
+</ol>
+
+<p>After running the algorithm, the output directory will contain:</p>
+<ol>
+ <li>clusters-N: directories containing SequenceFiles(Text, Cluster) produced
+by the algorithm for each iteration. The Text <em>key</em> is a cluster identifier
+string.</li>
+ <li>clusteredPoints: (if --clustering enabled) a directory containing
+SequenceFile(IntWritable, WeightedVectorWritable). The IntWritable <em>key</em> is
+the clusterId. The WeightedVectorWritable <em>value</em> is a bean containing a
+double <em>weight</em> and a VectorWritable <em>vector</em> where the weight indicates
+the probability that the vector is a member of the cluster. For k-Means
+clustering, the weights are computed as 1/(1+distance) where the distance
+is between the cluster center and the vector using the chosen
+DistanceMeasure.</li>
+</ol>
+
+<p><a name="K-MeansClustering-Examples"></a></p>
+<h1 id="examples">Examples</h1>
+
+<p>The following images illustrate k-Means clustering applied to a set of
+randomly-generated 2-d data points. The points are generated using a normal
+distribution centered at a mean location and with a constant standard
+deviation. See the README file in the <a href="https://github.com/apache/mahout/blob/master/examples/src/main/java/org/apache/mahout/clustering/display/README.txt">/examples/src/main/java/org/apache/mahout/clustering/display/README.txt</a>
+ for details on running similar examples.</p>
+
+<p>The points are generated as follows:</p>
+
+<ul>
+ <li>500 samples m=[1.0, 1.0](1.0,-1.0.html)
+ sd=3.0</li>
+ <li>300 samples m=[1.0, 0.0](1.0,-0.0.html)
+ sd=0.5</li>
+ <li>300 samples m=[0.0, 2.0](0.0,-2.0.html)
+ sd=0.1</li>
+</ul>
+
+<p>In the first image, the points are plotted and the 3-sigma boundaries of
+their generator are superimposed.</p>
+
+<p><img src="../../images/SampleData.png" alt="Sample data graph" /></p>
+
+<p>In the second image, the resulting clusters (k=3) are shown superimposed upon the sample data. As k-Means is an iterative algorithm, the centers of the clusters in each recent iteration are shown using different colors. Bold red is the final clustering and previous iterations are shown in [orange, yellow, green, blue, violet and gray](orange,-yellow,-green,-blue,-violet-and-gray.html)
+. Although it misses a lot of the points and cannot capture the original,
+superimposed cluster centers, it does a decent job of clustering this data.</p>
+
+<p><img src="../../images/KMeans.png" alt="kmeans" /></p>
+
+<p>The third image shows the results of running k-Means on a different dataset, which is generated using asymmetrical standard deviations.
+K-Means does a fair job handling this data set as well.</p>
+
+<p><img src="../../images/2dKMeans.png" alt="2d kmeans" /></p>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/latent-dirichlet-allocation.html b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/latent-dirichlet-allocation.html
new file mode 100644
index 0000000..fadbcbc
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/latent-dirichlet-allocation.html
@@ -0,0 +1,337 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) Latent Dirichlet Allocation
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/clustering/latent-dirichlet-allocation.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <p><a name="LatentDirichletAllocation-Overview"></a></p>
+<h1 id="overview">Overview</h1>
+
+<p>Latent Dirichlet Allocation (Blei et al, 2003) is a powerful learning
+algorithm for automatically and jointly clustering words into “topics” and
+documents into mixtures of topics. It has been successfully applied to
+model change in scientific fields over time (Griffiths and Steyvers, 2004;
+Hall, et al. 2008).</p>
+
+<p>A topic model is, roughly, a hierarchical Bayesian model that associates
+with each document a probability distribution over “topics”, which are in
+turn distributions over words. For instance, a topic in a collection of
+newswire might include words about “sports”, such as “baseball”, “home
+run”, “player”, and a document about steroid use in baseball might include
+“sports”, “drugs”, and “politics”. Note that the labels “sports”, “drugs”,
+and “politics”, are post-hoc labels assigned by a human, and that the
+algorithm itself only assigns associate words with probabilities. The task
+of parameter estimation in these models is to learn both what the topics
+are, and which documents employ them in what proportions.</p>
+
+<p>Another way to view a topic model is as a generalization of a mixture model
+like <a href="http://en.wikipedia.org/wiki/Dirichlet_process">Dirichlet Process Clustering</a>
+. Starting from a normal mixture model, in which we have a single global
+mixture of several distributions, we instead say that <em>each</em> document has
+its own mixture distribution over the globally shared mixture components.
+Operationally in Dirichlet Process Clustering, each document has its own
+latent variable drawn from a global mixture that specifies which model it
+belongs to, while in LDA each word in each document has its own parameter
+drawn from a document-wide mixture.</p>
+
+<p>The idea is that we use a probabilistic mixture of a number of models that
+we use to explain some observed data. Each observed data point is assumed
+to have come from one of the models in the mixture, but we don’t know
+which. The way we deal with that is to use a so-called latent parameter
+which specifies which model each data point came from.</p>
+
+<p><a name="LatentDirichletAllocation-CollapsedVariationalBayes"></a></p>
+<h1 id="collapsed-variational-bayes">Collapsed Variational Bayes</h1>
+<p>The CVB algorithm which is implemented in Mahout for LDA combines
+advantages of both regular Variational Bayes and Gibbs Sampling. The
+algorithm relies on modeling dependence of parameters on latest variables
+which are in turn mutually independent. The algorithm uses 2
+methodologies to marginalize out parameters when calculating the joint
+distribution and the other other is to model the posterior of theta and phi
+given the inputs z and x.</p>
+
+<p>A common solution to the CVB algorithm is to compute each expectation term
+by using simple Gaussian approximation which is accurate and requires low
+computational overhead. The specifics behind the approximation involve
+computing the sum of the means and variances of the individual Bernoulli
+variables.</p>
+
+<p>CVB with Gaussian approximation is implemented by tracking the mean and
+variance and subtracting the mean and variance of the corresponding
+Bernoulli variables. The computational cost for the algorithm scales on
+the order of O(K) with each update to q(z(i,j)). Also for each
+document/word pair only 1 copy of the variational posterior is required
+over the latent variable.</p>
+
+<p><a name="LatentDirichletAllocation-InvocationandUsage"></a></p>
+<h1 id="invocation-and-usage">Invocation and Usage</h1>
+
+<p>Mahout’s implementation of LDA operates on a collection of SparseVectors of
+word counts. These word counts should be non-negative integers, though
+things will– probably –work fine if you use non-negative reals. (Note
+that the probabilistic model doesn’t make sense if you do!) To create these
+vectors, it’s recommended that you follow the instructions in <a href="../basics/creating-vectors-from-text.html">Creating Vectors From Text</a>
+, making sure to use TF and not TFIDF as the scorer.</p>
+
+<p>Invocation takes the form:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>bin/mahout cvb \
+ -i <input path for document vectors> \
+ -dict <path to term-dictionary file(s) , glob expression supported> \
+ -o <output path for topic-term distributions>
+ -dt <output path for doc-topic distributions> \
+ -k <number of latent topics> \
+ -nt <number of unique features defined by input document vectors> \
+ -mt <path to store model state after each iteration> \
+ -maxIter <max number of iterations> \
+ -mipd <max number of iterations per doc for learning> \
+ -a <smoothing for doc topic distributions> \
+ -e <smoothing for term topic distributions> \
+ -seed <random seed> \
+ -tf <fraction of data to hold for testing> \
+ -block <number of iterations per perplexity check, ignored unless test_set_percentage>0> \
+</code></pre>
+</div>
+
+<p>Topic smoothing should generally be about 50/K, where K is the number of
+topics. The number of words in the vocabulary can be an upper bound, though
+it shouldn’t be too high (for memory concerns).</p>
+
+<p>Choosing the number of topics is more art than science, and it’s
+recommended that you try several values.</p>
+
+<p>After running LDA you can obtain an output of the computed topics using the
+LDAPrintTopics utility:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>bin/mahout ldatopics \
+ -i <input vectors directory> \
+ -d <input dictionary file> \
+ -w <optional number of words to print> \
+ -o <optional output working directory. Default is to console> \
+ -h <print out help> \
+ -dt <optional dictionary type (text|sequencefile). Default is text>
+</code></pre>
+</div>
+
+<p><a name="LatentDirichletAllocation-Example"></a></p>
+<h1 id="example">Example</h1>
+
+<p>An example is located in mahout/examples/bin/build-reuters.sh. The script
+automatically downloads the Reuters-21578 corpus, builds a Lucene index and
+converts the Lucene index to vectors. By uncommenting the last two lines
+you can then cause it to run LDA on the vectors and finally print the
+resultant topics to the console.</p>
+
+<p>To adapt the example yourself, you should note that Lucene has specialized
+support for Reuters, and that building your own index will require some
+adaptation. The rest should hopefully not differ too much.</p>
+
+<p><a name="LatentDirichletAllocation-ParameterEstimation"></a></p>
+<h1 id="parameter-estimation">Parameter Estimation</h1>
+
+<p>We use mean field variational inference to estimate the models. Variational
+inference can be thought of as a generalization of <a href="expectation-maximization.html">EM</a>
+ for hierarchical Bayesian models. The E-Step takes the form of, for each
+document, inferring the posterior probability of each topic for each word
+in each document. We then take the sufficient statistics and emit them in
+the form of (log) pseudo-counts for each word in each topic. The M-Step is
+simply to sum these together and (log) normalize them so that we have a
+distribution over the entire vocabulary of the corpus for each topic.</p>
+
+<p>In implementation, the E-Step is implemented in the Map, and the M-Step is
+executed in the reduce step, with the final normalization happening as a
+post-processing step.</p>
+
+<p><a name="LatentDirichletAllocation-References"></a></p>
+<h1 id="references">References</h1>
+
+<p><a href="-http://machinelearning.wustl.edu/mlpapers/paper_files/BleiNJ03.pdf">David M. Blei, Andrew Y. Ng, Michael I. Jordan, John Lafferty. 2003. Latent Dirichlet Allocation. JMLR.</a></p>
+
+<p><a href="http://psiexp.ss.uci.edu/research/papers/sciencetopics.pdf">Thomas L. Griffiths and Mark Steyvers. 2004. Finding scientific topics. PNAS. </a></p>
+
+<p><a href="-http://aclweb.org/anthology//D/D08/D08-1038.pdf">David Hall, Dan Jurafsky, and Christopher D. Manning. 2008. Studying the History of Ideas Using Topic Models </a></p>
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+</body>
+
+</html>
diff --git a/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/llr---log-likelihood-ratio.html b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/llr---log-likelihood-ratio.html
new file mode 100644
index 0000000..1cfecb5
--- /dev/null
+++ b/docs/0.13.1/api/docs/algorithms/map-reduce/clustering/llr---log-likelihood-ratio.html
@@ -0,0 +1,231 @@
+<!DOCTYPE html>
+<html lang=" en ">
+
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+
+ <title>
+ (Deprecated) LLR - Log-likelihood Ratio
+
+ </title>
+
+ <meta name="description" content="Distributed Linear Algebra">
+
+ <link rel="stylesheet" href="/assets/css/main.css">
+
+ <!-- Font Awesome -->
+ <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet" integrity="sha384-wvfXpqpZZVQGK6TAh5PVlGOfQNHSoD2xbE+QkPxCAFlNEevoEH3Sl0sibVcOQVnN" crossorigin="anonymous">
+
+ <!-- Google Fonts -->
+ <link href="https://fonts.googleapis.com/css?family=Maven+Pro:400,500" rel="stylesheet">
+ <link href="https://fonts.googleapis.com/css?family=Muli:400,400i,700,700i" rel="stylesheet">
+
+ <link rel="canonical" href="http://mahout.apache.org//docs/latest/algorithms/map-reduce/clustering/llr---log-likelihood-ratio.html">
+ <link rel="alternate" type="application/rss+xml" title="Apache Mahout" href="/%20/feed.xml">
+
+
+</head>
+
+
+<body>
+
+ <nav class="navbar navbar-expand-lg navbar-light bg-light navbar-mahout">
+
+ <div class="container">
+
+ <a class="navbar-brand" href="/">
+ <img src="/assets/mahout-logo-blue.svg" alt="">
+ </a>
+
+ <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+
+ <div class="collapse navbar-collapse" id="navbarSupportedContent">
+
+ <div class="navbar-nav ml-auto">
+
+ <!-- Quick Start -->
+ <li class="nav-item">
+ <a class="nav-link" href="/docs/latest" >Overview</a>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Key Concepts</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/index.html">Mahout Overview</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Scala DSL</h6>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/in-core-reference.html">In-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/out-of-core-reference.html">Out-of-core Reference</a>
+ <a class="dropdown-item" href="/docs/latest/mahout-samsara/faq.html">Samsara FAQ</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Distributed Engine Bindings</h6>
+ <a class="dropdown-item" href="/docs/latest/distributed/spark-bindings/">Spark Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">Flink Bindings</a>
+ <a class="dropdown-item" href="/docs/latest/distributed/flink-bindings.html">H20 Bindings</a>
+ <!--<div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Native Solvers</h6>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl.html">ViennaCL</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/viennacl-omp.html">ViennaCL-OMP</a></li>
+ <a class="dropdown-item" href="/docs/latest/native-solvers/cuda.html">CUDA</a></li>-->
+ </div>
+ </li>
+
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Tutorial</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Recommenders</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/cco-lastfm">CCO Example with Last.FM Data</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/intro-cooccurrence-spark">Introduction to Cooccurrence in Spark</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Mahout Samsara</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/play-with-shell.html">Playing with Samsara in Spark Shell</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/playing-with-samsara-flink-batch.html">Playing with Samsara in Flink Batch</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/classify-a-doc-from-the-shell.html">Text Classification (Shell)</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/samsara/spark-naive-bayes.html">Spark Naive Bayes</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Misc</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/mahout-in-zeppelin">Mahout in Apache Zeppelin</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/contributing-algos">How To Contribute a New Algorithm</a>
+ <a class="dropdown-item" href="/docs/latest/tutorials/misc/how-to-build-an-app.html">How To Build An App</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/tutorials/map-reduce">MapReduce</a>
+ </div>
+ </li>
+
+
+ <!-- Algorithms (Samsara / MR) -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Algorithms</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/latest/algorithms/linear-algebra">Distributed Linear Algebra</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/preprocessors">Preprocessors</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/regression">Regression</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/clustering">Clustering</a>
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders">Recommenders</a>
+ <div class="dropdown-divider"></div>
+ <h6 class="dropdown-header">Deprecated</h6>
+ <a class="dropdown-item" href="/docs/latest/algorithms/map-reduce">MapReduce <i>(deprecated)</i></a>
+ </div>
+ <!--<a class="dropdown-item" href="/docs/latest/algorithms/recommenders/recommender-overview.html">Reccomender Overview</a></li> Do we still need? seems like short version of next post-->
+ <!--
+ <a class="dropdown-item" href="/docs/latest/algorithms/recommenders/intro-cooccurrence-spark.html">Intro to Coocurrence With Spark</a></li>
+ <li role="separator" class="divider"></li>
+ <li><span> <a href="/docs/latest/algorithms/map-reduce"><b>MapReduce</b> (deprecated)</a><span></li>
+
+
+ -->
+ </li>
+
+ <!-- Scala /docs -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">API /docs</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="/docs/0.13.0/api/docs/">0.13.0</a>
+ </div>
+ </li>
+
+ <!-- Apache -->
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="" id="navbarDropdownMenuLink" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+ <a class="dropdown-item" href="http://www.apache.org/foundation/how-it-works.html">Apache Software Foundation</a>
+ <a class="dropdown-item" href="http://www.apache.org/licenses/">Apache License</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
+ <a class="dropdown-item" href="http://www.apache.org/foundation/thanks.html">Thanks</a>
+ </div>
+ </li>
+
+ </ul>
+
+ <!--<form class="navbar-form navbar-left">-->
+ <!--<div class="form-group">-->
+ <!--<input type="text" class="form-control" placeholder="Search">-->
+ <!--</div>-->
+ <!--<button type="submit" class="btn btn-default">Submit</button>-->
+ <!--</form>-->
+ <!--<ul class="nav navbar-nav navbar-right">-->
+ <!--<a class="dropdown-item" href="http://github.com/apache/mahout">Github</a></li>-->
+
+
+
+ <!--</ul>-->
+ </div><!-- /.navbar-collapse -->
+ </div>
+</nav>
+
+
+ <div class="container mt-5 pb-4">
+
+ <div class="row">
+
+ <div class="col-lg-8">
+ <h1 id="likelihood-ratio-test">Likelihood ratio test</h1>
+
+<p><em>Likelihood ratio test is used to compare the fit of two models one
+of which is nested within the other.</em></p>
+
+<p>In the context of machine learning and the Mahout project in particular,
+the term LLR is usually meant to refer to a test of significance for two
+binomial distributions, also known as the G squared statistic. This is a
+special case of the multinomial test and is closely related to mutual
+information. The value of this statistic is not normally used in this
+context as a true frequentist test of significance since there would be
+obvious and dreadful problems to do with multiple comparisons, but rather
+as a heuristic score to order pairs of items with the most interestingly
+connected items having higher scores. In this usage, the LLR has proven
+very useful for discriminating pairs of features that have interesting
+degrees of cooccurrence and those that do not with usefully small false
+positive and false negative rates. The LLR is typically far more suitable
+in the case of small than many other measures such as Pearson’s
+correlation, Pearson’s chi squared statistic or z statistics. The LLR as
+stated does not, however, make any use of rating data which can limit its
+applicability in problems such as the Netflix competition.</p>
+
+<p>The actual value of the LLR is not usually very helpful other than as a way
+of ordering pairs of items. As such, it is often used to determine a
+sparse set of coefficients to be estimated by other means such as TF-IDF.
+Since the actual estimation of these coefficients can be done in a way that
+is independent of the training data such as by general corpus statistics,
+and since the ordering imposed by the LLR is relatively robust to counting
+fluctuation, this technique can provide very strong results in very sparse
+problems where the potential number of features vastly out-numbers the
+number of training examples and where features are highly interdependent.</p>
+
+<p>See Also:</p>
+
+<ul>
+ <li><a href="http://tdunning.blogspot.com/2008/03/surprise-and-coincidence.html">Blog post “surprise and coincidence”</a></li>
+ <li><a href="http://en.wikipedia.org/wiki/G-test">G-Test</a></li>
+ <li><a href="http://en.wikipedia.org/wiki/Likelihood-ratio_test">Likelihood Ratio Test</a></li>
+</ul>
+
+
+ </div>
+
+
+ </div>
+
+</div>
+
+
+ <footer class="footer bg-light">
+ <div class="container text-center small">
+ Copyright © 2014-2019 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
+ </div>
+</footer>
+
+ <script src="/assets/vendor/jquery/jquery-slim.min.js"></script>
+ <script src="/assets/vendor/popper/popper.min.js"></script>
+ <script src="/assets/vendor/bootstrap/js/bootstrap.min.js"></script>
+ <script src="/assets/header.js"></script>
... 62784 lines suppressed ...