You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mahout.apache.org by bu...@apache.org on 2015/03/19 22:21:47 UTC
svn commit: r944380 [22/24] - in /websites/staging/mahout/trunk/content: ./
developers/ general/ users/basics/ users/classification/ users/clustering/
users/dim-reduction/ users/mapreduce/ users/mapreduce/classification/
users/mapreduce/clustering/ use...
Added: websites/staging/mahout/trunk/content/users/mapreduce/recommender/quickstart.html
==============================================================================
--- websites/staging/mahout/trunk/content/users/mapreduce/recommender/quickstart.html (added)
+++ websites/staging/mahout/trunk/content/users/mapreduce/recommender/quickstart.html Thu Mar 19 21:21:45 2015
@@ -0,0 +1,294 @@
+<!DOCTYPE html>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+ <title>Apache Mahout: Scalable machine learning and data mining</title>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+ <meta name="Distribution" content="Global">
+ <meta name="Robots" content="index,follow">
+ <meta name="keywords" content="apache, apache hadoop, apache lucene,
+ business data mining, cluster analysis,
+ collaborative filtering, data extraction, data filtering, data framework, data integration,
+ data matching, data mining, data mining algorithms, data mining analysis, data mining data,
+ data mining introduction, data mining software,
+ data mining techniques, data representation, data set, datamining,
+ feature extraction, fuzzy k means, genetic algorithm, hadoop,
+ hierarchical clustering, high dimensional, introduction to data mining, kmeans,
+ knowledge discovery, learning approach, learning approaches, learning methods,
+ learning techniques, lucene, machine learning, machine translation, mahout apache,
+ mahout taste, map reduce hadoop, mining data, mining methods, naive bayes,
+ natural language processing,
+ supervised, text mining, time series data, unsupervised, web data mining">
+ <link rel="shortcut icon" type="image/x-icon" href="http://mahout.apache.org/images/favicon.ico">
+ <script type="text/javascript" src="/js/prototype.js"></script>
+ <script type="text/javascript" src="/js/effects.js"></script>
+ <script type="text/javascript" src="/js/search.js"></script>
+ <script type="text/javascript" src="/js/slides.js"></script>
+
+ <link href="/css/bootstrap.min.css" rel="stylesheet" media="screen">
+ <link href="/css/bootstrap-responsive.css" rel="stylesheet">
+ <link rel="stylesheet" href="/css/global.css" type="text/css">
+
+ <!-- mathJax stuff -- use `\(...\)` for inline style math in markdown -->
+ <script type="text/x-mathjax-config">
+ MathJax.Hub.Config({
+ tex2jax: {
+ skipTags: ['script', 'noscript', 'style', 'textarea', 'pre']
+ }
+ });
+ MathJax.Hub.Queue(function() {
+ var all = MathJax.Hub.getAllJax(), i;
+ for(i = 0; i < all.length; i += 1) {
+ all[i].SourceElement().parentNode.className += ' has-jax';
+ }
+ });
+ </script>
+ <script type="text/javascript">
+ var mathjax = document.createElement('script');
+ mathjax.type = 'text/javascript';
+ mathjax.async = true;
+
+ mathjax.src = ('https:' == document.location.protocol) ?
+ 'https://c328740.ssl.cf1.rackcdn.com/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' :
+ 'http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML';
+
+ var s = document.getElementsByTagName('script')[0];
+ s.parentNode.insertBefore(mathjax, s);
+ </script>
+</head>
+
+<body id="home" data-twttr-rendered="true">
+ <div id="wrap">
+ <div id="header">
+ <div id="logo"><a href="/overview.html"></a></div>
+ <div id="search">
+ <form id="search-form" action="http://www.google.com/search" method="get" class="navbar-search pull-right">
+ <input value="http://mahout.apache.org" name="sitesearch" type="hidden">
+ <input class="search-query" name="q" id="query" type="text">
+ <input id="submission" type="image" src="/images/mahout-lupe.png" alt="Search" />
+ </form>
+ </div>
+
+ <div class="navbar navbar-inverse" style="position:absolute;top:133px;padding-right:0px;padding-left:0px;">
+ <div class="navbar-inner" style="border: none; background: #999; border: none; border-radius: 0px;">
+ <div class="container">
+ <button type="button" class="btn btn-navbar" data-toggle="collapse" data-target=".nav-collapse">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <!-- <a class="brand" href="#">Apache Community Development Project</a> -->
+ <div class="nav-collapse collapse">
+ <ul class="nav">
+ <li><a href="/">Home</a></li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">General<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/general/downloads.html">Downloads</a>
+ <li><a href="/general/who-we-are.html">Who we are</a>
+ <li><a href="/general/mailing-lists,-irc-and-archives.html">Mailing Lists</a>
+ <li><a href="/general/release-notes.html">Release Notes</a>
+ <li><a href="/general/books-tutorials-and-talks.html">Books, Tutorials, Talks</a></li>
+ <li><a href="/general/powered-by-mahout.html">Powered By Mahout</a>
+ <li><a href="/general/professional-support.html">Professional Support</a>
+ <li class="divider"></li>
+ <li class="nav-header">Resources</li>
+ <li><a href="/general/reference-reading.html">Reference Reading</a>
+ <li><a href="/general/faq.html">FAQ</a>
+ <li class="divider"></li>
+ <li class="nav-header">Legal</li>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ <li><a href="/general/privacy-policy.html">Privacy Policy</a>
+ </ul>
+ </li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Developers<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/developers/developer-resources.html">Developer resources</a></li>
+ <li><a href="/developers/version-control.html">Version control</a></li>
+ <li><a href="/developers/buildingmahout.html">Build from source</a></li>
+ <li><a href="/developers/issue-tracker.html">Issue tracker</a></li>
+ <li><a href="https://builds.apache.org/job/Mahout-Quality/" target="_blank">Code quality reports</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Contributions</li>
+ <li><a href="/developers/how-to-contribute.html">How to contribute</a></li>
+ <li><a href="/developers/how-to-become-a-committer.html">How to become a committer</a></li>
+ <li><a href="/developers/gsoc.html">GSoC</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">For committers</li>
+ <li><a href="/developers/how-to-update-the-website.html">How to update the website</a></li>
+ <li><a href="/developers/patch-check-list.html">Patch check list</a></li>
+ <li><a href="/developers/github.html">Handling Github PRs</a></li>
+ <li><a href="/developers/how-to-release.html">How to release</a></li>
+ <li><a href="/developers/thirdparty-dependencies.html">Third party dependencies</a></li>
+ </ul>
+ </li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Basics<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/basics/algorithms.html">List of algorithms</a>
+ <li><a href="/users/basics/quickstart.html">Quickstart</a>
+ <li class="divider"></li>
+ <li class="nav-header">Working with text</li>
+ <li><a href="/users/basics/creating-vectors-from-text.html">Creating vectors from text</a>
+ <li><a href="/users/basics/collocations.html">Collocations</a>
+ <li class="divider"></li>
+ <li class="nav-header">Dimensionality reduction</li>
+ <li><a href="/users/dim-reduction/dimensional-reduction.html">Singular Value Decomposition</a></li>
+ <li><a href="/users/dim-reduction/ssvd.html">Stochastic SVD</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Topic Models</li>
+ <li><a href="/users/clustering/latent-dirichlet-allocation.html">Latent Dirichlet Allocation</a></li>
+ </ul>
+ </li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Spark<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/sparkbindings/home.html">Scala & Spark Bindings Overview</a></li>
+ <li><a href="/users/sparkbindings/play-with-shell.html">Playing with Mahout's Spark Shell</a></li>
+ <li class="divider"></li>
+ <li><a href="/users/sparkbindings/faq.html">FAQ</a></li>
+ </ul>
+ </li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Classification<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/mapreduce/classification/bayesian.html">Naive Bayes</a></li>
+ <li><a href="/users/mapreduce/classification/hidden-markov-models.html">Hidden Markov Models</a></li>
+ <li><a href="/users/mapreduce/classification/logistic-regression.html">Logistic Regression</a></li>
+ <li><a href="/users/mapreduce/classification/partial-implementation.html">Random Forest</a></li>
+
+ <li class="divider"></li>
+ <li class="nav-header">Examples</li>
+ <li><a href="/users/mapreduce/classification/breiman-example.html">Breiman example</a></li>
+ <li><a href="/users/mapreduce/classification/twenty-newsgroups.html">20 newsgroups example</a></li>
+ </ul></li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Clustering<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/mapreduce/clustering/k-means-clustering.html">k-Means</a></li>
+ <li><a href="/users/mapreduce/clustering/canopy-clustering.html">Canopy</a></li>
+ <li><a href="/users/mapreduce/clustering/fuzzy-k-means.html">Fuzzy k-Means</a></li>
+ <li><a href="/users/mapreduce/clustering/streaming-k-means.html">Streaming KMeans</a></li>
+ <li><a href="/users/mapreduce/clustering/spectral-clustering.html">Spectral Clustering</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Commandline usage</li>
+ <li><a href="/users/mapreduce/clustering/k-means-commandline.html">Options for k-Means</a></li>
+ <li><a href="/users/mapreduce/clustering/canopy-commandline.html">Options for Canopy</a></li>
+ <li><a href="/users/mapreduce/clustering/fuzzy-k-means-commandline.html">Options for Fuzzy k-Means</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Examples</li>
+ <li><a href="/users/mapreduce/clustering/clustering-of-synthetic-control-data.html">Synthetic data</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Post processing</li>
+ <li><a href="/users/mapreduce/clustering/cluster-dumper.html">Cluster Dumper tool</a></li>
+ <li><a href="/users/mapreduce/clustering/visualizing-sample-clusters.html">Cluster visualisation</a></li>
+ </ul></li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Recommendations<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/mapreduce/recommender/quickstart.html">Quickstart</a></li>
+ <li><a href="/users/mapreduce/recommender/recommender-first-timer-faq.html">First Timer FAQ</a></li>
+ <li><a href="/users/mapreduce/recommender/userbased-5-minutes.html">A user-based recommender <br/>in 5 minutes</a></li>
+ <li><a href="/users/mapreduce/recommender/matrix-factorization.html">Matrix factorization-based<br/> recommenders</a></li>
+ <li><a href="/users/mapreduce/recommender/recommender-documentation.html">Overview</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Hadoop</li>
+ <li><a href="/users/mapreduce/recommender/intro-itembased-hadoop.html">Intro to item-based recommendations<br/> with Hadoop</a></li>
+ <li><a href="/users/mapreduce/recommender/intro-als-hadoop.html">Intro to ALS recommendations<br/> with Hadoop</a></li>
+ <li class="nav-header">Spark</li>
+ <li><a href="/users/mapreduce/recommender/intro-cooccurrence-spark.html">Intro to cooccurrence-based<br/> recommendations with Spark</a></li>
+ </ul>
+ </li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ </div>
+ </div>
+ </div>
+
+</div>
+
+ <div id="sidebar">
+ <div id="sidebar-wrap">
+ <h2>Twitter</h2>
+ <ul class="sidemenu">
+ <li>
+<a class="twitter-timeline" href="https://twitter.com/ApacheMahout" data-widget-id="422861673444028416">Tweets by @ApacheMahout</a>
+<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+"://platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
+</li>
+ </ul>
+ <h2>Apache Software Foundation</h2>
+ <ul class="sidemenu">
+ <li><a href="http://www.apache.org/foundation/how-it-works.html">How the ASF works</a></li>
+ <li><a href="http://www.apache.org/foundation/getinvolved.html">Get Involved</a></li>
+ <li><a href="http://www.apache.org/dev/">Developer Resources</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ </ul>
+ <h2>Related Projects</h2>
+ <ul class="sidemenu">
+ <li><a href="http://lucene.apache.org/">Lucene</a></li>
+ <li><a href="http://hadoop.apache.org/">Hadoop</a></li>
+ </ul>
+ </div>
+</div>
+
+ <div id="content-wrap" class="clearfix">
+ <div id="main">
+ <h1 id="recommender-overview">Recommender Overview</h1>
+<p>Recommenders have changed over the years. Mahout contains a long list of them, which you can still use. But to get the best out of our more modern aproach we'll need to think of the Recommender as a "model creation" component—supplied by Mahout's new spark-itemsimilarity job, and a "serving" component—supplied by a modern scalable search engine, like Solr.</p>
+<p><img alt="image" src="http://i.imgur.com/fliHMBo.png" /></p>
+<p>To integrate with your application you will collect user interactions storing them in a DB and also in a from usable by Mahout. The simplest way to do this is to log user interactions to csv files (user-id, item-id). The DB should be setup to contain the last n user interactions, which will form part of the query for recommendations.</p>
+<p>Mahout's spark-itemsimilarity will create a table of (item-id, list-of-similar-items) in csv form. Think of this as an item collection with one field containing the item-ids of similar items. Index this with your search engine. </p>
+<p>When your application needs recommendations for a specific person, get the latest user history of interactions from the DB and query the indicator collection with this history. You will get back an ordered list of item-ids. These are your recommendations. You may wish to filter out any that the user has already seen but that will depend on your use case.</p>
+<p>All ids for users and items are preserved as string tokens and so work as an external key in DBs or as doc ids for search engines, they also work as tokens for search queries.</p>
+<h2 id="references">References</h2>
+<ol>
+<li>A free ebook, which talks about the general idea: <a href="https://www.mapr.com/practical-machine-learning">Practical Machine Learning</a></li>
+<li>A slide deck, which talks about mixing actions or other indicators: <a href="http://occamsmachete.com/ml/2014/10/07/creating-a-unified-recommender-with-mahout-and-a-search-engine/">Creating a Multimodal Recommender with Mahout and a Search Engine</a></li>
+<li>Two blog posts: <a href="http://occamsmachete.com/ml/2014/08/11/mahout-on-spark-whats-new-in-recommenders/">What's New in Recommenders: part #1</a>
+and <a href="http://occamsmachete.com/ml/2014/09/09/mahout-on-spark-whats-new-in-recommenders-part-2/">What's New in Recommenders: part #2</a></li>
+<li>A post describing the loglikelihood ratio: <a href="http://tdunning.blogspot.com/2008/03/surprise-and-coincidence.html">Surprise and Coinsidense</a> LLR is used to reduce noise in the data while keeping the calculations O(n) complexity.</li>
+</ol>
+<h2 id="mahout-model-creation">Mahout Model Creation</h2>
+<p>See the page describing <a href="http://mahout.apache.org/users/recommender/intro-cooccurrence-spark.html"><em>spark-itemsimilarity</em></a> for more details.</p>
+ </div>
+ </div>
+</div>
+ <footer class="footer" align="center">
+ <div class="container">
+ <p>
+ Copyright © 2014 The Apache Software Foundation, Licensed under
+ the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.
+ <br />
+ Apache and the Apache feather logos are trademarks of The Apache Software Foundation.
+ </p>
+ </div>
+ </footer>
+
+ <script src="/js/jquery-1.9.1.min.js"></script>
+ <script src="/js/bootstrap.min.js"></script>
+ <script>
+ (function() {
+ var cx = '012254517474945470291:vhsfv7eokdc';
+ var gcse = document.createElement('script');
+ gcse.type = 'text/javascript';
+ gcse.async = true;
+ gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +
+ '//www.google.com/cse/cse.js?cx=' + cx;
+ var s = document.getElementsByTagName('script')[0];
+ s.parentNode.insertBefore(gcse, s);
+ })();
+ </script>
+</body>
+</html>
Added: websites/staging/mahout/trunk/content/users/mapreduce/recommender/recommender-documentation.html
==============================================================================
--- websites/staging/mahout/trunk/content/users/mapreduce/recommender/recommender-documentation.html (added)
+++ websites/staging/mahout/trunk/content/users/mapreduce/recommender/recommender-documentation.html Thu Mar 19 21:21:45 2015
@@ -0,0 +1,511 @@
+<!DOCTYPE html>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+ <title>Apache Mahout: Scalable machine learning and data mining</title>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+ <meta name="Distribution" content="Global">
+ <meta name="Robots" content="index,follow">
+ <meta name="keywords" content="apache, apache hadoop, apache lucene,
+ business data mining, cluster analysis,
+ collaborative filtering, data extraction, data filtering, data framework, data integration,
+ data matching, data mining, data mining algorithms, data mining analysis, data mining data,
+ data mining introduction, data mining software,
+ data mining techniques, data representation, data set, datamining,
+ feature extraction, fuzzy k means, genetic algorithm, hadoop,
+ hierarchical clustering, high dimensional, introduction to data mining, kmeans,
+ knowledge discovery, learning approach, learning approaches, learning methods,
+ learning techniques, lucene, machine learning, machine translation, mahout apache,
+ mahout taste, map reduce hadoop, mining data, mining methods, naive bayes,
+ natural language processing,
+ supervised, text mining, time series data, unsupervised, web data mining">
+ <link rel="shortcut icon" type="image/x-icon" href="http://mahout.apache.org/images/favicon.ico">
+ <script type="text/javascript" src="/js/prototype.js"></script>
+ <script type="text/javascript" src="/js/effects.js"></script>
+ <script type="text/javascript" src="/js/search.js"></script>
+ <script type="text/javascript" src="/js/slides.js"></script>
+
+ <link href="/css/bootstrap.min.css" rel="stylesheet" media="screen">
+ <link href="/css/bootstrap-responsive.css" rel="stylesheet">
+ <link rel="stylesheet" href="/css/global.css" type="text/css">
+
+ <!-- mathJax stuff -- use `\(...\)` for inline style math in markdown -->
+ <script type="text/x-mathjax-config">
+ MathJax.Hub.Config({
+ tex2jax: {
+ skipTags: ['script', 'noscript', 'style', 'textarea', 'pre']
+ }
+ });
+ MathJax.Hub.Queue(function() {
+ var all = MathJax.Hub.getAllJax(), i;
+ for(i = 0; i < all.length; i += 1) {
+ all[i].SourceElement().parentNode.className += ' has-jax';
+ }
+ });
+ </script>
+ <script type="text/javascript">
+ var mathjax = document.createElement('script');
+ mathjax.type = 'text/javascript';
+ mathjax.async = true;
+
+ mathjax.src = ('https:' == document.location.protocol) ?
+ 'https://c328740.ssl.cf1.rackcdn.com/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' :
+ 'http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML';
+
+ var s = document.getElementsByTagName('script')[0];
+ s.parentNode.insertBefore(mathjax, s);
+ </script>
+</head>
+
+<body id="home" data-twttr-rendered="true">
+ <div id="wrap">
+ <div id="header">
+ <div id="logo"><a href="/overview.html"></a></div>
+ <div id="search">
+ <form id="search-form" action="http://www.google.com/search" method="get" class="navbar-search pull-right">
+ <input value="http://mahout.apache.org" name="sitesearch" type="hidden">
+ <input class="search-query" name="q" id="query" type="text">
+ <input id="submission" type="image" src="/images/mahout-lupe.png" alt="Search" />
+ </form>
+ </div>
+
+ <div class="navbar navbar-inverse" style="position:absolute;top:133px;padding-right:0px;padding-left:0px;">
+ <div class="navbar-inner" style="border: none; background: #999; border: none; border-radius: 0px;">
+ <div class="container">
+ <button type="button" class="btn btn-navbar" data-toggle="collapse" data-target=".nav-collapse">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <!-- <a class="brand" href="#">Apache Community Development Project</a> -->
+ <div class="nav-collapse collapse">
+ <ul class="nav">
+ <li><a href="/">Home</a></li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">General<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/general/downloads.html">Downloads</a>
+ <li><a href="/general/who-we-are.html">Who we are</a>
+ <li><a href="/general/mailing-lists,-irc-and-archives.html">Mailing Lists</a>
+ <li><a href="/general/release-notes.html">Release Notes</a>
+ <li><a href="/general/books-tutorials-and-talks.html">Books, Tutorials, Talks</a></li>
+ <li><a href="/general/powered-by-mahout.html">Powered By Mahout</a>
+ <li><a href="/general/professional-support.html">Professional Support</a>
+ <li class="divider"></li>
+ <li class="nav-header">Resources</li>
+ <li><a href="/general/reference-reading.html">Reference Reading</a>
+ <li><a href="/general/faq.html">FAQ</a>
+ <li class="divider"></li>
+ <li class="nav-header">Legal</li>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ <li><a href="/general/privacy-policy.html">Privacy Policy</a>
+ </ul>
+ </li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Developers<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/developers/developer-resources.html">Developer resources</a></li>
+ <li><a href="/developers/version-control.html">Version control</a></li>
+ <li><a href="/developers/buildingmahout.html">Build from source</a></li>
+ <li><a href="/developers/issue-tracker.html">Issue tracker</a></li>
+ <li><a href="https://builds.apache.org/job/Mahout-Quality/" target="_blank">Code quality reports</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Contributions</li>
+ <li><a href="/developers/how-to-contribute.html">How to contribute</a></li>
+ <li><a href="/developers/how-to-become-a-committer.html">How to become a committer</a></li>
+ <li><a href="/developers/gsoc.html">GSoC</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">For committers</li>
+ <li><a href="/developers/how-to-update-the-website.html">How to update the website</a></li>
+ <li><a href="/developers/patch-check-list.html">Patch check list</a></li>
+ <li><a href="/developers/github.html">Handling Github PRs</a></li>
+ <li><a href="/developers/how-to-release.html">How to release</a></li>
+ <li><a href="/developers/thirdparty-dependencies.html">Third party dependencies</a></li>
+ </ul>
+ </li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Basics<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/basics/algorithms.html">List of algorithms</a>
+ <li><a href="/users/basics/quickstart.html">Quickstart</a>
+ <li class="divider"></li>
+ <li class="nav-header">Working with text</li>
+ <li><a href="/users/basics/creating-vectors-from-text.html">Creating vectors from text</a>
+ <li><a href="/users/basics/collocations.html">Collocations</a>
+ <li class="divider"></li>
+ <li class="nav-header">Dimensionality reduction</li>
+ <li><a href="/users/dim-reduction/dimensional-reduction.html">Singular Value Decomposition</a></li>
+ <li><a href="/users/dim-reduction/ssvd.html">Stochastic SVD</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Topic Models</li>
+ <li><a href="/users/clustering/latent-dirichlet-allocation.html">Latent Dirichlet Allocation</a></li>
+ </ul>
+ </li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Spark<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/sparkbindings/home.html">Scala & Spark Bindings Overview</a></li>
+ <li><a href="/users/sparkbindings/play-with-shell.html">Playing with Mahout's Spark Shell</a></li>
+ <li class="divider"></li>
+ <li><a href="/users/sparkbindings/faq.html">FAQ</a></li>
+ </ul>
+ </li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Classification<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/mapreduce/classification/bayesian.html">Naive Bayes</a></li>
+ <li><a href="/users/mapreduce/classification/hidden-markov-models.html">Hidden Markov Models</a></li>
+ <li><a href="/users/mapreduce/classification/logistic-regression.html">Logistic Regression</a></li>
+ <li><a href="/users/mapreduce/classification/partial-implementation.html">Random Forest</a></li>
+
+ <li class="divider"></li>
+ <li class="nav-header">Examples</li>
+ <li><a href="/users/mapreduce/classification/breiman-example.html">Breiman example</a></li>
+ <li><a href="/users/mapreduce/classification/twenty-newsgroups.html">20 newsgroups example</a></li>
+ </ul></li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Clustering<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/mapreduce/clustering/k-means-clustering.html">k-Means</a></li>
+ <li><a href="/users/mapreduce/clustering/canopy-clustering.html">Canopy</a></li>
+ <li><a href="/users/mapreduce/clustering/fuzzy-k-means.html">Fuzzy k-Means</a></li>
+ <li><a href="/users/mapreduce/clustering/streaming-k-means.html">Streaming KMeans</a></li>
+ <li><a href="/users/mapreduce/clustering/spectral-clustering.html">Spectral Clustering</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Commandline usage</li>
+ <li><a href="/users/mapreduce/clustering/k-means-commandline.html">Options for k-Means</a></li>
+ <li><a href="/users/mapreduce/clustering/canopy-commandline.html">Options for Canopy</a></li>
+ <li><a href="/users/mapreduce/clustering/fuzzy-k-means-commandline.html">Options for Fuzzy k-Means</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Examples</li>
+ <li><a href="/users/mapreduce/clustering/clustering-of-synthetic-control-data.html">Synthetic data</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Post processing</li>
+ <li><a href="/users/mapreduce/clustering/cluster-dumper.html">Cluster Dumper tool</a></li>
+ <li><a href="/users/mapreduce/clustering/visualizing-sample-clusters.html">Cluster visualisation</a></li>
+ </ul></li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Recommendations<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/mapreduce/recommender/quickstart.html">Quickstart</a></li>
+ <li><a href="/users/mapreduce/recommender/recommender-first-timer-faq.html">First Timer FAQ</a></li>
+ <li><a href="/users/mapreduce/recommender/userbased-5-minutes.html">A user-based recommender <br/>in 5 minutes</a></li>
+ <li><a href="/users/mapreduce/recommender/matrix-factorization.html">Matrix factorization-based<br/> recommenders</a></li>
+ <li><a href="/users/mapreduce/recommender/recommender-documentation.html">Overview</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Hadoop</li>
+ <li><a href="/users/mapreduce/recommender/intro-itembased-hadoop.html">Intro to item-based recommendations<br/> with Hadoop</a></li>
+ <li><a href="/users/mapreduce/recommender/intro-als-hadoop.html">Intro to ALS recommendations<br/> with Hadoop</a></li>
+ <li class="nav-header">Spark</li>
+ <li><a href="/users/mapreduce/recommender/intro-cooccurrence-spark.html">Intro to cooccurrence-based<br/> recommendations with Spark</a></li>
+ </ul>
+ </li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ </div>
+ </div>
+ </div>
+
+</div>
+
+ <div id="sidebar">
+ <div id="sidebar-wrap">
+ <h2>Twitter</h2>
+ <ul class="sidemenu">
+ <li>
+<a class="twitter-timeline" href="https://twitter.com/ApacheMahout" data-widget-id="422861673444028416">Tweets by @ApacheMahout</a>
+<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+"://platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
+</li>
+ </ul>
+ <h2>Apache Software Foundation</h2>
+ <ul class="sidemenu">
+ <li><a href="http://www.apache.org/foundation/how-it-works.html">How the ASF works</a></li>
+ <li><a href="http://www.apache.org/foundation/getinvolved.html">Get Involved</a></li>
+ <li><a href="http://www.apache.org/dev/">Developer Resources</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ </ul>
+ <h2>Related Projects</h2>
+ <ul class="sidemenu">
+ <li><a href="http://lucene.apache.org/">Lucene</a></li>
+ <li><a href="http://hadoop.apache.org/">Hadoop</a></li>
+ </ul>
+ </div>
+</div>
+
+ <div id="content-wrap" class="clearfix">
+ <div id="main">
+ <p><a name="RecommenderDocumentation-Overview"></a></p>
+<h2 id="overview">Overview</h2>
+<p><em>This documentation concerns the non-distributed, non-Hadoop-based
+recommender engine / collaborative filtering code inside Mahout. It was
+formerly a separate project called "Taste" and has continued development
+inside Mahout alongside other Hadoop-based code. It may be viewed as a
+somewhat separate, more comprehensive and more mature aspect of this
+code, compared to current development efforts focusing on Hadoop-based
+distributed recommenders. This remains the best entry point into Mahout
+recommender engines of all kinds.</em></p>
+<p>A Mahout-based collaborative filtering engine takes users' preferences for
+items ("tastes") and returns estimated preferences for other items. For
+example, a site that sells books or CDs could easily use Mahout to figure
+out, from past purchase data, which CDs a customer might be interested in
+listening to.</p>
+<p>Mahout provides a rich set of components from which you can construct a
+customized recommender system from a selection of algorithms. Mahout is
+designed to be enterprise-ready; it's designed for performance, scalability
+and flexibility.</p>
+<p>Top-level packages define the Mahout interfaces to these key abstractions:</p>
+<ul>
+<li><strong>DataModel</strong></li>
+<li><strong>UserSimilarity</strong></li>
+<li><strong>ItemSimilarity</strong></li>
+<li><strong>UserNeighborhood</strong></li>
+<li><strong>Recommender</strong></li>
+</ul>
+<p>Subpackages of <em>org.apache.mahout.cf.taste.impl</em> hold implementations of
+these interfaces. These are the pieces from which you will build your own
+recommendation engine. That's it! </p>
+<p><a name="RecommenderDocumentation-Architecture"></a></p>
+<h2 id="architecture">Architecture</h2>
+<p><img alt="doc" src="../../images/taste-architecture.png" /></p>
+<p>This diagram shows the relationship between various Mahout components in a
+user-based recommender. An item-based recommender system is similar except
+that there are no Neighborhood algorithms involved.</p>
+<p><a name="RecommenderDocumentation-Recommender"></a></p>
+<h3 id="recommender">Recommender</h3>
+<p>A Recommender is the core abstraction in Mahout. Given a DataModel, it can
+produce recommendations. Applications will most likely use the
+<strong>GenericUserBasedRecommender</strong> or <strong>GenericItemBasedRecommender</strong>,
+possibly decorated by <strong>CachingRecommender</strong>.</p>
+<p><a name="RecommenderDocumentation-DataModel"></a></p>
+<h3 id="datamodel">DataModel</h3>
+<p>A <strong>DataModel</strong> is the interface to information about user preferences. An
+implementation might draw this data from any source, but a database is the
+most likely source. Be sure to wrap this with a <strong>ReloadFromJDBCDataModel</strong> to get good performance! Mahout provides <strong>MySQLJDBCDataModel</strong>, for example, to access preference data from a database via JDBC and MySQL. Another exists for PostgreSQL. Mahout also provides a <strong>FileDataModel</strong>, which is fine for small applications.</p>
+<p>Users and items are identified solely by an ID value in the
+framework. Further, this ID value must be numeric; it is a Java long type
+through the APIs. A <strong>Preference</strong> object or <strong>PreferenceArray</strong> object
+encapsulates the relation between user and preferred items (or items and
+users preferring them).</p>
+<p>Finally, Mahout supports, in various ways, a so-called "boolean" data model
+in which users do not express preferences of varying strengths for items,
+but simply express an association or none at all. For example, while users
+might express a preference from 1 to 5 in the context of a movie
+recommender site, there may be no notion of a preference value between
+users and pages in the context of recommending pages on a web site: there
+is only a notion of an association, or none, between a user and pages that
+have been visited.</p>
+<p><a name="RecommenderDocumentation-UserSimilarity"></a></p>
+<h3 id="usersimilarity">UserSimilarity</h3>
+<p>A <strong>UserSimilarity</strong> defines a notion of similarity between two users. This is
+a crucial part of a recommendation engine. These are attached to a
+<strong>Neighborhood</strong> implementation. <strong>ItemSimilarity</strong> is analagous, but find
+similarity between items.</p>
+<p><a name="RecommenderDocumentation-UserNeighborhood"></a></p>
+<h3 id="userneighborhood">UserNeighborhood</h3>
+<p>In a user-based recommender, recommendations are produced by finding a
+"neighborhood" of similar users near a given user. A <strong>UserNeighborhood</strong>
+defines a means of determining that neighborhood — for example,
+nearest 10 users. Implementations typically need a <strong>UserSimilarity</strong> to
+operate.</p>
+<p><a name="RecommenderDocumentation-Examples"></a></p>
+<h2 id="examples">Examples</h2>
+<p><a name="RecommenderDocumentation-User-basedRecommender"></a></p>
+<h3 id="user-based-recommender">User-based Recommender</h3>
+<p>User-based recommenders are the "original", conventional style of
+recommender systems. They can produce good recommendations when tweaked
+properly; they are not necessarily the fastest recommender systems and are
+thus suitable for small data sets (roughly, less than ten million ratings).
+We'll start with an example of this.</p>
+<p>First, create a <strong>DataModel</strong> of some kind. Here, we'll use a simple on based
+on data in a file. The file should be in CSV format, with lines of the form
+"userID,itemID,prefValue" (e.g. "39505,290002,3.5"):</p>
+<div class="codehilite"><pre><span class="n">DataModel</span> <span class="n">model</span> <span class="p">=</span> <span class="n">new</span> <span class="n">FileDataModel</span><span class="p">(</span><span class="n">new</span> <span class="n">File</span><span class="p">(</span>"<span class="n">data</span><span class="p">.</span><span class="n">txt</span>"<span class="p">));</span>
+</pre></div>
+
+
+<p>We'll use the <strong>PearsonCorrelationSimilarity</strong> implementation of <strong>UserSimilarity</strong>
+as our user correlation algorithm, and add an optional preference inference
+algorithm:</p>
+<div class="codehilite"><pre><span class="n">UserSimilarity</span> <span class="n">userSimilarity</span> <span class="p">=</span> <span class="n">new</span> <span class="n">PearsonCorrelationSimilarity</span><span class="p">(</span><span class="n">model</span><span class="p">);</span>
+</pre></div>
+
+
+<p>Now we create a <strong>UserNeighborhood</strong> algorithm. Here we use nearest-3:</p>
+<div class="codehilite"><pre><span class="n">UserNeighborhood</span> <span class="n">neighborhood</span> <span class="p">=</span>
+ <span class="n">new</span> <span class="n">NearestNUserNeighborhood</span><span class="p">(</span>3<span class="p">,</span> <span class="n">userSimilarity</span><span class="p">,</span> <span class="n">model</span><span class="p">);{</span><span class="n">code</span><span class="p">}</span>
+</pre></div>
+
+
+<p>Now we can create our <strong>Recommender</strong>, and add a caching decorator:</p>
+<div class="codehilite"><pre><span class="n">Recommender</span> <span class="n">recommender</span> <span class="p">=</span>
+ <span class="n">new</span> <span class="n">GenericUserBasedRecommender</span><span class="p">(</span><span class="n">model</span><span class="p">,</span> <span class="n">neighborhood</span><span class="p">,</span> <span class="n">userSimilarity</span><span class="p">);</span>
+<span class="n">Recommender</span> <span class="n">cachingRecommender</span> <span class="p">=</span> <span class="n">new</span> <span class="n">CachingRecommender</span><span class="p">(</span><span class="n">recommender</span><span class="p">);</span>
+</pre></div>
+
+
+<p>Now we can get 10 recommendations for user ID "1234" — done!</p>
+<div class="codehilite"><pre><span class="n">List</span><span class="o"><</span><span class="n">RecommendedItem</span><span class="o">></span> <span class="n">recommendations</span> <span class="p">=</span>
+ <span class="n">cachingRecommender</span><span class="p">.</span><span class="n">recommend</span><span class="p">(</span>1234<span class="p">,</span> 10<span class="p">);</span>
+</pre></div>
+
+
+<h2 id="item-based-recommender">Item-based Recommender</h2>
+<p>We could have created an item-based recommender instead. Item-based
+recommenders base recommendation not on user similarity, but on item
+similarity. In theory these are about the same approach to the problem,
+just from different angles. However the similarity of two items is
+relatively fixed, more so than the similarity of two users. So, item-based
+recommenders can use pre-computed similarity values in the computations,
+which make them much faster. For large data sets, item-based recommenders
+are more appropriate.</p>
+<p>Let's start over, again with a <strong>FileDataModel</strong> to start:</p>
+<div class="codehilite"><pre><span class="n">DataModel</span> <span class="n">model</span> <span class="p">=</span> <span class="n">new</span> <span class="n">FileDataModel</span><span class="p">(</span><span class="n">new</span> <span class="n">File</span><span class="p">(</span>"<span class="n">data</span><span class="p">.</span><span class="n">txt</span>"<span class="p">));</span>
+</pre></div>
+
+
+<p>We'll also need an <strong>ItemSimilarity</strong>. We could use
+<strong>PearsonCorrelationSimilarity</strong>, which computes item similarity in realtime,
+but, this is generally too slow to be useful. Instead, in a real
+application, you would feed a list of pre-computed correlations to a
+<strong>GenericItemSimilarity</strong>: </p>
+<div class="codehilite"><pre><span class="c1">// Construct the list of pre-computed correlations</span>
+<span class="n">Collection</span><span class="o"><</span><span class="n">GenericItemSimilarity</span><span class="p">.</span><span class="n">ItemItemSimilarity</span><span class="o">></span> <span class="n">correlations</span> <span class="o">=</span>
+ <span class="p">...;</span>
+<span class="n">ItemSimilarity</span> <span class="n">itemSimilarity</span> <span class="o">=</span>
+ <span class="k">new</span> <span class="n">GenericItemSimilarity</span><span class="p">(</span><span class="n">correlations</span><span class="p">);</span>
+</pre></div>
+
+
+<p>Then we can finish as before to produce recommendations:</p>
+<div class="codehilite"><pre><span class="n">Recommender</span> <span class="n">recommender</span> <span class="p">=</span>
+ <span class="n">new</span> <span class="n">GenericItemBasedRecommender</span><span class="p">(</span><span class="n">model</span><span class="p">,</span> <span class="n">itemSimilarity</span><span class="p">);</span>
+<span class="n">Recommender</span> <span class="n">cachingRecommender</span> <span class="p">=</span> <span class="n">new</span> <span class="n">CachingRecommender</span><span class="p">(</span><span class="n">recommender</span><span class="p">);</span>
+<span class="p">...</span>
+<span class="n">List</span><span class="o"><</span><span class="n">RecommendedItem</span><span class="o">></span> <span class="n">recommendations</span> <span class="p">=</span>
+ <span class="n">cachingRecommender</span><span class="p">.</span><span class="n">recommend</span><span class="p">(</span>1234<span class="p">,</span> 10<span class="p">);</span>
+</pre></div>
+
+
+<p><a name="RecommenderDocumentation-Integrationwithyourapplication"></a></p>
+<h2 id="integration-with-your-application">Integration with your application</h2>
+<p>You can create a Recommender, as shown above, wherever you like in your
+Java application, and use it. This includes simple Java applications or GUI
+applications, server applications, and J2EE web applications.</p>
+<p><a name="RecommenderDocumentation-Performance"></a></p>
+<h2 id="performance">Performance</h2>
+<p><a name="RecommenderDocumentation-RuntimePerformance"></a></p>
+<h3 id="runtime-performance">Runtime Performance</h3>
+<p>The more data you give, the better. Though Mahout is designed for
+performance, you will undoubtedly run into performance issues at some
+point. For best results, consider using the following command-line flags to
+your JVM:</p>
+<ul>
+<li>-server: Enables the server VM, which is generally appropriate for
+long-running, computation-intensive applications.</li>
+<li>-Xms1024m -Xmx1024m: Make the heap as big as possible -- a gigabyte
+doesn't hurt when dealing with tens millions of preferences. Mahout
+recommenders will generally use as much memory as you give it for caching,
+which helps performance. Set the initial and max size to the same value to
+avoid wasting time growing the heap, and to avoid having the JVM run minor
+collections to avoid growing the heap, which will clear cached values.</li>
+<li>-da -dsa: Disable all assertions.</li>
+<li>-XX:NewRatio=9: Increase heap allocated to 'old' objects, which is most
+of them in this framework</li>
+<li>-XX:+UseParallelGC -XX:+UseParallelOldGC (multi-processor machines only):
+Use a GC algorithm designed to take advantage of multiple processors, and
+designed for throughput. This is a default in J2SE 5.0.</li>
+<li>-XX:-DisableExplicitGC: Disable calls to System.gc(). These calls can
+only hurt in the presence of modern GC algorithms; they may force Mahout to
+remove cached data needlessly. This flag isn't needed if you're sure your
+code and third-party code you use doesn't call this method.</li>
+</ul>
+<p>Also consider the following tips:</p>
+<ul>
+<li>Use <strong>CachingRecommender</strong> on top of your custom <strong>Recommender</strong> implementation.</li>
+<li>When using <strong>JDBCDataModel</strong>, make sure you wrap it with the <strong>ReloadFromJDBCDataModel</strong> to load data into memory!. </li>
+</ul>
+<p><a name="RecommenderDocumentation-AlgorithmPerformance:WhichOneIsBest?"></a></p>
+<h3 id="algorithm-performance-which-one-is-best">Algorithm Performance: Which One Is Best?</h3>
+<p>There is no right answer; it depends on your data, your application,
+environment, and performance needs. Mahout provides the building blocks
+from which you can construct the best Recommender for your application. The
+links below provide research on this topic. You will probably need a bit of
+trial-and-error to find a setup that works best. The code sample above
+provides a good starting point.</p>
+<p>Fortunately, Mahout provides a way to evaluate the accuracy of your
+Recommender on your own data, in org.apache.mahout.cf.taste.eval</p>
+<div class="codehilite"><pre><span class="n">DataModel</span> <span class="n">myModel</span> <span class="p">=</span> <span class="p">...;</span>
+<span class="n">RecommenderBuilder</span> <span class="n">builder</span> <span class="p">=</span> <span class="n">new</span> <span class="n">RecommenderBuilder</span><span class="p">()</span> <span class="p">{</span>
+ <span class="n">public</span> <span class="n">Recommender</span> <span class="n">buildRecommender</span><span class="p">(</span><span class="n">DataModel</span> <span class="n">model</span><span class="p">)</span> <span class="p">{</span>
+ <span class="o">//</span> <span class="n">build</span> <span class="n">and</span> <span class="k">return</span> <span class="n">the</span> <span class="n">Recommender</span> <span class="n">to</span> <span class="n">evaluate</span> <span class="n">here</span>
+ <span class="p">}</span>
+<span class="p">};</span>
+<span class="n">RecommenderEvaluator</span> <span class="n">evaluator</span> <span class="p">=</span>
+ <span class="n">new</span> <span class="n">AverageAbsoluteDifferenceRecommenderEvaluator</span><span class="p">();</span>
+<span class="n">double</span> <span class="n">evaluation</span> <span class="p">=</span> <span class="n">evaluator</span><span class="p">.</span><span class="n">evaluate</span><span class="p">(</span><span class="n">builder</span><span class="p">,</span> <span class="n">myModel</span><span class="p">,</span> 0<span class="p">.</span>9<span class="p">,</span> 1<span class="p">.</span>0<span class="p">);</span>
+</pre></div>
+
+
+<p>For "boolean" data model situations, where there are no notions of
+preference value, the above evaluation based on estimated preference does
+not make sense. In this case, try a <em>RecommenderIRStatsEvaluator</em>, which presents
+traditional information retrieval figures like precision and recall, which
+are more meaningful.</p>
+<p><a name="RecommenderDocumentation-UsefulLinks"></a></p>
+<h2 id="useful-links">Useful Links</h2>
+<p>Here's a handful of research papers that I've read and found particularly
+useful:</p>
+<p>J.S. Breese, D. Heckerman and C. Kadie, "<a href="http://research.microsoft.com/research/pubs/view.aspx?tr_id=166">Empirical Analysis of Predictive Algorithms for Collaborative Filtering</a>
+," in Proceedings of the Fourteenth Conference on Uncertainity in
+Artificial Intelligence (UAI 1998), 1998.</p>
+<p>B. Sarwar, G. Karypis, J. Konstan and J. Riedl, "<a href="http://www10.org/cdrom/papers/519/">Item-based collaborative filtering recommendation algorithms</a>
+" in Proceedings of the Tenth International Conference on the World Wide
+Web (WWW 10), pp. 285-295, 2001.</p>
+<p>P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom and J. Riedl, "<a href="http://doi.acm.org/10.1145/192844.192905">GroupLens: an open architecture for collaborative filtering of netnews</a>
+" in Proceedings of the 1994 ACM conference on Computer Supported
+Cooperative Work (CSCW 1994), pp. 175-186, 1994.</p>
+<p>J.L. Herlocker, J.A. Konstan, A. Borchers and J. Riedl, "<a href="http://www.grouplens.org/papers/pdf/algs.pdf">An algorithmic framework for performing collaborative filtering</a>
+" in Proceedings of the 22nd annual international ACM SIGIR Conference on
+Research and Development in Information Retrieval (SIGIR 99), pp. 230-237,
+1999.</p>
+ </div>
+ </div>
+</div>
+ <footer class="footer" align="center">
+ <div class="container">
+ <p>
+ Copyright © 2014 The Apache Software Foundation, Licensed under
+ the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.
+ <br />
+ Apache and the Apache feather logos are trademarks of The Apache Software Foundation.
+ </p>
+ </div>
+ </footer>
+
+ <script src="/js/jquery-1.9.1.min.js"></script>
+ <script src="/js/bootstrap.min.js"></script>
+ <script>
+ (function() {
+ var cx = '012254517474945470291:vhsfv7eokdc';
+ var gcse = document.createElement('script');
+ gcse.type = 'text/javascript';
+ gcse.async = true;
+ gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +
+ '//www.google.com/cse/cse.js?cx=' + cx;
+ var s = document.getElementsByTagName('script')[0];
+ s.parentNode.insertBefore(gcse, s);
+ })();
+ </script>
+</body>
+</html>
Added: websites/staging/mahout/trunk/content/users/mapreduce/recommender/recommender-first-timer-faq.html
==============================================================================
--- websites/staging/mahout/trunk/content/users/mapreduce/recommender/recommender-first-timer-faq.html (added)
+++ websites/staging/mahout/trunk/content/users/mapreduce/recommender/recommender-first-timer-faq.html Thu Mar 19 21:21:45 2015
@@ -0,0 +1,314 @@
+<!DOCTYPE html>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+ <title>Apache Mahout: Scalable machine learning and data mining</title>
+ <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+ <meta name="Distribution" content="Global">
+ <meta name="Robots" content="index,follow">
+ <meta name="keywords" content="apache, apache hadoop, apache lucene,
+ business data mining, cluster analysis,
+ collaborative filtering, data extraction, data filtering, data framework, data integration,
+ data matching, data mining, data mining algorithms, data mining analysis, data mining data,
+ data mining introduction, data mining software,
+ data mining techniques, data representation, data set, datamining,
+ feature extraction, fuzzy k means, genetic algorithm, hadoop,
+ hierarchical clustering, high dimensional, introduction to data mining, kmeans,
+ knowledge discovery, learning approach, learning approaches, learning methods,
+ learning techniques, lucene, machine learning, machine translation, mahout apache,
+ mahout taste, map reduce hadoop, mining data, mining methods, naive bayes,
+ natural language processing,
+ supervised, text mining, time series data, unsupervised, web data mining">
+ <link rel="shortcut icon" type="image/x-icon" href="http://mahout.apache.org/images/favicon.ico">
+ <script type="text/javascript" src="/js/prototype.js"></script>
+ <script type="text/javascript" src="/js/effects.js"></script>
+ <script type="text/javascript" src="/js/search.js"></script>
+ <script type="text/javascript" src="/js/slides.js"></script>
+
+ <link href="/css/bootstrap.min.css" rel="stylesheet" media="screen">
+ <link href="/css/bootstrap-responsive.css" rel="stylesheet">
+ <link rel="stylesheet" href="/css/global.css" type="text/css">
+
+ <!-- mathJax stuff -- use `\(...\)` for inline style math in markdown -->
+ <script type="text/x-mathjax-config">
+ MathJax.Hub.Config({
+ tex2jax: {
+ skipTags: ['script', 'noscript', 'style', 'textarea', 'pre']
+ }
+ });
+ MathJax.Hub.Queue(function() {
+ var all = MathJax.Hub.getAllJax(), i;
+ for(i = 0; i < all.length; i += 1) {
+ all[i].SourceElement().parentNode.className += ' has-jax';
+ }
+ });
+ </script>
+ <script type="text/javascript">
+ var mathjax = document.createElement('script');
+ mathjax.type = 'text/javascript';
+ mathjax.async = true;
+
+ mathjax.src = ('https:' == document.location.protocol) ?
+ 'https://c328740.ssl.cf1.rackcdn.com/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' :
+ 'http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML';
+
+ var s = document.getElementsByTagName('script')[0];
+ s.parentNode.insertBefore(mathjax, s);
+ </script>
+</head>
+
+<body id="home" data-twttr-rendered="true">
+ <div id="wrap">
+ <div id="header">
+ <div id="logo"><a href="/overview.html"></a></div>
+ <div id="search">
+ <form id="search-form" action="http://www.google.com/search" method="get" class="navbar-search pull-right">
+ <input value="http://mahout.apache.org" name="sitesearch" type="hidden">
+ <input class="search-query" name="q" id="query" type="text">
+ <input id="submission" type="image" src="/images/mahout-lupe.png" alt="Search" />
+ </form>
+ </div>
+
+ <div class="navbar navbar-inverse" style="position:absolute;top:133px;padding-right:0px;padding-left:0px;">
+ <div class="navbar-inner" style="border: none; background: #999; border: none; border-radius: 0px;">
+ <div class="container">
+ <button type="button" class="btn btn-navbar" data-toggle="collapse" data-target=".nav-collapse">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <!-- <a class="brand" href="#">Apache Community Development Project</a> -->
+ <div class="nav-collapse collapse">
+ <ul class="nav">
+ <li><a href="/">Home</a></li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">General<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/general/downloads.html">Downloads</a>
+ <li><a href="/general/who-we-are.html">Who we are</a>
+ <li><a href="/general/mailing-lists,-irc-and-archives.html">Mailing Lists</a>
+ <li><a href="/general/release-notes.html">Release Notes</a>
+ <li><a href="/general/books-tutorials-and-talks.html">Books, Tutorials, Talks</a></li>
+ <li><a href="/general/powered-by-mahout.html">Powered By Mahout</a>
+ <li><a href="/general/professional-support.html">Professional Support</a>
+ <li class="divider"></li>
+ <li class="nav-header">Resources</li>
+ <li><a href="/general/reference-reading.html">Reference Reading</a>
+ <li><a href="/general/faq.html">FAQ</a>
+ <li class="divider"></li>
+ <li class="nav-header">Legal</li>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ <li><a href="/general/privacy-policy.html">Privacy Policy</a>
+ </ul>
+ </li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Developers<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/developers/developer-resources.html">Developer resources</a></li>
+ <li><a href="/developers/version-control.html">Version control</a></li>
+ <li><a href="/developers/buildingmahout.html">Build from source</a></li>
+ <li><a href="/developers/issue-tracker.html">Issue tracker</a></li>
+ <li><a href="https://builds.apache.org/job/Mahout-Quality/" target="_blank">Code quality reports</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Contributions</li>
+ <li><a href="/developers/how-to-contribute.html">How to contribute</a></li>
+ <li><a href="/developers/how-to-become-a-committer.html">How to become a committer</a></li>
+ <li><a href="/developers/gsoc.html">GSoC</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">For committers</li>
+ <li><a href="/developers/how-to-update-the-website.html">How to update the website</a></li>
+ <li><a href="/developers/patch-check-list.html">Patch check list</a></li>
+ <li><a href="/developers/github.html">Handling Github PRs</a></li>
+ <li><a href="/developers/how-to-release.html">How to release</a></li>
+ <li><a href="/developers/thirdparty-dependencies.html">Third party dependencies</a></li>
+ </ul>
+ </li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Basics<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/basics/algorithms.html">List of algorithms</a>
+ <li><a href="/users/basics/quickstart.html">Quickstart</a>
+ <li class="divider"></li>
+ <li class="nav-header">Working with text</li>
+ <li><a href="/users/basics/creating-vectors-from-text.html">Creating vectors from text</a>
+ <li><a href="/users/basics/collocations.html">Collocations</a>
+ <li class="divider"></li>
+ <li class="nav-header">Dimensionality reduction</li>
+ <li><a href="/users/dim-reduction/dimensional-reduction.html">Singular Value Decomposition</a></li>
+ <li><a href="/users/dim-reduction/ssvd.html">Stochastic SVD</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Topic Models</li>
+ <li><a href="/users/clustering/latent-dirichlet-allocation.html">Latent Dirichlet Allocation</a></li>
+ </ul>
+ </li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Spark<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/sparkbindings/home.html">Scala & Spark Bindings Overview</a></li>
+ <li><a href="/users/sparkbindings/play-with-shell.html">Playing with Mahout's Spark Shell</a></li>
+ <li class="divider"></li>
+ <li><a href="/users/sparkbindings/faq.html">FAQ</a></li>
+ </ul>
+ </li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Classification<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/mapreduce/classification/bayesian.html">Naive Bayes</a></li>
+ <li><a href="/users/mapreduce/classification/hidden-markov-models.html">Hidden Markov Models</a></li>
+ <li><a href="/users/mapreduce/classification/logistic-regression.html">Logistic Regression</a></li>
+ <li><a href="/users/mapreduce/classification/partial-implementation.html">Random Forest</a></li>
+
+ <li class="divider"></li>
+ <li class="nav-header">Examples</li>
+ <li><a href="/users/mapreduce/classification/breiman-example.html">Breiman example</a></li>
+ <li><a href="/users/mapreduce/classification/twenty-newsgroups.html">20 newsgroups example</a></li>
+ </ul></li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Clustering<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/mapreduce/clustering/k-means-clustering.html">k-Means</a></li>
+ <li><a href="/users/mapreduce/clustering/canopy-clustering.html">Canopy</a></li>
+ <li><a href="/users/mapreduce/clustering/fuzzy-k-means.html">Fuzzy k-Means</a></li>
+ <li><a href="/users/mapreduce/clustering/streaming-k-means.html">Streaming KMeans</a></li>
+ <li><a href="/users/mapreduce/clustering/spectral-clustering.html">Spectral Clustering</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Commandline usage</li>
+ <li><a href="/users/mapreduce/clustering/k-means-commandline.html">Options for k-Means</a></li>
+ <li><a href="/users/mapreduce/clustering/canopy-commandline.html">Options for Canopy</a></li>
+ <li><a href="/users/mapreduce/clustering/fuzzy-k-means-commandline.html">Options for Fuzzy k-Means</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Examples</li>
+ <li><a href="/users/mapreduce/clustering/clustering-of-synthetic-control-data.html">Synthetic data</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Post processing</li>
+ <li><a href="/users/mapreduce/clustering/cluster-dumper.html">Cluster Dumper tool</a></li>
+ <li><a href="/users/mapreduce/clustering/visualizing-sample-clusters.html">Cluster visualisation</a></li>
+ </ul></li>
+ <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Recommendations<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+ <li><a href="/users/mapreduce/recommender/quickstart.html">Quickstart</a></li>
+ <li><a href="/users/mapreduce/recommender/recommender-first-timer-faq.html">First Timer FAQ</a></li>
+ <li><a href="/users/mapreduce/recommender/userbased-5-minutes.html">A user-based recommender <br/>in 5 minutes</a></li>
+ <li><a href="/users/mapreduce/recommender/matrix-factorization.html">Matrix factorization-based<br/> recommenders</a></li>
+ <li><a href="/users/mapreduce/recommender/recommender-documentation.html">Overview</a></li>
+ <li class="divider"></li>
+ <li class="nav-header">Hadoop</li>
+ <li><a href="/users/mapreduce/recommender/intro-itembased-hadoop.html">Intro to item-based recommendations<br/> with Hadoop</a></li>
+ <li><a href="/users/mapreduce/recommender/intro-als-hadoop.html">Intro to ALS recommendations<br/> with Hadoop</a></li>
+ <li class="nav-header">Spark</li>
+ <li><a href="/users/mapreduce/recommender/intro-cooccurrence-spark.html">Intro to cooccurrence-based<br/> recommendations with Spark</a></li>
+ </ul>
+ </li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ </div>
+ </div>
+ </div>
+
+</div>
+
+ <div id="sidebar">
+ <div id="sidebar-wrap">
+ <h2>Twitter</h2>
+ <ul class="sidemenu">
+ <li>
+<a class="twitter-timeline" href="https://twitter.com/ApacheMahout" data-widget-id="422861673444028416">Tweets by @ApacheMahout</a>
+<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+"://platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
+</li>
+ </ul>
+ <h2>Apache Software Foundation</h2>
+ <ul class="sidemenu">
+ <li><a href="http://www.apache.org/foundation/how-it-works.html">How the ASF works</a></li>
+ <li><a href="http://www.apache.org/foundation/getinvolved.html">Get Involved</a></li>
+ <li><a href="http://www.apache.org/dev/">Developer Resources</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ </ul>
+ <h2>Related Projects</h2>
+ <ul class="sidemenu">
+ <li><a href="http://lucene.apache.org/">Lucene</a></li>
+ <li><a href="http://hadoop.apache.org/">Hadoop</a></li>
+ </ul>
+ </div>
+</div>
+
+ <div id="content-wrap" class="clearfix">
+ <div id="main">
+ <h1 id="recommender-first-timer-dos-and-donts">Recommender First Timer Dos and Don'ts</h1>
+<p>Many people with an interest in recommenders arrive at Mahout since they're
+building a first recommender system. Some starting questions have been
+asked enough times to warrant a FAQ collecting advice and rules-of-thumb to
+newcomers.</p>
+<p>For the interested, these topics are treated in detail in the book <a href="http://manning.com/owen/">Mahout in Action</a>.</p>
+<p>Don't start with a distributed, Hadoop-based recommender; take on that
+complexity only if necessary. Start with non-distributed recommenders. It
+is simpler, has fewer requirements, and is more flexible. </p>
+<p>As a crude rule of thumb, a system with up to 100M user-item associations
+(ratings, preferences) should "fit" onto one modern server machine with 4GB
+of heap available and run acceptably as a real-time recommender. The system
+is invariably memory-bound since keeping data in memory is essential to
+performance.</p>
+<p>Beyond this point it gets expensive to deploy a machine with enough RAM,
+so, designing for a distributed makes sense when nearing this scale.
+However most applications don't "really" have 100M associations to process.
+Data can be sampled; noisy and old data can often be aggressively pruned
+without significant impact on the result.</p>
+<p>The next question is whether or not your system has preference values, or
+ratings. Do users and items merely have an association or not, such as the
+existence or lack of a click? or is behavior translated into some scalar
+value representing the user's degree of preference for the item.</p>
+<p>If you have ratings, then a good place to start is a
+GenericItemBasedRecommender, plus a PearsonCorrelationSimilarity similarity
+metric. If you don't have ratings, then a good place to start is
+GenericBooleanPrefItemBasedRecommender and LogLikelihoodSimilarity.</p>
+<p>If you want to do content-based item-item similarity, you need to implement
+your own ItemSimilarity.</p>
+<p>If your data can be simply exported to a CSV file, use FileDataModel and
+push new files periodically.
+If your data is in a database, use MySQLJDBCDataModel (or its "BooleanPref"
+counterpart if appropriate, or its PostgreSQL counterpart, etc.) and put on
+top a ReloadFromJDBCDataModel.</p>
+<p>This should give a reasonable starter system which responds fast. The
+nature of the system is that new data comes in from the file or database
+only periodically -- perhaps on the order of minutes. </p>
+ </div>
+ </div>
+</div>
+ <footer class="footer" align="center">
+ <div class="container">
+ <p>
+ Copyright © 2014 The Apache Software Foundation, Licensed under
+ the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.
+ <br />
+ Apache and the Apache feather logos are trademarks of The Apache Software Foundation.
+ </p>
+ </div>
+ </footer>
+
+ <script src="/js/jquery-1.9.1.min.js"></script>
+ <script src="/js/bootstrap.min.js"></script>
+ <script>
+ (function() {
+ var cx = '012254517474945470291:vhsfv7eokdc';
+ var gcse = document.createElement('script');
+ gcse.type = 'text/javascript';
+ gcse.async = true;
+ gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +
+ '//www.google.com/cse/cse.js?cx=' + cx;
+ var s = document.getElementsByTagName('script')[0];
+ s.parentNode.insertBefore(gcse, s);
+ })();
+ </script>
+</body>
+</html>