You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@madlib.apache.org by ri...@apache.org on 2016/03/30 02:58:48 UTC
[08/51] [partial] incubator-madlib-site git commit: Add all files
from old site (madlib.net)
http://git-wip-us.apache.org/repos/asf/incubator-madlib-site/blob/7258f51a/docs/latest/group__grp__path.html
----------------------------------------------------------------------
diff --git a/docs/latest/group__grp__path.html b/docs/latest/group__grp__path.html
new file mode 100644
index 0000000..dee1dbb
--- /dev/null
+++ b/docs/latest/group__grp__path.html
@@ -0,0 +1,272 @@
+<!-- HTML header for doxygen 1.8.4-->
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
+<meta http-equiv="X-UA-Compatible" content="IE=9"/>
+<meta name="generator" content="Doxygen 1.8.10"/>
+<meta name="keywords" content="madlib,postgres,greenplum,machine learning,data mining,deep learning,ensemble methods,data science,market basket analysis,affinity analysis,pca,lda,regression,elastic net,huber white,proportional hazards,k-means,latent dirichlet allocation,bayes,support vector machines,svm"/>
+<title>MADlib: Grp_path</title>
+<link href="tabs.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="jquery.js"></script>
+<script type="text/javascript" src="dynsections.js"></script>
+<link href="navtree.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="resize.js"></script>
+<script type="text/javascript" src="navtreedata.js"></script>
+<script type="text/javascript" src="navtree.js"></script>
+<script type="text/javascript">
+ $(document).ready(initResizable);
+ $(window).load(resizeHeight);
+</script>
+<link href="search/search.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="search/searchdata.js"></script>
+<script type="text/javascript" src="search/search.js"></script>
+<script type="text/javascript">
+ $(document).ready(function() { init_search(); });
+</script>
+<script type="text/x-mathjax-config">
+ MathJax.Hub.Config({
+ extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
+ jax: ["input/TeX","output/HTML-CSS"],
+});
+</script><script src="../mathjax/MathJax.js"></script>
+<!-- hack in the navigation tree -->
+<script type="text/javascript" src="navtree_hack.js"></script>
+<link href="doxygen.css" rel="stylesheet" type="text/css" />
+<!-- google analytics -->
+<script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+ ga('create', 'UA-45382226-1', 'auto');
+ ga('send', 'pageview');
+</script>
+</head>
+<body>
+<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
+<div id="titlearea">
+<table cellspacing="0" cellpadding="0">
+ <tbody>
+ <tr style="height: 56px;">
+ <td id="projectlogo"><a href="http://madlib.incubator.apache.org"><img alt="Logo" src="madlib.png" height="50" style="padding-left:0.5em;" border="0"/ ></a></td>
+ <td style="padding-left: 0.5em;">
+ <div id="projectname">
+ <span id="projectnumber">1.8</span>
+ </div>
+ <div id="projectbrief">User Documentation for MADlib</div>
+ </td>
+ <td> <div id="MSearchBox" class="MSearchBoxInactive">
+ <span class="left">
+ <img id="MSearchSelect" src="search/mag_sel.png"
+ onmouseover="return searchBox.OnSearchSelectShow()"
+ onmouseout="return searchBox.OnSearchSelectHide()"
+ alt=""/>
+ <input type="text" id="MSearchField" value="Search" accesskey="S"
+ onfocus="searchBox.OnSearchFieldFocus(true)"
+ onblur="searchBox.OnSearchFieldFocus(false)"
+ onkeyup="searchBox.OnSearchFieldChange(event)"/>
+ </span><span class="right">
+ <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
+ </span>
+ </div>
+</td>
+ </tr>
+ </tbody>
+</table>
+</div>
+<!-- end header part -->
+<!-- Generated by Doxygen 1.8.10 -->
+<script type="text/javascript">
+var searchBox = new SearchBox("searchBox", "search",false,'Search');
+</script>
+</div><!-- top -->
+<div id="side-nav" class="ui-resizable side-nav-resizable">
+ <div id="nav-tree">
+ <div id="nav-tree-contents">
+ <div id="nav-sync" class="sync"></div>
+ </div>
+ </div>
+ <div id="splitbar" style="-moz-user-select:none;"
+ class="ui-resizable-handle">
+ </div>
+</div>
+<script type="text/javascript">
+$(document).ready(function(){initNavTree('group__grp__path.html','');});
+</script>
+<div id="doc-content">
+<!-- window showing the filter options -->
+<div id="MSearchSelectWindow"
+ onmouseover="return searchBox.OnSearchSelectShow()"
+ onmouseout="return searchBox.OnSearchSelectHide()"
+ onkeydown="return searchBox.OnSearchSelectKey(event)">
+</div>
+
+<!-- iframe showing the search results (closed by default) -->
+<div id="MSearchResultsWindow">
+<iframe src="javascript:void(0)" frameborder="0"
+ name="MSearchResults" id="MSearchResults">
+</iframe>
+</div>
+
+<div class="header">
+ <div class="headertitle">
+<div class="title">Grp_path</div> </div>
+</div><!--header-->
+<div class="contents">
+<div class="toc"><b>Contents</b> </p><ul>
+<li>
+<a href="#syntax">Function Syntax</a> </li>
+<li>
+<a href="#examples">Examples</a> </li>
+</ul>
+</div><p>The goal of the MADlib path function is to perform regular pattern matching over a sequence of rows, and to extract useful information about the matches. The useful information could be a simple count of matches or something more involved like aggregation.</p>
+<p>There are many use cases where path functions are typically used:</p><ul>
+<li>Web analytics (clickstream)</li>
+<li>Marketing revenue attribution</li>
+<li>Telephone calling patterns</li>
+<li>Stock market trading sequences</li>
+<li>Predictive maintenance</li>
+<li>Genomics sequencing</li>
+</ul>
+<p><a class="anchor" id="syntax"></a></p><dl class="section user"><dt>Function Syntax</dt><dd><pre class="syntax">
+path(
+ source_table,
+ output_table,
+ partition_expr,
+ order_expr,
+ pattern,
+ symbol,
+ aggregate_func,
+ persist_rows
+)
+</pre></dd></dl>
+<p><b>Arguments</b> </p><dl class="arglist">
+<dt>source_table </dt>
+<dd><p class="startdd">VARCHAR. Name of the source table, containing data for path analysis.</p>
+<p class="enddd"></p>
+</dd>
+<dt>output_table </dt>
+<dd><p class="startdd">VARCHAR. Name of the result table. This table contains columns predicated by the <em>result</em> argument (shown below).</p>
+<p class="enddd"></p>
+</dd>
+<dt>partition_expr </dt>
+<dd><p class="startdd">VARCHAR. The 'partition_expr' can be a list of columns or expressions (separated by comma) to divide all rows into groups, or partitions, that share the same values of the partition expression(s). For each row, the matching is applied across the rows that fall into the same partition. This can be NULL or '' to indicate the matching is to be applied on the whole table.</p>
+<p class="enddd"></p>
+</dd>
+<dt>order_expr </dt>
+<dd><p class="startdd">VARCHAR. This expression controls the order in which rows are processed or matched in a partition. </p>
+<p class="enddd"></p>
+</dd>
+<dt>pattern </dt>
+<dd><p class="startdd">VARCHAR. The PATTERN clause defines the pattern that path searches for. You express the PATTERN using symbols and operators. Symbols defined using more than 1 characters need to be wrapped in parentheses '()'. The following pattern-matching metacharacters are supported: </p><ul>
+<li>
+| denotes alternation (either of two alternatives). </li>
+<li>
+? denotes repetition of the previous item zero or one time. </li>
+<li>
+* denotes repetition of the previous item zero or more times. </li>
+<li>
++ denotes repetition of the previous item one or more times. </li>
+<li>
+{m} denotes repetition of the previous item exactly m times. </li>
+<li>
+{m,} denotes repetition of the previous item m or more times. </li>
+<li>
+{m,n} denotes repetition of the previous item at least m and not more than n times. </li>
+<li>
+Parentheses () can be used to group items into a single logical item. </li>
+</ul>
+<p class="enddd"></p>
+</dd>
+<dt>symbol </dt>
+<dd><p class="startdd">VARCHAR. A symbol represents a row of a particular type that you’re searching for as part of a row sequence. In the SYMBOLS clause, you write a predicate to define the type of row that matches the symbol. </p>
+<p class="enddd"></p>
+</dd>
+<dt>aggregate_func </dt>
+<dd><p class="startdd">VARCHAR. A comma-separated list of window functions and aggregates to be applied on the matched window. </p>
+<p class="enddd"></p>
+</dd>
+<dt>persist_rows </dt>
+<dd><p class="startdd">BOOLEAN. If TRUE the matched rows are persisted in another table. This table is named as <output_table>_tuples (the string "_tuples" is added as suffix to the value of <em>output_table</em>). </p>
+<p class="enddd"></p>
+</dd>
+</dl>
+<p><a class="anchor" id="examples"></a></p><dl class="section user"><dt>Examples</dt><dd></dd></dl>
+<ul>
+<li>Build sample dataset <pre class="example">
+CREATE TABLE data (id integer, sessionid integer, starttime timestamp, topic varchar, portfolio integer);
+COPY data FROM STDIN DELIMITER AS '|';
+1|1|1/01/00 5:00 AM|Real-time Equity pricing|769
+1|2|1/01/00 5:30 AM|Real-time Index pricing|9898
+1|3|1/01/00 6:00 AM|Real-time Index pricing|9898
+1|4|1/01/00 5:00 AM|Calendar|98977
+1|5|1/01/00 10:00 AM|Real-time Equity pricing|769
+1|6|1/01/00 10:30 AM|Real-time Equity pricing|9898
+1|7|1/01/00 12:00 PM|Calendar|1325
+1|8|1/01/00 12:00 PM|Calendar|6777
+1|9|1/01/00 4:05 PM|Real-time Equity pricing|769
+1|10|1/01/00 4:10 PM|Report Dates and Analyst Recommendations|34346
+1|11|1/01/00 4:15 PM|Real-time Index pricing|9898
+1|12|1/01/00 4:20 PM|Snapshot Futures pricing|5568
+1|13|1/01/00 4:21 PM|Real-time Equity pricing|769
+1|14|1/01/00 4:25 PM|Bond security master|789
+1|15|1/01/00 10:00 PM|Bond pricing|55
+1|16|1/01/00 10:00 PM|Bond pricing|55
+1|17|1/01/00 10:00 PM|Bond pricing|55
+1|18|1/01/00 10:00 PM|Bond Yield Analysis|655
+1|19|1/01/00 10:00 PM|Bond pricing|55
+1|20|1/01/00 10:00 PM|Bond pricing|55
+2|1|1/04/00 12:00 AM|Real-time Equity pricing|769
+2|2|1/04/00 12:00 AM|Real-time Equity pricing|234234
+2|3|1/04/00 12:00 AM|Calendar|1325
+2|4|1/04/00 12:00 AM|Calendar|6777
+2|5|1/04/00 12:00 AM|Real-time Equity pricing|769
+2|6|1/04/00 12:00 AM|Report Dates and Analyst Recommendations|34346
+2|7|1/04/00 12:00 AM|Real-time Index pricing|9898
+2|8|1/04/00 12:00 AM|Snapshot Futures pricing|5568
+2|9|1/04/00 12:00 AM|Real-time Equity pricing|769
+2|10|1/04/00 12:00 AM|Bond security master|789
+2|11|1/04/00 12:00 AM|Bond pricing|55
+2|12|1/04/00 12:00 AM|Bond pricing|55
+2|13|1/04/00 12:00 AM|Bond pricing|55
+2|14|1/04/00 12:00 AM|Bond pricing|55
+\.
+ 
+CREATE TABLE trades AS
+SELECT *, starttimestamp::date startdate,
+ CASE WHEN THEN 'before'
+ WHEN THEN 'market'
+ WHEN THEN 'close'
+ WHEN THEN 'after'
+ END tradingperiod
+FROM sessiontable
+</pre></li>
+<li>Compare within each day for the same Topic/Portfolio across every user <pre class="example">
+ SELECT madlib.path(
+ 'trades',
+ 'trades_out',
+ 'startdate, topic, portfolio' -- each day of activity is looked at independently
+ 'starttime' -- order by time
+ 'BEFORE*.MARKET+.CLOSE+.AFTER*' -- at least one event during each of MARKET and CLOSE, but gather up the rest
+ 'BEFORE:=starttimestamp::time >= ''0:00:00'' and starttimestamp::time < ''9:30:00''::time,
+ MARKET:=starttimestamp::time >= ''9:30:00'' and starttimestamp::time < ''16:00:00''::time,
+ CLOSE:= starttimestamp::time <= ''16:00:00'' and starttimestamp::time < ''16:30:00''::time,
+ AFTER:= starttimestamp::time <= ''16:30:00'' and starttimestamp::time < ''24:00:00''::time
+ ',
+ 'first(startdate) as starttime, array_agg(id) as all_users, count(*) as num_matches'
+ )
+</pre> </li>
+</ul>
+</div><!-- contents -->
+</div><!-- doc-content -->
+<!-- start footer part -->
+<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
+ <ul>
+ <li class="footer">Generated on Tue Mar 22 2016 17:55:59 for MADlib by
+ <a href="http://www.doxygen.org/index.html">
+ <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.10 </li>
+ </ul>
+</div>
+</body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-madlib-site/blob/7258f51a/docs/latest/group__grp__pca.html
----------------------------------------------------------------------
diff --git a/docs/latest/group__grp__pca.html b/docs/latest/group__grp__pca.html
new file mode 100644
index 0000000..f4a176c
--- /dev/null
+++ b/docs/latest/group__grp__pca.html
@@ -0,0 +1,143 @@
+<!-- HTML header for doxygen 1.8.4-->
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
+<meta http-equiv="X-UA-Compatible" content="IE=9"/>
+<meta name="generator" content="Doxygen 1.8.10"/>
+<meta name="keywords" content="madlib,postgres,greenplum,machine learning,data mining,deep learning,ensemble methods,data science,market basket analysis,affinity analysis,pca,lda,regression,elastic net,huber white,proportional hazards,k-means,latent dirichlet allocation,bayes,support vector machines,svm"/>
+<title>MADlib: Dimensionality Reduction</title>
+<link href="tabs.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="jquery.js"></script>
+<script type="text/javascript" src="dynsections.js"></script>
+<link href="navtree.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="resize.js"></script>
+<script type="text/javascript" src="navtreedata.js"></script>
+<script type="text/javascript" src="navtree.js"></script>
+<script type="text/javascript">
+ $(document).ready(initResizable);
+ $(window).load(resizeHeight);
+</script>
+<link href="search/search.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="search/searchdata.js"></script>
+<script type="text/javascript" src="search/search.js"></script>
+<script type="text/javascript">
+ $(document).ready(function() { init_search(); });
+</script>
+<script type="text/x-mathjax-config">
+ MathJax.Hub.Config({
+ extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
+ jax: ["input/TeX","output/HTML-CSS"],
+});
+</script><script src="../mathjax/MathJax.js"></script>
+<!-- hack in the navigation tree -->
+<script type="text/javascript" src="navtree_hack.js"></script>
+<link href="doxygen.css" rel="stylesheet" type="text/css" />
+<link href="madlib_extra.css" rel="stylesheet" type="text/css"/>
+<!-- google analytics -->
+<script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+ ga('create', 'UA-45382226-1', 'auto');
+ ga('send', 'pageview');
+</script>
+</head>
+<body>
+<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
+<div id="titlearea">
+<table cellspacing="0" cellpadding="0">
+ <tbody>
+ <tr style="height: 56px;">
+ <td id="projectlogo"><a href="http://madlib.incubator.apache.org"><img alt="Logo" src="madlib.png" height="50" style="padding-left:0.5em;" border="0"/ ></a></td>
+ <td style="padding-left: 0.5em;">
+ <div id="projectname">
+ <span id="projectnumber">1.8</span>
+ </div>
+ <div id="projectbrief">User Documentation for MADlib</div>
+ </td>
+ <td> <div id="MSearchBox" class="MSearchBoxInactive">
+ <span class="left">
+ <img id="MSearchSelect" src="search/mag_sel.png"
+ onmouseover="return searchBox.OnSearchSelectShow()"
+ onmouseout="return searchBox.OnSearchSelectHide()"
+ alt=""/>
+ <input type="text" id="MSearchField" value="Search" accesskey="S"
+ onfocus="searchBox.OnSearchFieldFocus(true)"
+ onblur="searchBox.OnSearchFieldFocus(false)"
+ onkeyup="searchBox.OnSearchFieldChange(event)"/>
+ </span><span class="right">
+ <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
+ </span>
+ </div>
+</td>
+ </tr>
+ </tbody>
+</table>
+</div>
+<!-- end header part -->
+<!-- Generated by Doxygen 1.8.10 -->
+<script type="text/javascript">
+var searchBox = new SearchBox("searchBox", "search",false,'Search');
+</script>
+</div><!-- top -->
+<div id="side-nav" class="ui-resizable side-nav-resizable">
+ <div id="nav-tree">
+ <div id="nav-tree-contents">
+ <div id="nav-sync" class="sync"></div>
+ </div>
+ </div>
+ <div id="splitbar" style="-moz-user-select:none;"
+ class="ui-resizable-handle">
+ </div>
+</div>
+<script type="text/javascript">
+$(document).ready(function(){initNavTree('group__grp__pca.html','');});
+</script>
+<div id="doc-content">
+<!-- window showing the filter options -->
+<div id="MSearchSelectWindow"
+ onmouseover="return searchBox.OnSearchSelectShow()"
+ onmouseout="return searchBox.OnSearchSelectHide()"
+ onkeydown="return searchBox.OnSearchSelectKey(event)">
+</div>
+
+<!-- iframe showing the search results (closed by default) -->
+<div id="MSearchResultsWindow">
+<iframe src="javascript:void(0)" frameborder="0"
+ name="MSearchResults" id="MSearchResults">
+</iframe>
+</div>
+
+<div class="header">
+ <div class="summary">
+<a href="#groups">Modules</a> </div>
+ <div class="headertitle">
+<div class="title">Dimensionality Reduction<div class="ingroups"><a class="el" href="group__grp__utility__functions.html">Utility Functions</a></div></div> </div>
+</div><!--header-->
+<div class="contents">
+<a name="details" id="details"></a><h2 class="groupheader">Detailed Description</h2>
+<p>A collection of methods for dimensionality reduction. </p>
+<table class="memberdecls">
+<tr class="heading"><td colspan="2"><h2 class="groupheader"><a name="groups"></a>
+Modules</h2></td></tr>
+<tr class="memitem:group__grp__pca__train"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="group__grp__pca__train.html">Principal Component Analysis</a></td></tr>
+<tr class="memdesc:group__grp__pca__train"><td class="mdescLeft"> </td><td class="mdescRight">Produces a model that transforms a number of (possibly) correlated variables into a (smaller) number of uncorrelated variables called principal components. <br /></td></tr>
+<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
+<tr class="memitem:group__grp__pca__project"><td class="memItemLeft" align="right" valign="top"> </td><td class="memItemRight" valign="bottom"><a class="el" href="group__grp__pca__project.html">Principal Component Projection</a></td></tr>
+<tr class="memdesc:group__grp__pca__project"><td class="mdescLeft"> </td><td class="mdescRight">Projects a higher dimensional data point to a lower dimensional subspace spanned by principal components learned through the PCA training procedure. <br /></td></tr>
+<tr class="separator:"><td class="memSeparator" colspan="2"> </td></tr>
+</table>
+</div><!-- contents -->
+</div><!-- doc-content -->
+<!-- start footer part -->
+<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
+ <ul>
+ <li class="footer">Generated on Mon Jul 27 2015 20:37:45 for MADlib by
+ <a href="http://www.doxygen.org/index.html">
+ <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.10 </li>
+ </ul>
+</div>
+</body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-madlib-site/blob/7258f51a/docs/latest/group__grp__pca.js
----------------------------------------------------------------------
diff --git a/docs/latest/group__grp__pca.js b/docs/latest/group__grp__pca.js
new file mode 100644
index 0000000..2863cf8
--- /dev/null
+++ b/docs/latest/group__grp__pca.js
@@ -0,0 +1,5 @@
+var group__grp__pca =
+[
+ [ "Principal Component Analysis", "group__grp__pca__train.html", null ],
+ [ "Principal Component Projection", "group__grp__pca__project.html", null ]
+];
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-madlib-site/blob/7258f51a/docs/latest/group__grp__pca__project.html
----------------------------------------------------------------------
diff --git a/docs/latest/group__grp__pca__project.html b/docs/latest/group__grp__pca__project.html
new file mode 100644
index 0000000..a9518e5
--- /dev/null
+++ b/docs/latest/group__grp__pca__project.html
@@ -0,0 +1,298 @@
+<!-- HTML header for doxygen 1.8.4-->
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
+<meta http-equiv="X-UA-Compatible" content="IE=9"/>
+<meta name="generator" content="Doxygen 1.8.10"/>
+<meta name="keywords" content="madlib,postgres,greenplum,machine learning,data mining,deep learning,ensemble methods,data science,market basket analysis,affinity analysis,pca,lda,regression,elastic net,huber white,proportional hazards,k-means,latent dirichlet allocation,bayes,support vector machines,svm"/>
+<title>MADlib: Principal Component Projection</title>
+<link href="tabs.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="jquery.js"></script>
+<script type="text/javascript" src="dynsections.js"></script>
+<link href="navtree.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="resize.js"></script>
+<script type="text/javascript" src="navtreedata.js"></script>
+<script type="text/javascript" src="navtree.js"></script>
+<script type="text/javascript">
+ $(document).ready(initResizable);
+ $(window).load(resizeHeight);
+</script>
+<link href="search/search.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="search/searchdata.js"></script>
+<script type="text/javascript" src="search/search.js"></script>
+<script type="text/javascript">
+ $(document).ready(function() { init_search(); });
+</script>
+<script type="text/x-mathjax-config">
+ MathJax.Hub.Config({
+ extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
+ jax: ["input/TeX","output/HTML-CSS"],
+});
+</script><script src="../mathjax/MathJax.js"></script>
+<!-- hack in the navigation tree -->
+<script type="text/javascript" src="navtree_hack.js"></script>
+<link href="doxygen.css" rel="stylesheet" type="text/css" />
+<link href="madlib_extra.css" rel="stylesheet" type="text/css"/>
+<!-- google analytics -->
+<script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+ ga('create', 'UA-45382226-1', 'auto');
+ ga('send', 'pageview');
+</script>
+</head>
+<body>
+<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
+<div id="titlearea">
+<table cellspacing="0" cellpadding="0">
+ <tbody>
+ <tr style="height: 56px;">
+ <td id="projectlogo"><a href="http://madlib.incubator.apache.org"><img alt="Logo" src="madlib.png" height="50" style="padding-left:0.5em;" border="0"/ ></a></td>
+ <td style="padding-left: 0.5em;">
+ <div id="projectname">
+ <span id="projectnumber">1.8</span>
+ </div>
+ <div id="projectbrief">User Documentation for MADlib</div>
+ </td>
+ <td> <div id="MSearchBox" class="MSearchBoxInactive">
+ <span class="left">
+ <img id="MSearchSelect" src="search/mag_sel.png"
+ onmouseover="return searchBox.OnSearchSelectShow()"
+ onmouseout="return searchBox.OnSearchSelectHide()"
+ alt=""/>
+ <input type="text" id="MSearchField" value="Search" accesskey="S"
+ onfocus="searchBox.OnSearchFieldFocus(true)"
+ onblur="searchBox.OnSearchFieldFocus(false)"
+ onkeyup="searchBox.OnSearchFieldChange(event)"/>
+ </span><span class="right">
+ <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
+ </span>
+ </div>
+</td>
+ </tr>
+ </tbody>
+</table>
+</div>
+<!-- end header part -->
+<!-- Generated by Doxygen 1.8.10 -->
+<script type="text/javascript">
+var searchBox = new SearchBox("searchBox", "search",false,'Search');
+</script>
+</div><!-- top -->
+<div id="side-nav" class="ui-resizable side-nav-resizable">
+ <div id="nav-tree">
+ <div id="nav-tree-contents">
+ <div id="nav-sync" class="sync"></div>
+ </div>
+ </div>
+ <div id="splitbar" style="-moz-user-select:none;"
+ class="ui-resizable-handle">
+ </div>
+</div>
+<script type="text/javascript">
+$(document).ready(function(){initNavTree('group__grp__pca__project.html','');});
+</script>
+<div id="doc-content">
+<!-- window showing the filter options -->
+<div id="MSearchSelectWindow"
+ onmouseover="return searchBox.OnSearchSelectShow()"
+ onmouseout="return searchBox.OnSearchSelectHide()"
+ onkeydown="return searchBox.OnSearchSelectKey(event)">
+</div>
+
+<!-- iframe showing the search results (closed by default) -->
+<div id="MSearchResultsWindow">
+<iframe src="javascript:void(0)" frameborder="0"
+ name="MSearchResults" id="MSearchResults">
+</iframe>
+</div>
+
+<div class="header">
+ <div class="headertitle">
+<div class="title">Principal Component Projection<div class="ingroups"><a class="el" href="group__grp__utility__functions.html">Utility Functions</a> » <a class="el" href="group__grp__pca.html">Dimensionality Reduction</a></div></div> </div>
+</div><!--header-->
+<div class="contents">
+<div class="toc"><b>Contents</b> </p><ul>
+<li class="level1">
+<a href="#project">Projection Function</a> </li>
+<li class="level1">
+<a href="#examples">Examples</a> </li>
+<li class="level1">
+<a href="#notes">Notes</a> </li>
+<li class="level1">
+<a href="#background">Technical Background</a> </li>
+<li class="level1">
+<a href="#related">Related Topics</a> </li>
+</ul>
+</div><p>Principal component projection is a mathematical procedure that projects high dimensional data onto a lower dimensional space. This lower dimensional space is defined by the \( k \) principal components with the highest variance in the training data. More details on the mathematics of PCA can be found in <a class="el" href="pca_8sql__in.html#a31abf88e67a446a4f789764aa2c61e85">pca_train</a> and some details about the principal component projection calculations can be found in the <a class="el" href="group__grp__mlogreg.html#background">Technical Background</a>.</p>
+<p><a class="anchor" id="project"></a></p><dl class="section user"><dt>Projection Function</dt><dd>The projection functions have the following formats: <pre class="syntax">
+madlib.pca_project( source_table,
+ pc_table,
+ out_table,
+ row_id,
+ residual_table,
+ result_summary_table
+ )
+</pre> and <pre class="syntax">
+madlib.pca_sparse_project( source_table,
+ pc_table,
+ out_table,
+ row_id,
+ col_id,
+ val_id,
+ row_dim,
+ col_dim,
+ residual_table,
+ result_summary_table
+ )
+</pre></dd></dl>
+<dl class="section user"><dt>Arguments</dt><dd><dl class="arglist">
+<dt>source_table </dt>
+<dd><p class="startdd">TEXT. Source table name. Identical to <a class="el" href="pca_8sql__in.html#a31abf88e67a446a4f789764aa2c61e85">pca_train</a>, the input data matrix should have \( N \) rows and \( M \) columns, where \( N \) is the number of data points, and \( M \) is the number of features for each data point.</p>
+<p>The input table for <em> pca_project </em> is expected to be in the one of the two standard MADlib dense matrix formats, and the sparse input table for <em> pca_sparse_project </em> should be in the standard MADlib sparse matrix format. These formats are described in the documentation for <a class="el" href="pca_8sql__in.html#a31abf88e67a446a4f789764aa2c61e85">pca_train</a>.</p>
+<p class="enddd"></p>
+</dd>
+<dt>pc_table </dt>
+<dd><p class="startdd">TEXT. Table name for the table containing principal components. </p>
+<p class="enddd"></p>
+</dd>
+<dt>out_table </dt>
+<dd><p class="startdd">TEXT. Name of the table that will contain the low-dimensional representation of the input data.</p>
+<p>The <em>out_table</em> encodes a dense matrix with the projection onto the principal components. The table has the following columns:</p>
+<table class="output">
+<tr>
+<th>row_id </th><td>Row id of the output matrix. </td></tr>
+<tr>
+<th>row_vec </th><td>A vector containing elements in the row of the matrix. </td></tr>
+</table>
+<p></p>
+<p class="enddd"></p>
+</dd>
+<dt>row_id </dt>
+<dd><p class="startdd">TEXT. Column name containing the row IDs in the input source table.</p>
+<p class="enddd"></p>
+</dd>
+<dt>col_id </dt>
+<dd><p class="startdd">TEXT. Name of 'col_id' column in sparse matrix representation (sparse matrices only). </p>
+<p class="enddd"></p>
+</dd>
+<dt>val_id </dt>
+<dd><p class="startdd">TEXT. Name of 'val_id' column in sparse matrix representation (sparse matrices only). </p>
+<p class="enddd"></p>
+</dd>
+<dt>row_dim </dt>
+<dd><p class="startdd">INTEGER. The number of rows in the sparse matrix (sparse matrices only). </p>
+<p class="enddd"></p>
+</dd>
+<dt>col_dim </dt>
+<dd><p class="startdd">INTEGER. The number of columns in the sparse matrix (sparse matrices only). </p>
+<p class="enddd"></p>
+</dd>
+<dt>residual_table (optional) </dt>
+<dd><p class="startdd">TEXT, default: NULL. Name of the optional residual table.</p>
+<p>The <em>residual_table</em> encodes a dense residual matrix. The table has the following columns:</p>
+<table class="output">
+<tr>
+<th>row_id </th><td>Row id of the output matrix. </td></tr>
+<tr>
+<th>row_vec </th><td>A vector containing elements in the row of the residual matrix. </td></tr>
+</table>
+<p class="enddd"></p>
+</dd>
+<dt>result_summary_table (optional) </dt>
+<dd><p class="startdd">TEXT, default: NULL. Name of the optional summary table.</p>
+<p class="enddd">The <em>result_summary_table</em> contains information about the performance time of the PCA projection. The table has the following columns: </p><table class="output">
+<tr>
+<th>exec_time </th><td>Wall clock time (ms) of the function. </td></tr>
+<tr>
+<th>residual_norm </th><td>Absolute error of the residuals. </td></tr>
+<tr>
+<th>relative_residual_norm </th><td>Relative error of the residuals. </td></tr>
+</table>
+</dd>
+</dl>
+</dd></dl>
+<p><a class="anchor" id="examples"></a></p><dl class="section user"><dt>Examples</dt><dd><ol type="1">
+<li>View online help for the PCA projection function. <pre class="example">
+SELECT madlib.pca_project();
+</pre></li>
+<li>Create the sample data. <pre class="example">
+DROP TABLE IF EXISTS mat;
+CREATE TABLE mat (
+ row_id integer,
+ row_vec double precision[]
+);
+COPY mat (row_id, row_vec) FROM stdin;
+1 {1,2,5}
+0 {4,7,5}
+3 {9,2,4}
+2 {7,4,4}
+5 {0,5,5}
+4 {8,5,7}
+\.
+</pre></li>
+<li>Run the PCA function and keep only the top two PCs: <pre class="example">
+DROP TABLE IF EXISTS result_table;
+SELECT pca_train ( 'mat',
+ 'result_table',
+ 'row_id',
+ 2
+ );
+</pre></li>
+<li>Project the original data into a low-dimensional representation. <pre class="example">
+DROP TABLE IF EXISTS residual_table, result_summary_table, out_table;
+SELECT pca_project( 'mat',
+ 'result_table',
+ 'out_table'
+ 'row_id',
+ 'residual_table',
+ 'result_summary_table'
+ );
+</pre></li>
+<li>Check the error in the projection. <pre class="example">
+SELECT * FROM result_summary_table;
+</pre> Result: <pre class="result">
+ exec_time | residual_norm | relative_residual_norm
+---------------+---------------+------------------------
+ 5685.40501595 | 2.19726255664 | 0.099262204234
+</pre></li>
+</ol>
+</dd></dl>
+<p><a class="anchor" id="notes"></a></p><dl class="section user"><dt>Notes</dt><dd><ul>
+<li>This function is intended to operate on the principal component tables generated by <em> pca_train </em> or <em> pca_sparse_train</em>. The MADlib PCA functions generate a table containing the column-means in addition to a table containing the principal components. If this table is not found by the MADlib projection function, it will trigger an error. As long the principal component tables are created with MADlib functions, then the column-means table will be automatically found by the MADlib projection functions.</li>
+<li>Because of the centering step in PCA projection (see "Technical Background"), sparse matrices almost always become dense during the projection process. Thus, this implementation automatically densifies sparse matrix input, and there should be no expected performance improvement in using sparse matrix input over dense matrix input.</li>
+<li>Table names can be optionally schema qualified (current_schemas() is searched if a schema name is not provided) and all table and column names should follow case-sensitivity and quoting rules per the database. (For instance, 'mytable' and 'MyTable' both resolve to the same entity, i.e. 'mytable'. If mixed-case or multi-byte characters are desired for entity names then the string should be double-quoted; in this case the input would be '"MyTable"').</li>
+</ul>
+</dd></dl>
+<p><a class="anchor" id="background"></a></p><dl class="section user"><dt>Technical Background</dt><dd></dd></dl>
+<p>Given a table containing some principal components \( \boldsymbol P \) and some input data \( \boldsymbol X \), the low-dimensional representation \( {\boldsymbol X}' \) is computed as </p><p class="formulaDsp">
+\begin{align*} {\boldsymbol {\hat{X}}} & = {\boldsymbol X} - \vec{e} \hat{x}^T \\ {\boldsymbol X}' & = {\boldsymbol {\hat {X}}} {\boldsymbol P}. \end{align*}
+</p>
+<p> where \(\hat{x} \) is the column means of \( \boldsymbol X \) and \( \vec{e} \) is the vector of all ones. This step is equivalent to centering the data around the origin.</p>
+<p>The residual table \( \boldsymbol R \) is a measure of how well the low-dimensional representation approximates the true input data, and is computed as </p><p class="formulaDsp">
+\[ {\boldsymbol R} = {\boldsymbol {\hat{X}}} - {\boldsymbol X}' {\boldsymbol P}^T. \]
+</p>
+<p> A residual matrix with entries mostly close to zero indicates a good representation.</p>
+<p>The residual norm \( r \) is simply </p><p class="formulaDsp">
+\[ r = \|{\boldsymbol R}\|_F \]
+</p>
+<p> where \( \|\cdot\|_F \) is the Frobenius norm. The relative residual norm \( r' \) is </p><p class="formulaDsp">
+\[ r' = \frac{ \|{\boldsymbol R}\|_F }{\|{\boldsymbol X}\|_F } \]
+</p>
+<p><a class="anchor" id="related"></a></p><dl class="section user"><dt>Related Topics</dt><dd>File <a class="el" href="pca__project_8sql__in.html" title="Principal Component Analysis Projection. ">pca_project.sql_in</a> documenting the SQL functions</dd></dl>
+<p><a class="el" href="group__grp__pca__train.html">Principal Component Analysis</a> </p>
+</div><!-- contents -->
+</div><!-- doc-content -->
+<!-- start footer part -->
+<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
+ <ul>
+ <li class="footer">Generated on Mon Jul 27 2015 20:37:45 for MADlib by
+ <a href="http://www.doxygen.org/index.html">
+ <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.10 </li>
+ </ul>
+</div>
+</body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-madlib-site/blob/7258f51a/docs/latest/group__grp__pca__train.html
----------------------------------------------------------------------
diff --git a/docs/latest/group__grp__pca__train.html b/docs/latest/group__grp__pca__train.html
new file mode 100644
index 0000000..1cff7d8
--- /dev/null
+++ b/docs/latest/group__grp__pca__train.html
@@ -0,0 +1,320 @@
+<!-- HTML header for doxygen 1.8.4-->
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
+<meta http-equiv="X-UA-Compatible" content="IE=9"/>
+<meta name="generator" content="Doxygen 1.8.10"/>
+<meta name="keywords" content="madlib,postgres,greenplum,machine learning,data mining,deep learning,ensemble methods,data science,market basket analysis,affinity analysis,pca,lda,regression,elastic net,huber white,proportional hazards,k-means,latent dirichlet allocation,bayes,support vector machines,svm"/>
+<title>MADlib: Principal Component Analysis</title>
+<link href="tabs.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="jquery.js"></script>
+<script type="text/javascript" src="dynsections.js"></script>
+<link href="navtree.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="resize.js"></script>
+<script type="text/javascript" src="navtreedata.js"></script>
+<script type="text/javascript" src="navtree.js"></script>
+<script type="text/javascript">
+ $(document).ready(initResizable);
+ $(window).load(resizeHeight);
+</script>
+<link href="search/search.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="search/searchdata.js"></script>
+<script type="text/javascript" src="search/search.js"></script>
+<script type="text/javascript">
+ $(document).ready(function() { init_search(); });
+</script>
+<script type="text/x-mathjax-config">
+ MathJax.Hub.Config({
+ extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
+ jax: ["input/TeX","output/HTML-CSS"],
+});
+</script><script src="../mathjax/MathJax.js"></script>
+<!-- hack in the navigation tree -->
+<script type="text/javascript" src="navtree_hack.js"></script>
+<link href="doxygen.css" rel="stylesheet" type="text/css" />
+<link href="madlib_extra.css" rel="stylesheet" type="text/css"/>
+<!-- google analytics -->
+<script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+ ga('create', 'UA-45382226-1', 'auto');
+ ga('send', 'pageview');
+</script>
+</head>
+<body>
+<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
+<div id="titlearea">
+<table cellspacing="0" cellpadding="0">
+ <tbody>
+ <tr style="height: 56px;">
+ <td id="projectlogo"><a href="http://madlib.incubator.apache.org"><img alt="Logo" src="madlib.png" height="50" style="padding-left:0.5em;" border="0"/ ></a></td>
+ <td style="padding-left: 0.5em;">
+ <div id="projectname">
+ <span id="projectnumber">1.8</span>
+ </div>
+ <div id="projectbrief">User Documentation for MADlib</div>
+ </td>
+ <td> <div id="MSearchBox" class="MSearchBoxInactive">
+ <span class="left">
+ <img id="MSearchSelect" src="search/mag_sel.png"
+ onmouseover="return searchBox.OnSearchSelectShow()"
+ onmouseout="return searchBox.OnSearchSelectHide()"
+ alt=""/>
+ <input type="text" id="MSearchField" value="Search" accesskey="S"
+ onfocus="searchBox.OnSearchFieldFocus(true)"
+ onblur="searchBox.OnSearchFieldFocus(false)"
+ onkeyup="searchBox.OnSearchFieldChange(event)"/>
+ </span><span class="right">
+ <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
+ </span>
+ </div>
+</td>
+ </tr>
+ </tbody>
+</table>
+</div>
+<!-- end header part -->
+<!-- Generated by Doxygen 1.8.10 -->
+<script type="text/javascript">
+var searchBox = new SearchBox("searchBox", "search",false,'Search');
+</script>
+</div><!-- top -->
+<div id="side-nav" class="ui-resizable side-nav-resizable">
+ <div id="nav-tree">
+ <div id="nav-tree-contents">
+ <div id="nav-sync" class="sync"></div>
+ </div>
+ </div>
+ <div id="splitbar" style="-moz-user-select:none;"
+ class="ui-resizable-handle">
+ </div>
+</div>
+<script type="text/javascript">
+$(document).ready(function(){initNavTree('group__grp__pca__train.html','');});
+</script>
+<div id="doc-content">
+<!-- window showing the filter options -->
+<div id="MSearchSelectWindow"
+ onmouseover="return searchBox.OnSearchSelectShow()"
+ onmouseout="return searchBox.OnSearchSelectHide()"
+ onkeydown="return searchBox.OnSearchSelectKey(event)">
+</div>
+
+<!-- iframe showing the search results (closed by default) -->
+<div id="MSearchResultsWindow">
+<iframe src="javascript:void(0)" frameborder="0"
+ name="MSearchResults" id="MSearchResults">
+</iframe>
+</div>
+
+<div class="header">
+ <div class="headertitle">
+<div class="title">Principal Component Analysis<div class="ingroups"><a class="el" href="group__grp__utility__functions.html">Utility Functions</a> » <a class="el" href="group__grp__pca.html">Dimensionality Reduction</a></div></div> </div>
+</div><!--header-->
+<div class="contents">
+<div class="toc"><b>Contents</b> </p><ul>
+<li class="level1">
+<a href="#train">Training Function</a> </li>
+<li class="level1">
+<a href="#examples">Examples</a> </li>
+<li class="level1">
+<a href="#notes">Notes</a> </li>
+<li class="level1">
+<a href="#background_pca">Technical Background</a> </li>
+<li class="level1">
+<a href="#literature">Literature</a> </li>
+<li class="level1">
+<a href="#related">Related Topics</a> </li>
+</ul>
+</div><p>Principal component analysis (PCA) is a mathematical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components. This transformation is defined in such a way that the first principal component has the largest possible variance (i.e., accounts for as much of the variability in the data as possible), and each succeeding component in turn has the highest variance possible under the constraint that it be orthogonal to (i.e., uncorrelated with) the preceding components.</p>
+<p>See the <a class="el" href="group__grp__pca__train.html#background_pca">Technical Background</a> for an introduction to principal component analysis and the implementation notes.</p>
+<p><a class="anchor" id="train"></a></p><dl class="section user"><dt>Training Function</dt><dd>The training functions have the following formats: <pre class="syntax">
+pca_train( source_table,
+ out_table,
+ row_id,
+ k,
+ grouping_cols,
+ lanczos_iter,
+ use_correlation,
+ result_summary_table
+ )
+</pre> and <pre class="syntax">
+pca_sparse_train( source_table,
+ out_table,
+ row_id,
+ col_id,
+ val_id,
+ row_dim,
+ col_dim,
+ k,
+ grouping_cols,
+ lanczos_iter,
+ use_correlation,
+ result_summary_table
+ )
+</pre></dd></dl>
+<p><b>Arguments</b> </p><dl class="arglist">
+<dt>source_table </dt>
+<dd><p class="startdd">TEXT. Name of the input table containing the data for PCA training. The input data matrix should have \( N \) rows and \( M \) columns, where \( N \) is the number of data points, and \( M \) is the number of features for each data point.</p>
+<p>A dense input table is expected to be in the one of the two standard MADlib dense matrix formats, and a sparse input table should be in the standard MADlib sparse matrix format.</p>
+<p>The two standard MADlib dense matrix formats are </p><pre>{TABLE|VIEW} <em>source_table</em> (
+ <em>row_id</em> INTEGER,
+ <em>row_vec</em> FLOAT8[],
+)</pre><p> and </p><pre>{TABLE|VIEW} <em>source_table</em> (
+ <em>row_id</em> INTEGER,
+ <em>col1</em> FLOAT8,
+ <em>col2</em> FLOAT8,
+ ...
+)</pre><p>Note that the column name <em>row_id</em> is taken as an input parameter, and should be an INTEGER and contain a list of continguous row indices from 0 to N-1, where N = number of rows.</p>
+<p>The input table for sparse PCA is expected to be in the form:</p>
+<pre>{TABLE|VIEW} <em>source_table</em> (
+ ...
+ <em>row_id</em> INTEGER,
+ <em>col_id</em> INTEGER,
+ <em>val_id</em> FLOAT8,
+ ...
+)</pre><p>The <em>row_id</em> and <em>col_id</em> columns specify which entries in the matrix are nonzero, and the <em>val_id</em> column defines the values of the nonzero entries. </p>
+<p class="enddd"></p>
+</dd>
+<dt>out_table </dt>
+<dd><p class="startdd">TEXT. The name of the table that will contain the output. The output is divided into three tables.</p>
+<p>The primary output table (<em>out_table</em>) encodes the principal components with the <em>k</em> highest eigenvalues. The table has the following columns: </p><table class="output">
+<tr>
+<th>row_id </th><td>Eigenvalue rank in descending order of the eigenvalue size. </td></tr>
+<tr>
+<th>principal_components </th><td>Vectors containing elements of the principal components. </td></tr>
+<tr>
+<th>eigen_values </th><td>The eigenvalues associated with each principal component. </td></tr>
+</table>
+<p>The table <em>out_table</em>_means contains the column means. This table has just one column: </p><table class="output">
+<tr>
+<th>column_means </th><td>A vector containing the column means for the input matrix. </td></tr>
+</table>
+<p>The optional table <em>result_summary_table</em> contains information about the performance of the PCA. The contents of this table are described under the <em>result_summary_table</em> argument. </p>
+<p class="enddd"></p>
+</dd>
+<dt>row_id </dt>
+<dd><p class="startdd">TEXT. Column name containing the row IDs in the input source table.</p>
+<p class="enddd"></p>
+</dd>
+<dt>col_id </dt>
+<dd><p class="startdd">TEXT. Name of 'col_id' column in sparse matrix representation (sparse matrices only). </p>
+<p class="enddd"></p>
+</dd>
+<dt>val_id </dt>
+<dd><p class="startdd">TEXT. Name of 'val_id' column in sparse matrix representation (sparse matrices only). </p>
+<p class="enddd"></p>
+</dd>
+<dt>row_dim </dt>
+<dd><p class="startdd">INTEGER. The number of rows in the sparse matrix (sparse matrices only). </p>
+<p class="enddd"></p>
+</dd>
+<dt>col_dim </dt>
+<dd><p class="startdd">INTEGER. The number of columns in the sparse matrix (sparse matrices only). </p>
+<p class="enddd"></p>
+</dd>
+<dt>k </dt>
+<dd><p class="startdd">INTEGER. The number of principal components to calculate from the input data. </p>
+<p class="enddd"></p>
+</dd>
+<dt>grouping_cols (optional) </dt>
+<dd><p class="startdd">TEXT, default: NULL. Currently <em>grouping_cols</em> is present as a placeholder for forward compatibility. The parameter is planned to be implemented as a comma-separated list of column names, with the source data grouped using the combination of all the columns. An independent PCA model will be computed for each combination of the grouping columns.</p>
+<p class="enddd"></p>
+</dd>
+<dt>lanczos_iter (optional) </dt>
+<dd><p class="startdd">INTEGER, default: minimum of {k+40, smallest matrix dimension}. The number of Lanczos iterations for the SVD calculation. The Lanczos iteration number roughly corresponds to the accuracy of the SVD calculation, and a higher iteration number corresponds to greater accuracy but longer computation time. The number of iterations must be at least as large as the value of <em>k</em>, but no larger than the smallest dimension of the matrix. If the iteration number is given as zero, then the default number of iterations is used.</p>
+<p class="enddd"></p>
+</dd>
+<dt>use_correlation (optional) </dt>
+<dd><p class="startdd">BOOLEAN, default FALSE. Whether to use the correlation matrix for calculating the principal components instead of the covariance matrix. Currently <em>use_correlation</em> is a placeholder for forward compatibility, and this value must be set to false.</p>
+<p class="enddd"></p>
+</dd>
+<dt>result_summary_table (optional) </dt>
+<dd><p class="startdd">TEXT, default NULL. Name of the optional summary table. When NULL, no summary table is generated.</p>
+<p class="enddd">This sumary table has the following columns: </p><table class="output">
+<tr>
+<th>rows_used </th><td>INTEGER. Number of data points in the input. </td></tr>
+<tr>
+<th>exec_time (ms) </th><td>FLOAT8. Number of milliseconds for the PCA calculation to run. </td></tr>
+<tr>
+<th>iter </th><td>INTEGER. Number of iterations used in the SVD calculation. </td></tr>
+<tr>
+<th>recon_error </th><td>FLOAT8. The absolute error in the SVD approximation. </td></tr>
+<tr>
+<th>relative_recon_error </th><td>FLOAT8. The relative error in the SVD approximation. </td></tr>
+<tr>
+<th>use_correlation </th><td>BOOLEAN. Indicates if the correlation matrix was used. </td></tr>
+</table>
+</dd>
+</dl>
+<p><a class="anchor" id="examples"></a></p><dl class="section user"><dt>Examples</dt><dd></dd></dl>
+<ol type="1">
+<li>View online help for the PCA training function. <pre class="example">
+SELECT madlib.pca_train();
+</pre></li>
+<li>Create the sample data. <pre class="example">
+DROP TABLE IF EXISTS mat;
+CREATE TABLE mat (
+ row_id integer,
+ row_vec double precision[]
+);
+COPY mat (row_id, row_vec) FROM stdin;
+0 {1,2,3}
+1 {2,1,2}
+2 {3,2,1}
+\.
+</pre></li>
+<li>Run the PCA function: <pre class="example">
+DROP TABLE result_table;
+SELECT pca_train( 'mat',
+ 'result_table',
+ 'row_id',
+ 3
+ );
+</pre></li>
+<li>View the PCA results: <pre class="example">
+SELECT * FROM result_table;
+</pre> Result <pre class="result">
+ row_id | principal_components | eigen_values
+--------+--------------------------------------------------------------+----------------------
+ 0 | {0.707106781186547,0.408248290459781,-0.577350269192513} | 2
+ 2 | {-0.707106781186547,0.408248290459781,-0.577350269192512} | 1.26294130828989e-08
+ 1 | {2.08166817117217e-17,-0.816496580931809,-0.577350269183852} | 0.816496580927726
+</pre></li>
+</ol>
+<p><a class="anchor" id="notes"></a></p><dl class="section user"><dt>Notes</dt><dd></dd></dl>
+<ul>
+<li>Table names can be optionally schema qualified (current_schemas() would be searched if a schema name is not provided) and all table and column names should follow case-sensitivity and quoting rules per the database. (For instance, 'mytable' and 'MyTable' both resolve to the same entity, i.e. 'mytable'. If mixed-case or multi-byte characters are desired for entity names then the string should be double-quoted; in this case the input would be '"MyTable"').</li>
+<li>Because of the centering step in PCA (see <a class="el" href="group__grp__pca__train.html#background_pca">Technical Background</a>), sparse matrices almost always become dense during the training process. Thus, this implementation automatically densifies sparse matrix input, and there should be no expected performance improvement in using sparse matrix input over dense matrix input.</li>
+</ul>
+<p><a class="anchor" id="background_pca"></a></p><dl class="section user"><dt>Technical Background</dt><dd></dd></dl>
+<p>The PCA implemented here uses an SVD decomposition implementation to recover the principal components (as opposed to the directly computing the eigenvectors of the covariance matrix). Let \( \boldsymbol X \) be the data matrix, and let \( \hat{x} \) be a vector of the column averages of \( \boldsymbol{X}\). PCA computes the matrix \( \hat{\boldsymbol X} \) as </p><p class="formulaDsp">
+\[ \hat{\boldsymbol X} = {\boldsymbol X} - \vec{e} \hat{x}^T \]
+</p>
+<p> where \( \vec{e} \) is the vector of all ones.</p>
+<p>PCA then computes the SVD matrix factorization </p><p class="formulaDsp">
+\[ \hat{\boldsymbol X} = {\boldsymbol U}{\boldsymbol \Sigma}{\boldsymbol V}^T \]
+</p>
+<p> where \( {\boldsymbol \Sigma} \) is a diagonal matrix. The eigenvalues are recovered as the entries of \( {\boldsymbol \Sigma}/(\sqrt{N-1}) \), and the principal components are the rows of \( {\boldsymbol V} \).</p>
+<p>It is important to note that the PCA implementation assumes that the user will use only the principal components that have non-zero eigenvalues. The SVD calculation is done with the Lanczos method, with does not guarantee correctness for singular vectors with zero-valued eigenvalues. Consequently, principal components with zero-valued eigenvalues are not guaranteed to be correct. Generally, this will not be problem unless the user wants to use the principal components for the entire eigenspectrum.</p>
+<p><a class="anchor" id="literature"></a></p><dl class="section user"><dt>Literature</dt><dd></dd></dl>
+<p>[1] Principal Component Analysis. <a href="http://en.wikipedia.org/wiki/Principal_component_analysis">http://en.wikipedia.org/wiki/Principal_component_analysis</a></p>
+<p>[2] Shlens, Jonathon (2009), A Tutorial on Principal Component Analysis</p>
+<p><a class="anchor" id="related"></a></p><dl class="section user"><dt>Related Topics</dt><dd></dd></dl>
+<p>File <a class="el" href="pca_8sql__in.html" title="Principal Component Analysis. ">pca.sql_in</a> documenting the SQL functions</p>
+<p><a class="el" href="group__grp__pca__project.html">Principal Component Projection</a> </p>
+</div><!-- contents -->
+</div><!-- doc-content -->
+<!-- start footer part -->
+<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
+ <ul>
+ <li class="footer">Generated on Mon Jul 27 2015 20:37:45 for MADlib by
+ <a href="http://www.doxygen.org/index.html">
+ <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.10 </li>
+ </ul>
+</div>
+</body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-madlib-site/blob/7258f51a/docs/latest/group__grp__pmml.html
----------------------------------------------------------------------
diff --git a/docs/latest/group__grp__pmml.html b/docs/latest/group__grp__pmml.html
new file mode 100644
index 0000000..bdef3d5
--- /dev/null
+++ b/docs/latest/group__grp__pmml.html
@@ -0,0 +1,333 @@
+<!-- HTML header for doxygen 1.8.4-->
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
+<meta http-equiv="X-UA-Compatible" content="IE=9"/>
+<meta name="generator" content="Doxygen 1.8.10"/>
+<meta name="keywords" content="madlib,postgres,greenplum,machine learning,data mining,deep learning,ensemble methods,data science,market basket analysis,affinity analysis,pca,lda,regression,elastic net,huber white,proportional hazards,k-means,latent dirichlet allocation,bayes,support vector machines,svm"/>
+<title>MADlib: PMML Export</title>
+<link href="tabs.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="jquery.js"></script>
+<script type="text/javascript" src="dynsections.js"></script>
+<link href="navtree.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="resize.js"></script>
+<script type="text/javascript" src="navtreedata.js"></script>
+<script type="text/javascript" src="navtree.js"></script>
+<script type="text/javascript">
+ $(document).ready(initResizable);
+ $(window).load(resizeHeight);
+</script>
+<link href="search/search.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="search/searchdata.js"></script>
+<script type="text/javascript" src="search/search.js"></script>
+<script type="text/javascript">
+ $(document).ready(function() { init_search(); });
+</script>
+<script type="text/x-mathjax-config">
+ MathJax.Hub.Config({
+ extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
+ jax: ["input/TeX","output/HTML-CSS"],
+});
+</script><script src="../mathjax/MathJax.js"></script>
+<!-- hack in the navigation tree -->
+<script type="text/javascript" src="navtree_hack.js"></script>
+<link href="doxygen.css" rel="stylesheet" type="text/css" />
+<link href="madlib_extra.css" rel="stylesheet" type="text/css"/>
+<!-- google analytics -->
+<script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+ ga('create', 'UA-45382226-1', 'auto');
+ ga('send', 'pageview');
+</script>
+</head>
+<body>
+<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
+<div id="titlearea">
+<table cellspacing="0" cellpadding="0">
+ <tbody>
+ <tr style="height: 56px;">
+ <td id="projectlogo"><a href="http://madlib.incubator.apache.org"><img alt="Logo" src="madlib.png" height="50" style="padding-left:0.5em;" border="0"/ ></a></td>
+ <td style="padding-left: 0.5em;">
+ <div id="projectname">
+ <span id="projectnumber">1.8</span>
+ </div>
+ <div id="projectbrief">User Documentation for MADlib</div>
+ </td>
+ <td> <div id="MSearchBox" class="MSearchBoxInactive">
+ <span class="left">
+ <img id="MSearchSelect" src="search/mag_sel.png"
+ onmouseover="return searchBox.OnSearchSelectShow()"
+ onmouseout="return searchBox.OnSearchSelectHide()"
+ alt=""/>
+ <input type="text" id="MSearchField" value="Search" accesskey="S"
+ onfocus="searchBox.OnSearchFieldFocus(true)"
+ onblur="searchBox.OnSearchFieldFocus(false)"
+ onkeyup="searchBox.OnSearchFieldChange(event)"/>
+ </span><span class="right">
+ <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
+ </span>
+ </div>
+</td>
+ </tr>
+ </tbody>
+</table>
+</div>
+<!-- end header part -->
+<!-- Generated by Doxygen 1.8.10 -->
+<script type="text/javascript">
+var searchBox = new SearchBox("searchBox", "search",false,'Search');
+</script>
+</div><!-- top -->
+<div id="side-nav" class="ui-resizable side-nav-resizable">
+ <div id="nav-tree">
+ <div id="nav-tree-contents">
+ <div id="nav-sync" class="sync"></div>
+ </div>
+ </div>
+ <div id="splitbar" style="-moz-user-select:none;"
+ class="ui-resizable-handle">
+ </div>
+</div>
+<script type="text/javascript">
+$(document).ready(function(){initNavTree('group__grp__pmml.html','');});
+</script>
+<div id="doc-content">
+<!-- window showing the filter options -->
+<div id="MSearchSelectWindow"
+ onmouseover="return searchBox.OnSearchSelectShow()"
+ onmouseout="return searchBox.OnSearchSelectHide()"
+ onkeydown="return searchBox.OnSearchSelectKey(event)">
+</div>
+
+<!-- iframe showing the search results (closed by default) -->
+<div id="MSearchResultsWindow">
+<iframe src="javascript:void(0)" frameborder="0"
+ name="MSearchResults" id="MSearchResults">
+</iframe>
+</div>
+
+<div class="header">
+ <div class="headertitle">
+<div class="title">PMML Export<div class="ingroups"><a class="el" href="group__grp__utility__functions.html">Utility Functions</a></div></div> </div>
+</div><!--header-->
+<div class="contents">
+<div class="toc"><b>Contents</b></p><ul>
+<li class="level1">
+<a href="#function">PMML Export Function</a> </li>
+<li class="level1">
+<a href="#examples">Examples</a> </li>
+<li class="level1">
+<a href="#background">Background</a> </li>
+<li class="level1">
+<a href="#related">Related Topics</a> </li>
+</ul>
+</div><p><a class="anchor" id="function"></a></p><dl class="section user"><dt>PMML Export Function</dt><dd>The PMML export function in MADlib has the following syntax: <pre class="syntax">
+pmml ( model_table,
+ name_spec
+ )
+</pre> <b>Arguments</b> <dl class="arglist">
+<dt>model_table </dt>
+<dd><p class="startdd">VARCHAR. The name of the table containing the model.</p>
+<p class="enddd"></p>
+</dd>
+<dt>name_spec (optional) </dt>
+<dd>VARCHAR or VARCHAR[]. Names to be used in the Data Dictionary of the PMML. See <a class="el" href="table__to__pmml_8sql__in.html#a9635b6989d9f972497b6b4164b77aa0a" title="Given the model constructed from a data mining algorithm, this function converts the model into PMML ...">pmml()</a> for detailed explanation. </dd>
+</dl>
+</dd></dl>
+<p><b>Output</b> XML. The output of this function is a standard PMML document, some examples of which are covered in the next section. </p>
+<dl class="section note"><dt>Note</dt><dd>In PostgreSQL, users may be required to install their database with XML support in order to use this function.</dd></dl>
+<p>Usually the user wants to export the resulting PMML contents into a PMML file so that external softwares can use it. The following method can be used (Note: the user needs to use unaligned table output mode for psql with '-A' flag. And inside psql client, both '\t' and '\o' should be used):</p>
+<pre class="example">
+> # under bash
+> psql -A my_database
+# -- in psql now
+# \t
+# \o test.pmml -- export to a file
+# select madlib.pmml('tree_out');
+# \o
+# \t
+</pre><p><a class="anchor" id="examples"></a></p><dl class="section user"><dt>Examples</dt><dd><ol type="1">
+<li>Create the training data table. <pre class="example">
+CREATE TABLE patients( id integer NOT NULL,
+ second_attack integer,
+ treatment integer,
+ trait_anxiety integer);
+INSERT INTO patients(id, second_attack, treatment, trait_anxiety) VALUES
+( 1, 1, 1, 70),
+( 3, 1, 1, 50),
+( 5, 1, 0, 40),
+( 7, 1, 0, 75),
+( 9, 1, 0, 70),
+(11, 0, 1, 65),
+(13, 0, 1, 45),
+(15, 0, 1, 40),
+(17, 0, 0, 55),
+(19, 0, 0, 50),
+( 2, 1, 1, 80),
+( 4, 1, 0, 60),
+( 6, 1, 0, 65),
+( 8, 1, 0, 80),
+(10, 1, 0, 60),
+(12, 0, 1, 50),
+(14, 0, 1, 35),
+(16, 0, 1, 50),
+(18, 0, 0, 45),
+(20, 0, 0, 60);
+</pre></li>
+<li>Train a regression model using <a class="el" href="logistic_8sql__in.html#a74210a7ef513dfcbdfdd9f3b37bfe428" title="Compute logistic-regression coefficients and diagnostic statistics. ">logregr_train()</a>. <pre class="example">
+SELECT madlib.logregr_train(
+ 'patients',
+ 'patients_logregr',
+ 'second_attack',
+ 'ARRAY[1, treatment, trait_anxiety]');
+</pre></li>
+<li>View the PMML export for this model. <pre class="example">
+SELECT madlib.pmml('patients_logregr');
+</pre> Result: <pre class="result">
+<?xml version="1.0" standalone="yes"?>
+<PMML version="4.1" xmlns="http://www.dmg.org/pmml-v4-1.html">
+ <Header copyright="Copyright (c) 2014 gpadmin">
+ <Extension extender="MADlib" name="user" value="gpadmin">
+ <Application name="MADlib" version="1.7">
+ <Timestamp>
+ 2014-06-13 17:30:14.527899 PDT
+ </Timestamp>
+ </Header>
+ <DataDictionary numberoffields="4">
+ <DataField datatype="boolean" name="second_attack_pmml_prediction" optype="categorical">
+ <DataField datatype="double" name="1" optype="continuous">
+ <DataField datatype="double" name="treatment" optype="continuous">
+ <DataField datatype="double" name="trait_anxiety" optype="continuous">
+ </DataDictionary>
+ <RegressionModel functionname="classification" normalizationmethod="softmax">
+ <MiningSchema>
+ <MiningField name="second_attack_pmml_prediction" usagetype="predicted">
+ <MiningField name="1">
+ <MiningField name="treatment">
+ <MiningField name="trait_anxiety">
+ </MiningSchema>
+ <RegressionTable intercept="0.0" targetcategory="True">
+ <NumericPredictor coefficient="-6.36346994178" name="1">
+ <NumericPredictor coefficient="-1.02410605239" name="treatment">
+ <NumericPredictor coefficient="0.119044916669" name="trait_anxiety">
+ </RegressionTable>
+ <RegressionTable intercept="0.0" targetcategory="False">
+ </RegressionModel>
+</PMML>
+</pre></li>
+</ol>
+</dd></dl>
+<p>Alternatively, the above can also be invoked as below if custom names are needed for fields in the Data Dictionary: </p><pre class="example">
+SELECT madlib.pmml('patients_logregr',
+ 'out_attack~1+in_trait_anxiety+in_treatment');
+</pre><p><b>Note:</b> If the second argument of 'pmml' function is not specified, a default suffix "_pmml_prediction" will be automatically append to the column name to be predicted. This can help avoid name conflicts.</p>
+<p>The following example demonstrates grouping columns in the model table for the same dataset as the previous example.</p>
+<ol type="1">
+<li>Train a different regression model with 'treatment' as the grouping column. <pre class="example">
+SELECT madlib.logregr_train(
+ 'patients',
+ 'patients_logregr_grouping',
+ 'second_attack',
+ 'ARRAY[1, trait_anxiety]',
+ 'treatment');
+</pre></li>
+<li>View the PMML export for this model. <pre class="example">
+SELECT madlib.pmml('patients_logregr_grouping',
+ ARRAY['second_attack','1','in_trait_anxiety']);
+</pre> Result: <pre class="result">
+<?xml version="1.0" standalone="yes"?>
+ <PMML version="4.1" xmlns="http://www.dmg.org/pmml-v4-1.html">
+ <Header copyright="Copyright (c) 2014 gpadmin">
+ <Extension extender="MADlib" name="user" value="gpadmin">
+ <Application name="MADlib" version="1.7">
+ <Timestamp>
+ 2014-06-13 17:37:55.786307 PDT
+ </Timestamp>
+ </Header>
+ <DataDictionary numberoffields="4">
+ <DataField datatype="boolean" name="second_attack" optype="categorical">
+ <DataField datatype="double" name="1" optype="continuous">
+ <DataField datatype="double" name="in_trait_anxiety" optype="continuous">
+ <DataField datatype="string" name="treatment" optype="categorical">
+ </DataDictionary>
+ <MiningModel functionname="classification">
+ <MiningSchema>
+ <MiningField name="second_attack" usagetype="predicted">
+ <MiningField name="1">
+ <MiningField name="in_trait_anxiety">
+ <MiningField name="treatment">
+ </MiningSchema>
+ <Segmentation multiplemodelmethod="selectFirst">
+ <Segment>
+ <SimplePredicate field="treatment" operator="equal" value="1">
+ <RegressionModel functionname="classification" normalizationmethod="softmax">
+ <MiningSchema>
+ <MiningField name="second_attack" usagetype="predicted">
+ <MiningField name="1">
+ <MiningField name="in_trait_anxiety">
+ </MiningSchema>
+ <RegressionTable intercept="0.0" targetcategory="True">
+ <NumericPredictor coefficient="-8.02068430057" name="1">
+ <NumericPredictor coefficient="0.130090428526" name="in_trait_anxiety">
+ </RegressionTable>
+ <RegressionTable intercept="0.0" targetcategory="False">
+ </RegressionModel>
+ </Segment>
+ <Segment>
+ <SimplePredicate field="treatment" operator="equal" value="0">
+ <RegressionModel functionname="classification" normalizationmethod="softmax">
+ <MiningSchema>
+ <MiningField name="second_attack" usagetype="predicted">
+ <MiningField name="1">
+ <MiningField name="in_trait_anxiety">
+ </MiningSchema>
+ <RegressionTable intercept="0.0" targetcategory="True">
+ <NumericPredictor coefficient="-5.75043192191" name="1">
+ <NumericPredictor coefficient="0.108282446319" name="in_trait_anxiety">
+ </RegressionTable>
+ <RegressionTable intercept="0.0" targetcategory="False">
+ </RegressionModel>
+ </Segment>
+ </Segmentation>
+ </MiningModel>
+ </PMML>
+</pre></li>
+</ol>
+<p><b>Note:</b> MADlib currently supports PMML export for Linear Regression, Logistic Regression, Generalized Linear Regression Model, Multinomial Logistic Regression, Ordinal Linear Regression, Decision Tree and Random Forests.</p>
+<p>In Ordinal Regression, the signs of feature coefficients will be different in PMML export and in the default output model table from ordinal(). This is due to the difference of model settings.</p>
+<p><a class="anchor" id="background"></a></p><dl class="section user"><dt>Background</dt><dd>The Predictive Model Markup Language (PMML) is an XML-based file format that provides a way for applications to describe and exchange models produced by data mining and machine learning algorithms. A PMML file comprises the following components:<ul>
+<li>Header: Contains general information of the model, such as copyright information and model description.</li>
+<li>Data Dictionary: Contains definitions of fields used in the model.</li>
+<li>Data Transformations: Contains transformations for mapping user data into a form that can be used by the mining model.</li>
+<li>Model: Contains definitions of the data mining model, which includes attributes such as the model name, function name, and algorithm name.</li>
+<li>Mining Schema: Contains specific information for the fields used in the model, which includes the name and usage type.</li>
+<li>Targets: Allows for post-processing of the predicted value.</li>
+<li>Output: Allows for naming of output fields expected from the model.</li>
+</ul>
+</dd></dl>
+<p>MADlib follows the PMML v4.1 standard. For more details about PMML, see <a href="http://www.dmg.org/v4-1/GeneralStructure.html">http://www.dmg.org/v4-1/GeneralStructure.html</a>.</p>
+<p><a class="anchor" id="related"></a></p><dl class="section user"><dt>Related Topics</dt><dd></dd></dl>
+<p>File <a class="el" href="table__to__pmml_8sql__in.html">table_to_pmml.sql_in</a> documenting the PMML export functions.</p>
+<p><a class="el" href="group__grp__linreg.html">Linear Regression</a></p>
+<p><a class="el" href="group__grp__logreg.html">Logistic Regression</a></p>
+<p><a class="el" href="group__grp__glm.html">Generalized Linear Models</a></p>
+<p><a class="el" href="group__grp__ordinal.html">Ordinal Regression</a></p>
+<p><a class="el" href="group__grp__multinom.html">Multinomial Regression</a></p>
+<p><a class="el" href="group__grp__decision__tree.html">Decision Tree</a></p>
+<p><a class="el" href="group__grp__random__forest.html">Random Forest</a> </p>
+</div><!-- contents -->
+</div><!-- doc-content -->
+<!-- start footer part -->
+<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
+ <ul>
+ <li class="footer">Generated on Mon Jul 27 2015 20:37:45 for MADlib by
+ <a href="http://www.doxygen.org/index.html">
+ <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.10 </li>
+ </ul>
+</div>
+</body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-madlib-site/blob/7258f51a/docs/latest/group__grp__prob.html
----------------------------------------------------------------------
diff --git a/docs/latest/group__grp__prob.html b/docs/latest/group__grp__prob.html
new file mode 100644
index 0000000..1bca282
--- /dev/null
+++ b/docs/latest/group__grp__prob.html
@@ -0,0 +1,165 @@
+<!-- HTML header for doxygen 1.8.4-->
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
+<meta http-equiv="X-UA-Compatible" content="IE=9"/>
+<meta name="generator" content="Doxygen 1.8.10"/>
+<meta name="keywords" content="madlib,postgres,greenplum,machine learning,data mining,deep learning,ensemble methods,data science,market basket analysis,affinity analysis,pca,lda,regression,elastic net,huber white,proportional hazards,k-means,latent dirichlet allocation,bayes,support vector machines,svm"/>
+<title>MADlib: Probability Functions</title>
+<link href="tabs.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="jquery.js"></script>
+<script type="text/javascript" src="dynsections.js"></script>
+<link href="navtree.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="resize.js"></script>
+<script type="text/javascript" src="navtreedata.js"></script>
+<script type="text/javascript" src="navtree.js"></script>
+<script type="text/javascript">
+ $(document).ready(initResizable);
+ $(window).load(resizeHeight);
+</script>
+<link href="search/search.css" rel="stylesheet" type="text/css"/>
+<script type="text/javascript" src="search/searchdata.js"></script>
+<script type="text/javascript" src="search/search.js"></script>
+<script type="text/javascript">
+ $(document).ready(function() { init_search(); });
+</script>
+<script type="text/x-mathjax-config">
+ MathJax.Hub.Config({
+ extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
+ jax: ["input/TeX","output/HTML-CSS"],
+});
+</script><script src="../mathjax/MathJax.js"></script>
+<!-- hack in the navigation tree -->
+<script type="text/javascript" src="navtree_hack.js"></script>
+<link href="doxygen.css" rel="stylesheet" type="text/css" />
+<link href="madlib_extra.css" rel="stylesheet" type="text/css"/>
+<!-- google analytics -->
+<script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+ ga('create', 'UA-45382226-1', 'auto');
+ ga('send', 'pageview');
+</script>
+</head>
+<body>
+<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
+<div id="titlearea">
+<table cellspacing="0" cellpadding="0">
+ <tbody>
+ <tr style="height: 56px;">
+ <td id="projectlogo"><a href="http://madlib.incubator.apache.org"><img alt="Logo" src="madlib.png" height="50" style="padding-left:0.5em;" border="0"/ ></a></td>
+ <td style="padding-left: 0.5em;">
+ <div id="projectname">
+ <span id="projectnumber">1.8</span>
+ </div>
+ <div id="projectbrief">User Documentation for MADlib</div>
+ </td>
+ <td> <div id="MSearchBox" class="MSearchBoxInactive">
+ <span class="left">
+ <img id="MSearchSelect" src="search/mag_sel.png"
+ onmouseover="return searchBox.OnSearchSelectShow()"
+ onmouseout="return searchBox.OnSearchSelectHide()"
+ alt=""/>
+ <input type="text" id="MSearchField" value="Search" accesskey="S"
+ onfocus="searchBox.OnSearchFieldFocus(true)"
+ onblur="searchBox.OnSearchFieldFocus(false)"
+ onkeyup="searchBox.OnSearchFieldChange(event)"/>
+ </span><span class="right">
+ <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
+ </span>
+ </div>
+</td>
+ </tr>
+ </tbody>
+</table>
+</div>
+<!-- end header part -->
+<!-- Generated by Doxygen 1.8.10 -->
+<script type="text/javascript">
+var searchBox = new SearchBox("searchBox", "search",false,'Search');
+</script>
+</div><!-- top -->
+<div id="side-nav" class="ui-resizable side-nav-resizable">
+ <div id="nav-tree">
+ <div id="nav-tree-contents">
+ <div id="nav-sync" class="sync"></div>
+ </div>
+ </div>
+ <div id="splitbar" style="-moz-user-select:none;"
+ class="ui-resizable-handle">
+ </div>
+</div>
+<script type="text/javascript">
+$(document).ready(function(){initNavTree('group__grp__prob.html','');});
+</script>
+<div id="doc-content">
+<!-- window showing the filter options -->
+<div id="MSearchSelectWindow"
+ onmouseover="return searchBox.OnSearchSelectShow()"
+ onmouseout="return searchBox.OnSearchSelectHide()"
+ onkeydown="return searchBox.OnSearchSelectKey(event)">
+</div>
+
+<!-- iframe showing the search results (closed by default) -->
+<div id="MSearchResultsWindow">
+<iframe src="javascript:void(0)" frameborder="0"
+ name="MSearchResults" id="MSearchResults">
+</iframe>
+</div>
+
+<div class="header">
+ <div class="headertitle">
+<div class="title">Probability Functions<div class="ingroups"><a class="el" href="group__grp__stats.html">Statistics</a></div></div> </div>
+</div><!--header-->
+<div class="contents">
+<div class="toc"><b>Contents</b> </p><ul>
+<li>
+<a href="#syntax">Function Syntax</a> </li>
+<li>
+<a href="#examples">Examples</a> </li>
+<li>
+<a href="#literature">Literature</a> </li>
+<li>
+<a href="#related">Related Topics</a> </li>
+</ul>
+</div><p>The Probability Functions module provides cumulative distribution, density/mass, and quantile functions for a wide range of probability distributions.</p>
+<p>Unless otherwise documented, all of these functions are wrappers around functionality provided by the boost C++ library [1, “<a href="http://www.boost.org/doc/libs/1_49_0/libs/math/doc/sf_and_dist/html/math_toolkit/dist.html">Statistical Distributions and Functions</a>”].</p>
+<p>For convenience, all cumulative distribution and density/mass functions (CDFs and PDF/PMFs in short) are defined over the range of all floating-point numbers including infinity. Inputs that are <code>NULL</code> or <code>NaN</code> (not a number) will always produce a <code>NULL</code> or <code>NaN</code> result, respectively. Inputs that are plus or minus infinity will return the respective limits.</p>
+<p>A quantile function for a probability distrution with CDF \( F \) takes a probability argument \( p \in [0,1] \) and returns the value \( x \) so that \( F(x) = p \), provided such an \( x \) exists and it is unique. If it does not, the result will be \( \sup \{ x \in D \mid F(x) \leq p \} \) (interpreted as 0 if the supremum is over an empty set) if \( p < 0.5 \), and \( \inf \{ x \in D \mid F(x) \geq p \} \) if \( p \geq 0.5 \). Here \( D \) denotes the domain of the distribution, which is the set of reals \( \mathbb R \) for continuous and the set of nonnegative integers \( \mathbb N_0 \) for discrete distributions.</p>
+<p>Intuitively, the formulas in the previous paragraph deal with the following special cases. The 0-quantile will always be the “left end” of the support, and the 1-quantile will be the “right end” of the support of the distribution. For discrete distributions, most values of \( p \in [0,1] \) do not admit an \( x \) with \( F(x) = p \). Instead, there is an \( x \in \mathbb N_0 \) so that \( F(x) < p < F(x + 1) \). The above formulas mean that the value returned as \( p \)-quantile is \( x \) if \( p < 0.5 \), and it is \( x + 1 \) if \( p \geq 0.5 \). (As a special case, in order to ensure that quantiles are always within the support, the \( p \)-quantile will be 0 if \( p < F(0) \)).</p>
+<p>The rationale for choosing this behavior is that \(p\)-quantiles for \( p < 0.5 \) are typically requested when interested in the value \( x \) such that with confidence level <b>at least</b> \( 1 - p \) a random variable will be \( > x \) (or equivalently, with probability <b>at most</b> \( p \), it will be \( \leq x \)). Likewise, \(p\)-quantiles for \( p \geq 0.5 \) are typically requested when interested in the value \( x \) such that with confidence level <b>at least</b> \( p \) a random variable will be \( \leq x \). See also [1, “<a href="http://www.boost.org/doc/libs/1_46_1/libs/math/doc/sf_and_dist/html/math_toolkit/policy/pol_tutorial/understand_dis_quant.html">Understanding Quantiles of Discrete Distributions</a>”].</p>
+<p><a class="anchor" id="syntax"></a></p><dl class="section user"><dt>Function Syntax</dt><dd></dd></dl>
+<p>Cumulative distribution functions:</p>
+<pre class="syntax"><em>distribution</em>_cdf(<em>random variate</em>[, <em>parameter1</em> [, <em>parameter2</em> [, <em>parameter3</em>] ] ])</pre><p>Probability density/mass functions: </p><pre class="syntax"><em>distribution</em>_{pdf|pmf}(<em>random variate</em>[, <em>parameter1</em> [, <em>parameter2</em> [, <em>parameter3</em>] ] ])</pre><p>Quantile functions: </p><pre class="syntax"><em>distribution</em>_quantile(<em>probability</em>[, <em>parameter1</em> [, <em>parameter2</em> [, <em>parameter3</em>] ] ])</pre><p>For concrete function signatures, see <a class="el" href="prob_8sql__in.html">prob.sql_in</a>.</p>
+<p><a class="anchor" id="examples"></a></p><dl class="section user"><dt>Examples</dt><dd></dd></dl>
+<pre class="example">
+SELECT madlib.normal_cdf(0);
+</pre><p> Result: </p><pre class="result">
+ normal_cdf
+ -----------
+ 0.5
+</pre> <pre class="example">
+SELECT madlib.normal_quantile(0.5, 0, 1);
+</pre><p> Result: </p><pre class="result">
+ normal_quantile
+ ----------------
+ 0
+(1 row)
+</pre><p><a class="anchor" id="literature"></a></p><dl class="section user"><dt>Literature</dt><dd></dd></dl>
+<p>[1] John Maddock, Paul A. Bristow, Hubert Holin, Xiaogang Zhang, Bruno Lalande, Johan Råde, Gautam Sewani and Thijs van den Berg: <em>Boost Math Toolkit</em>, Version 1.49, available at: <a href="http://www.boost.org/doc/libs/1_49_0/libs/math/doc/sf_and_dist/html/index.html">http://www.boost.org/doc/libs/1_49_0/libs/math/doc/sf_and_dist/html/index.html</a></p>
+<dl class="section user"><dt>Related Topics</dt><dd><a class="anchor" id="related"></a>File <a class="el" href="prob_8sql__in.html" title="SQL functions for evaluating probability functions. ">prob.sql_in</a> documenting the SQL functions. </dd></dl>
+</div><!-- contents -->
+</div><!-- doc-content -->
+<!-- start footer part -->
+<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
+ <ul>
+ <li class="footer">Generated on Mon Jul 27 2015 20:37:45 for MADlib by
+ <a href="http://www.doxygen.org/index.html">
+ <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.10 </li>
+ </ul>
+</div>
+</body>
+</html>