You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@systemml.apache.org by du...@apache.org on 2015/12/02 02:04:39 UTC
[01/47] incubator-systemml git commit: Add Algorithms and Language
References
Repository: incubator-systemml
Updated Branches:
refs/heads/gh-pages [created] cbc960226
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Language Reference/SystemML_Language_Reference.html
----------------------------------------------------------------------
diff --git a/Language Reference/SystemML_Language_Reference.html b/Language Reference/SystemML_Language_Reference.html
new file mode 100644
index 0000000..76e2c04
--- /dev/null
+++ b/Language Reference/SystemML_Language_Reference.html
@@ -0,0 +1,9801 @@
+<html xmlns:v="urn:schemas-microsoft-com:vml"
+xmlns:o="urn:schemas-microsoft-com:office:office"
+xmlns:w="urn:schemas-microsoft-com:office:word"
+xmlns:m="http://schemas.microsoft.com/office/2004/12/omml"
+xmlns="http://www.w3.org/TR/REC-html40">
+
+<head>
+<meta http-equiv=Content-Type content="text/html; charset=windows-1252">
+<meta name=ProgId content=Word.Document>
+<meta name=Generator content="Microsoft Word 14">
+<meta name=Originator content="Microsoft Word 14">
+<link rel=File-List href="SystemML_Language_Reference_files/filelist.xml">
+<title>DML</title>
+<!--[if gte mso 9]><xml>
+ <o:DocumentProperties>
+ <o:Author>reinwald</o:Author>
+ <o:LastAuthor>ADMINIBM</o:LastAuthor>
+ <o:Revision>612</o:Revision>
+ <o:TotalTime>5832</o:TotalTime>
+ <o:LastPrinted>2014-03-27T08:52:00Z</o:LastPrinted>
+ <o:Created>2014-08-09T07:22:00Z</o:Created>
+ <o:LastSaved>2015-07-30T11:30:00Z</o:LastSaved>
+ <o:Pages>48</o:Pages>
+ <o:Words>11282</o:Words>
+ <o:Characters>64308</o:Characters>
+ <o:Company>IBM</o:Company>
+ <o:Lines>535</o:Lines>
+ <o:Paragraphs>150</o:Paragraphs>
+ <o:CharactersWithSpaces>75440</o:CharactersWithSpaces>
+ <o:Version>14.00</o:Version>
+ </o:DocumentProperties>
+</xml><![endif]-->
+<link rel=themeData href="SystemML_Language_Reference_files/themedata.thmx">
+<link rel=colorSchemeMapping
+href="SystemML_Language_Reference_files/colorschememapping.xml">
+<!--[if gte mso 9]><xml>
+ <w:WordDocument>
+ <w:View>Print</w:View>
+ <w:Zoom>130</w:Zoom>
+ <w:SpellingState>Clean</w:SpellingState>
+ <w:GrammarState>Clean</w:GrammarState>
+ <w:TrackMoves>false</w:TrackMoves>
+ <w:TrackFormatting/>
+ <w:ValidateAgainstSchemas/>
+ <w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
+ <w:IgnoreMixedContent>false</w:IgnoreMixedContent>
+ <w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
+ <w:DoNotPromoteQF/>
+ <w:LidThemeOther>EN-US</w:LidThemeOther>
+ <w:LidThemeAsian>X-NONE</w:LidThemeAsian>
+ <w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
+ <w:Compatibility>
+ <w:BreakWrappedTables/>
+ <w:SnapToGridInCell/>
+ <w:WrapTextWithPunct/>
+ <w:UseAsianBreakRules/>
+ <w:DontGrowAutofit/>
+ <w:SplitPgBreakAndParaMark/>
+ <w:DontVertAlignCellWithSp/>
+ <w:DontBreakConstrainedForcedTables/>
+ <w:DontVertAlignInTxbx/>
+ <w:Word11KerningPairs/>
+ <w:CachedColBalance/>
+ </w:Compatibility>
+ <w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel>
+ <m:mathPr>
+ <m:mathFont m:val="Cambria Math"/>
+ <m:brkBin m:val="before"/>
+ <m:brkBinSub m:val="--"/>
+ <m:smallFrac m:val="off"/>
+ <m:dispDef/>
+ <m:lMargin m:val="0"/>
+ <m:rMargin m:val="0"/>
+ <m:defJc m:val="centerGroup"/>
+ <m:wrapIndent m:val="1440"/>
+ <m:intLim m:val="subSup"/>
+ <m:naryLim m:val="undOvr"/>
+ </m:mathPr></w:WordDocument>
+</xml><![endif]--><!--[if gte mso 9]><xml>
+ <w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true"
+ DefSemiHidden="true" DefQFormat="false" DefPriority="99"
+ LatentStyleCount="267">
+ <w:LsdException Locked="false" Priority="0" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="Normal"/>
+ <w:LsdException Locked="false" Priority="9" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="heading 1"/>
+ <w:LsdException Locked="false" Priority="9" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="heading 2"/>
+ <w:LsdException Locked="false" Priority="9" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="heading 3"/>
+ <w:LsdException Locked="false" Priority="9" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="heading 4"/>
+ <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/>
+ <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/>
+ <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/>
+ <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/>
+ <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/>
+ <w:LsdException Locked="false" Priority="39" Name="toc 1"/>
+ <w:LsdException Locked="false" Priority="39" Name="toc 2"/>
+ <w:LsdException Locked="false" Priority="39" Name="toc 3"/>
+ <w:LsdException Locked="false" Priority="39" Name="toc 4"/>
+ <w:LsdException Locked="false" Priority="39" Name="toc 5"/>
+ <w:LsdException Locked="false" Priority="39" Name="toc 6"/>
+ <w:LsdException Locked="false" Priority="39" Name="toc 7"/>
+ <w:LsdException Locked="false" Priority="39" Name="toc 8"/>
+ <w:LsdException Locked="false" Priority="39" Name="toc 9"/>
+ <w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/>
+ <w:LsdException Locked="false" Priority="10" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="Title"/>
+ <w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/>
+ <w:LsdException Locked="false" Priority="11" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/>
+ <w:LsdException Locked="false" Priority="22" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="Strong"/>
+ <w:LsdException Locked="false" Priority="20" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/>
+ <w:LsdException Locked="false" Priority="59" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Table Grid"/>
+ <w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/>
+ <w:LsdException Locked="false" Priority="1" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/>
+ <w:LsdException Locked="false" Priority="60" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Shading"/>
+ <w:LsdException Locked="false" Priority="61" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light List"/>
+ <w:LsdException Locked="false" Priority="62" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Grid"/>
+ <w:LsdException Locked="false" Priority="63" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 1"/>
+ <w:LsdException Locked="false" Priority="64" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 2"/>
+ <w:LsdException Locked="false" Priority="65" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 1"/>
+ <w:LsdException Locked="false" Priority="66" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 2"/>
+ <w:LsdException Locked="false" Priority="67" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 1"/>
+ <w:LsdException Locked="false" Priority="68" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 2"/>
+ <w:LsdException Locked="false" Priority="69" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 3"/>
+ <w:LsdException Locked="false" Priority="70" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Dark List"/>
+ <w:LsdException Locked="false" Priority="71" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Shading"/>
+ <w:LsdException Locked="false" Priority="72" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful List"/>
+ <w:LsdException Locked="false" Priority="73" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Grid"/>
+ <w:LsdException Locked="false" Priority="60" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Shading Accent 1"/>
+ <w:LsdException Locked="false" Priority="61" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light List Accent 1"/>
+ <w:LsdException Locked="false" Priority="62" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Grid Accent 1"/>
+ <w:LsdException Locked="false" Priority="63" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/>
+ <w:LsdException Locked="false" Priority="64" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/>
+ <w:LsdException Locked="false" Priority="65" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/>
+ <w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/>
+ <w:LsdException Locked="false" Priority="34" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/>
+ <w:LsdException Locked="false" Priority="29" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="Quote"/>
+ <w:LsdException Locked="false" Priority="30" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/>
+ <w:LsdException Locked="false" Priority="66" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/>
+ <w:LsdException Locked="false" Priority="67" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/>
+ <w:LsdException Locked="false" Priority="68" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/>
+ <w:LsdException Locked="false" Priority="69" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/>
+ <w:LsdException Locked="false" Priority="70" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Dark List Accent 1"/>
+ <w:LsdException Locked="false" Priority="71" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/>
+ <w:LsdException Locked="false" Priority="72" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful List Accent 1"/>
+ <w:LsdException Locked="false" Priority="73" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/>
+ <w:LsdException Locked="false" Priority="60" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Shading Accent 2"/>
+ <w:LsdException Locked="false" Priority="61" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light List Accent 2"/>
+ <w:LsdException Locked="false" Priority="62" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Grid Accent 2"/>
+ <w:LsdException Locked="false" Priority="63" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/>
+ <w:LsdException Locked="false" Priority="64" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/>
+ <w:LsdException Locked="false" Priority="65" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/>
+ <w:LsdException Locked="false" Priority="66" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/>
+ <w:LsdException Locked="false" Priority="67" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/>
+ <w:LsdException Locked="false" Priority="68" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/>
+ <w:LsdException Locked="false" Priority="69" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/>
+ <w:LsdException Locked="false" Priority="70" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Dark List Accent 2"/>
+ <w:LsdException Locked="false" Priority="71" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/>
+ <w:LsdException Locked="false" Priority="72" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful List Accent 2"/>
+ <w:LsdException Locked="false" Priority="73" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/>
+ <w:LsdException Locked="false" Priority="60" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Shading Accent 3"/>
+ <w:LsdException Locked="false" Priority="61" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light List Accent 3"/>
+ <w:LsdException Locked="false" Priority="62" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Grid Accent 3"/>
+ <w:LsdException Locked="false" Priority="63" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/>
+ <w:LsdException Locked="false" Priority="64" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/>
+ <w:LsdException Locked="false" Priority="65" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/>
+ <w:LsdException Locked="false" Priority="66" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/>
+ <w:LsdException Locked="false" Priority="67" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/>
+ <w:LsdException Locked="false" Priority="68" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/>
+ <w:LsdException Locked="false" Priority="69" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/>
+ <w:LsdException Locked="false" Priority="70" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Dark List Accent 3"/>
+ <w:LsdException Locked="false" Priority="71" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/>
+ <w:LsdException Locked="false" Priority="72" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful List Accent 3"/>
+ <w:LsdException Locked="false" Priority="73" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/>
+ <w:LsdException Locked="false" Priority="60" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Shading Accent 4"/>
+ <w:LsdException Locked="false" Priority="61" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light List Accent 4"/>
+ <w:LsdException Locked="false" Priority="62" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Grid Accent 4"/>
+ <w:LsdException Locked="false" Priority="63" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/>
+ <w:LsdException Locked="false" Priority="64" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/>
+ <w:LsdException Locked="false" Priority="65" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/>
+ <w:LsdException Locked="false" Priority="66" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/>
+ <w:LsdException Locked="false" Priority="67" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/>
+ <w:LsdException Locked="false" Priority="68" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/>
+ <w:LsdException Locked="false" Priority="69" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/>
+ <w:LsdException Locked="false" Priority="70" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Dark List Accent 4"/>
+ <w:LsdException Locked="false" Priority="71" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/>
+ <w:LsdException Locked="false" Priority="72" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful List Accent 4"/>
+ <w:LsdException Locked="false" Priority="73" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/>
+ <w:LsdException Locked="false" Priority="60" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Shading Accent 5"/>
+ <w:LsdException Locked="false" Priority="61" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light List Accent 5"/>
+ <w:LsdException Locked="false" Priority="62" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Grid Accent 5"/>
+ <w:LsdException Locked="false" Priority="63" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/>
+ <w:LsdException Locked="false" Priority="64" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/>
+ <w:LsdException Locked="false" Priority="65" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/>
+ <w:LsdException Locked="false" Priority="66" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/>
+ <w:LsdException Locked="false" Priority="67" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/>
+ <w:LsdException Locked="false" Priority="68" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/>
+ <w:LsdException Locked="false" Priority="69" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/>
+ <w:LsdException Locked="false" Priority="70" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Dark List Accent 5"/>
+ <w:LsdException Locked="false" Priority="71" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/>
+ <w:LsdException Locked="false" Priority="72" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful List Accent 5"/>
+ <w:LsdException Locked="false" Priority="73" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/>
+ <w:LsdException Locked="false" Priority="60" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Shading Accent 6"/>
+ <w:LsdException Locked="false" Priority="61" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light List Accent 6"/>
+ <w:LsdException Locked="false" Priority="62" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Light Grid Accent 6"/>
+ <w:LsdException Locked="false" Priority="63" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/>
+ <w:LsdException Locked="false" Priority="64" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/>
+ <w:LsdException Locked="false" Priority="65" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/>
+ <w:LsdException Locked="false" Priority="66" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/>
+ <w:LsdException Locked="false" Priority="67" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/>
+ <w:LsdException Locked="false" Priority="68" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/>
+ <w:LsdException Locked="false" Priority="69" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/>
+ <w:LsdException Locked="false" Priority="70" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Dark List Accent 6"/>
+ <w:LsdException Locked="false" Priority="71" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/>
+ <w:LsdException Locked="false" Priority="72" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful List Accent 6"/>
+ <w:LsdException Locked="false" Priority="73" SemiHidden="false"
+ UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/>
+ <w:LsdException Locked="false" Priority="19" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/>
+ <w:LsdException Locked="false" Priority="21" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/>
+ <w:LsdException Locked="false" Priority="31" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/>
+ <w:LsdException Locked="false" Priority="32" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/>
+ <w:LsdException Locked="false" Priority="33" SemiHidden="false"
+ UnhideWhenUsed="false" QFormat="true" Name="Book Title"/>
+ <w:LsdException Locked="false" Priority="37" Name="Bibliography"/>
+ <w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/>
+ </w:LatentStyles>
+</xml><![endif]-->
+<style>
+<!--
+ /* Font Definitions */
+ @font-face
+ {font-family:Courier;
+ panose-1:2 7 4 9 2 2 5 2 4 4;
+ mso-font-charset:0;
+ mso-generic-font-family:modern;
+ mso-font-format:other;
+ mso-font-pitch:fixed;
+ mso-font-signature:3 0 0 0 1 0;}
+@font-face
+ {font-family:Helv;
+ panose-1:2 11 6 4 2 2 2 3 2 4;
+ mso-font-alt:Arial;
+ mso-font-charset:0;
+ mso-generic-font-family:swiss;
+ mso-font-format:other;
+ mso-font-pitch:variable;
+ mso-font-signature:3 0 0 0 1 0;}
+@font-face
+ {font-family:Wingdings;
+ panose-1:5 0 0 0 0 0 0 0 0 0;
+ mso-font-charset:2;
+ mso-generic-font-family:auto;
+ mso-font-pitch:variable;
+ mso-font-signature:0 268435456 0 0 -2147483648 0;}
+@font-face
+ {font-family:Wingdings;
+ panose-1:5 0 0 0 0 0 0 0 0 0;
+ mso-font-charset:2;
+ mso-generic-font-family:auto;
+ mso-font-pitch:variable;
+ mso-font-signature:0 268435456 0 0 -2147483648 0;}
+@font-face
+ {font-family:Cambria;
+ panose-1:2 4 5 3 5 4 6 3 2 4;
+ mso-font-charset:0;
+ mso-generic-font-family:roman;
+ mso-font-pitch:variable;
+ mso-font-signature:-536870145 1073743103 0 0 415 0;}
+@font-face
+ {font-family:Calibri;
+ panose-1:2 15 5 2 2 2 4 3 2 4;
+ mso-font-charset:0;
+ mso-generic-font-family:swiss;
+ mso-font-pitch:variable;
+ mso-font-signature:-536870145 1073786111 1 0 415 0;}
+@font-face
+ {font-family:Tahoma;
+ panose-1:2 11 6 4 3 5 4 4 2 4;
+ mso-font-charset:0;
+ mso-generic-font-family:swiss;
+ mso-font-pitch:variable;
+ mso-font-signature:-520081665 -1073717157 41 0 66047 0;}
+@font-face
+ {font-family:Consolas;
+ panose-1:2 11 6 9 2 2 4 3 2 4;
+ mso-font-charset:0;
+ mso-generic-font-family:modern;
+ mso-font-pitch:fixed;
+ mso-font-signature:-520092929 1073806591 9 0 415 0;}
+ /* Style Definitions */
+ p.MsoNormal, li.MsoNormal, div.MsoNormal
+ {mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-style-parent:"";
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ font-size:12.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";}
+h1
+ {mso-style-priority:9;
+ mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-style-link:"Heading 1 Char";
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ mso-outline-level:1;
+ font-size:24.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:minor-fareast;
+ font-weight:bold;}
+h2
+ {mso-style-priority:9;
+ mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-style-link:"Heading 2 Char";
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ mso-outline-level:2;
+ font-size:18.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:minor-fareast;
+ font-weight:bold;}
+h3
+ {mso-style-priority:9;
+ mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-style-link:"Heading 3 Char";
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ mso-outline-level:3;
+ font-size:13.5pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:minor-fareast;
+ font-weight:bold;}
+h4
+ {mso-style-priority:9;
+ mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-style-link:"Heading 4 Char";
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ page-break-after:avoid;
+ mso-outline-level:4;
+ font-size:12.0pt;
+ font-family:"Cambria","serif";
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:minor-fareast;
+ color:#4F81BD;
+ font-weight:bold;
+ font-style:italic;}
+p.MsoToc1, li.MsoToc1, div.MsoToc1
+ {mso-style-update:auto;
+ mso-style-noshow:yes;
+ mso-style-priority:39;
+ mso-style-next:Normal;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ font-size:12.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";}
+p.MsoToc2, li.MsoToc2, div.MsoToc2
+ {mso-style-update:auto;
+ mso-style-noshow:yes;
+ mso-style-priority:39;
+ mso-style-next:Normal;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:12.0pt;
+ mso-pagination:widow-orphan;
+ font-size:12.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";}
+p.MsoToc3, li.MsoToc3, div.MsoToc3
+ {mso-style-update:auto;
+ mso-style-noshow:yes;
+ mso-style-priority:39;
+ mso-style-next:Normal;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:24.0pt;
+ mso-pagination:widow-orphan;
+ font-size:12.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";}
+p.MsoCaption, li.MsoCaption, div.MsoCaption
+ {mso-style-noshow:yes;
+ mso-style-priority:35;
+ mso-style-qformat:yes;
+ mso-style-next:Normal;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ font-size:9.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";
+ color:#4F81BD;
+ mso-themecolor:accent1;
+ font-weight:bold;}
+p.MsoTitle, li.MsoTitle, div.MsoTitle
+ {mso-style-noshow:yes;
+ mso-style-priority:10;
+ mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-style-link:"Title Char";
+ mso-style-next:Normal;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-add-space:auto;
+ mso-pagination:widow-orphan;
+ border:none;
+ mso-border-bottom-alt:solid #4F81BD 1.0pt;
+ mso-border-bottom-themecolor:accent1;
+ padding:0in;
+ mso-padding-alt:0in 0in 4.0pt 0in;
+ font-size:26.0pt;
+ font-family:"Cambria","serif";
+ mso-ascii-font-family:Cambria;
+ mso-ascii-theme-font:major-latin;
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:major-fareast;
+ mso-hansi-font-family:Cambria;
+ mso-hansi-theme-font:major-latin;
+ mso-bidi-font-family:"Times New Roman";
+ mso-bidi-theme-font:major-bidi;
+ color:#17365D;
+ mso-themecolor:text2;
+ mso-themeshade:191;
+ letter-spacing:.25pt;
+ mso-font-kerning:14.0pt;}
+p.MsoTitleCxSpFirst, li.MsoTitleCxSpFirst, div.MsoTitleCxSpFirst
+ {mso-style-noshow:yes;
+ mso-style-priority:10;
+ mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-style-link:"Title Char";
+ mso-style-next:Normal;
+ mso-style-type:export-only;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-add-space:auto;
+ mso-pagination:widow-orphan;
+ border:none;
+ mso-border-bottom-alt:solid #4F81BD 1.0pt;
+ mso-border-bottom-themecolor:accent1;
+ padding:0in;
+ mso-padding-alt:0in 0in 4.0pt 0in;
+ font-size:26.0pt;
+ font-family:"Cambria","serif";
+ mso-ascii-font-family:Cambria;
+ mso-ascii-theme-font:major-latin;
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:major-fareast;
+ mso-hansi-font-family:Cambria;
+ mso-hansi-theme-font:major-latin;
+ mso-bidi-font-family:"Times New Roman";
+ mso-bidi-theme-font:major-bidi;
+ color:#17365D;
+ mso-themecolor:text2;
+ mso-themeshade:191;
+ letter-spacing:.25pt;
+ mso-font-kerning:14.0pt;}
+p.MsoTitleCxSpMiddle, li.MsoTitleCxSpMiddle, div.MsoTitleCxSpMiddle
+ {mso-style-noshow:yes;
+ mso-style-priority:10;
+ mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-style-link:"Title Char";
+ mso-style-next:Normal;
+ mso-style-type:export-only;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-add-space:auto;
+ mso-pagination:widow-orphan;
+ border:none;
+ mso-border-bottom-alt:solid #4F81BD 1.0pt;
+ mso-border-bottom-themecolor:accent1;
+ padding:0in;
+ mso-padding-alt:0in 0in 4.0pt 0in;
+ font-size:26.0pt;
+ font-family:"Cambria","serif";
+ mso-ascii-font-family:Cambria;
+ mso-ascii-theme-font:major-latin;
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:major-fareast;
+ mso-hansi-font-family:Cambria;
+ mso-hansi-theme-font:major-latin;
+ mso-bidi-font-family:"Times New Roman";
+ mso-bidi-theme-font:major-bidi;
+ color:#17365D;
+ mso-themecolor:text2;
+ mso-themeshade:191;
+ letter-spacing:.25pt;
+ mso-font-kerning:14.0pt;}
+p.MsoTitleCxSpLast, li.MsoTitleCxSpLast, div.MsoTitleCxSpLast
+ {mso-style-noshow:yes;
+ mso-style-priority:10;
+ mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-style-link:"Title Char";
+ mso-style-next:Normal;
+ mso-style-type:export-only;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-add-space:auto;
+ mso-pagination:widow-orphan;
+ border:none;
+ mso-border-bottom-alt:solid #4F81BD 1.0pt;
+ mso-border-bottom-themecolor:accent1;
+ padding:0in;
+ mso-padding-alt:0in 0in 4.0pt 0in;
+ font-size:26.0pt;
+ font-family:"Cambria","serif";
+ mso-ascii-font-family:Cambria;
+ mso-ascii-theme-font:major-latin;
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:major-fareast;
+ mso-hansi-font-family:Cambria;
+ mso-hansi-theme-font:major-latin;
+ mso-bidi-font-family:"Times New Roman";
+ mso-bidi-theme-font:major-bidi;
+ color:#17365D;
+ mso-themecolor:text2;
+ mso-themeshade:191;
+ letter-spacing:.25pt;
+ mso-font-kerning:14.0pt;}
+p.MsoSubtitle, li.MsoSubtitle, div.MsoSubtitle
+ {mso-style-noshow:yes;
+ mso-style-priority:11;
+ mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-style-link:"Subtitle Char";
+ mso-style-next:Normal;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ font-size:12.0pt;
+ font-family:"Cambria","serif";
+ mso-ascii-font-family:Cambria;
+ mso-ascii-theme-font:major-latin;
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:major-fareast;
+ mso-hansi-font-family:Cambria;
+ mso-hansi-theme-font:major-latin;
+ mso-bidi-font-family:"Times New Roman";
+ mso-bidi-theme-font:major-bidi;
+ color:#4F81BD;
+ mso-themecolor:accent1;
+ letter-spacing:.75pt;
+ font-style:italic;}
+a:link, span.MsoHyperlink
+ {mso-style-noshow:yes;
+ mso-style-priority:99;
+ color:blue;
+ text-decoration:underline;
+ text-underline:single;}
+a:visited, span.MsoHyperlinkFollowed
+ {mso-style-noshow:yes;
+ mso-style-priority:99;
+ color:blue;
+ text-decoration:underline;
+ text-underline:single;}
+p
+ {mso-style-priority:99;
+ mso-style-link:"Normal \(Web\) Char";
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ font-size:12.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";}
+pre
+ {mso-style-noshow:yes;
+ mso-style-priority:99;
+ mso-style-link:"HTML Preformatted Char";
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;
+ font-size:10.0pt;
+ font-family:"Courier New";
+ mso-fareast-font-family:"Times New Roman";}
+p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
+ {mso-style-noshow:yes;
+ mso-style-priority:99;
+ mso-style-link:"Balloon Text Char";
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ font-size:8.0pt;
+ font-family:"Tahoma","sans-serif";
+ mso-fareast-font-family:"Times New Roman";}
+p.MsoNoSpacing, li.MsoNoSpacing, div.MsoNoSpacing
+ {mso-style-noshow:yes;
+ mso-style-priority:1;
+ mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ font-size:12.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";}
+p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
+ {mso-style-noshow:yes;
+ mso-style-priority:34;
+ mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:.5in;
+ mso-pagination:widow-orphan;
+ font-size:12.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";}
+p.MsoTocHeading, li.MsoTocHeading, div.MsoTocHeading
+ {mso-style-noshow:yes;
+ mso-style-priority:39;
+ mso-style-qformat:yes;
+ mso-style-parent:"Heading 1";
+ mso-style-next:Normal;
+ margin-top:24.0pt;
+ margin-right:0in;
+ margin-bottom:0in;
+ margin-left:0in;
+ margin-bottom:.0001pt;
+ line-height:115%;
+ mso-pagination:widow-orphan lines-together;
+ page-break-after:avoid;
+ font-size:14.0pt;
+ font-family:"Cambria","serif";
+ mso-ascii-font-family:Cambria;
+ mso-ascii-theme-font:major-latin;
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:major-fareast;
+ mso-hansi-font-family:Cambria;
+ mso-hansi-theme-font:major-latin;
+ mso-bidi-font-family:"Times New Roman";
+ mso-bidi-theme-font:major-bidi;
+ color:#365F91;
+ mso-themecolor:accent1;
+ mso-themeshade:191;
+ font-weight:bold;}
+span.Heading1Char
+ {mso-style-name:"Heading 1 Char";
+ mso-style-priority:9;
+ mso-style-unhide:no;
+ mso-style-locked:yes;
+ mso-style-link:"Heading 1";
+ font-family:"Cambria","serif";
+ mso-ascii-font-family:Cambria;
+ mso-hansi-font-family:Cambria;
+ color:#365F91;
+ font-weight:bold;}
+span.Heading2Char
+ {mso-style-name:"Heading 2 Char";
+ mso-style-noshow:yes;
+ mso-style-priority:9;
+ mso-style-unhide:no;
+ mso-style-locked:yes;
+ mso-style-link:"Heading 2";
+ font-family:"Cambria","serif";
+ mso-ascii-font-family:Cambria;
+ mso-hansi-font-family:Cambria;
+ color:#4F81BD;
+ font-weight:bold;}
+span.Heading3Char
+ {mso-style-name:"Heading 3 Char";
+ mso-style-noshow:yes;
+ mso-style-priority:9;
+ mso-style-unhide:no;
+ mso-style-locked:yes;
+ mso-style-link:"Heading 3";
+ font-family:"Cambria","serif";
+ mso-ascii-font-family:Cambria;
+ mso-hansi-font-family:Cambria;
+ color:#4F81BD;
+ font-weight:bold;}
+span.Heading4Char
+ {mso-style-name:"Heading 4 Char";
+ mso-style-noshow:yes;
+ mso-style-priority:9;
+ mso-style-unhide:no;
+ mso-style-locked:yes;
+ mso-style-link:"Heading 4";
+ font-family:"Cambria","serif";
+ mso-ascii-font-family:Cambria;
+ mso-hansi-font-family:Cambria;
+ color:#4F81BD;
+ font-weight:bold;
+ font-style:italic;}
+span.HTMLPreformattedChar
+ {mso-style-name:"HTML Preformatted Char";
+ mso-style-noshow:yes;
+ mso-style-priority:99;
+ mso-style-unhide:no;
+ mso-style-locked:yes;
+ mso-style-link:"HTML Preformatted";
+ font-family:Consolas;
+ mso-ascii-font-family:Consolas;
+ mso-hansi-font-family:Consolas;
+ mso-bidi-font-family:Consolas;
+ color:black;}
+span.NormalWebChar
+ {mso-style-name:"Normal \(Web\) Char";
+ mso-style-priority:99;
+ mso-style-unhide:no;
+ mso-style-locked:yes;
+ mso-style-link:"Normal \(Web\)";
+ mso-ansi-font-size:12.0pt;
+ mso-bidi-font-size:12.0pt;
+ font-family:"Times New Roman","serif";
+ mso-ascii-font-family:"Times New Roman";
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:minor-fareast;
+ mso-hansi-font-family:"Times New Roman";
+ mso-bidi-font-family:"Times New Roman";
+ color:black;}
+span.TitleChar
+ {mso-style-name:"Title Char";
+ mso-style-priority:10;
+ mso-style-unhide:no;
+ mso-style-locked:yes;
+ mso-style-link:Title;
+ mso-ansi-font-size:26.0pt;
+ mso-bidi-font-size:26.0pt;
+ font-family:"Cambria","serif";
+ mso-ascii-font-family:Cambria;
+ mso-ascii-theme-font:major-latin;
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:major-fareast;
+ mso-hansi-font-family:Cambria;
+ mso-hansi-theme-font:major-latin;
+ mso-bidi-font-family:"Times New Roman";
+ mso-bidi-theme-font:major-bidi;
+ color:#17365D;
+ mso-themecolor:text2;
+ mso-themeshade:191;
+ letter-spacing:.25pt;
+ mso-font-kerning:14.0pt;}
+span.SubtitleChar
+ {mso-style-name:"Subtitle Char";
+ mso-style-priority:11;
+ mso-style-unhide:no;
+ mso-style-locked:yes;
+ mso-style-link:Subtitle;
+ mso-ansi-font-size:12.0pt;
+ mso-bidi-font-size:12.0pt;
+ font-family:"Cambria","serif";
+ mso-ascii-font-family:Cambria;
+ mso-ascii-theme-font:major-latin;
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:major-fareast;
+ mso-hansi-font-family:Cambria;
+ mso-hansi-theme-font:major-latin;
+ mso-bidi-font-family:"Times New Roman";
+ mso-bidi-theme-font:major-bidi;
+ color:#4F81BD;
+ mso-themecolor:accent1;
+ letter-spacing:.75pt;
+ font-style:italic;}
+span.BalloonTextChar
+ {mso-style-name:"Balloon Text Char";
+ mso-style-noshow:yes;
+ mso-style-priority:99;
+ mso-style-unhide:no;
+ mso-style-locked:yes;
+ mso-style-link:"Balloon Text";
+ mso-ansi-font-size:8.0pt;
+ mso-bidi-font-size:8.0pt;
+ font-family:"Tahoma","sans-serif";
+ mso-ascii-font-family:Tahoma;
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:minor-fareast;
+ mso-hansi-font-family:Tahoma;
+ mso-bidi-font-family:Tahoma;
+ color:black;}
+p.msolistparagraphcxspfirst, li.msolistparagraphcxspfirst, div.msolistparagraphcxspfirst
+ {mso-style-name:msolistparagraphcxspfirst;
+ mso-style-noshow:yes;
+ mso-style-priority:99;
+ mso-style-unhide:no;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:.5in;
+ mso-pagination:widow-orphan;
+ font-size:12.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";}
+p.msolistparagraphcxspmiddle, li.msolistparagraphcxspmiddle, div.msolistparagraphcxspmiddle
+ {mso-style-name:msolistparagraphcxspmiddle;
+ mso-style-noshow:yes;
+ mso-style-priority:99;
+ mso-style-unhide:no;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:.5in;
+ mso-pagination:widow-orphan;
+ font-size:12.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";}
+p.msolistparagraphcxsplast, li.msolistparagraphcxsplast, div.msolistparagraphcxsplast
+ {mso-style-name:msolistparagraphcxsplast;
+ mso-style-noshow:yes;
+ mso-style-priority:99;
+ mso-style-unhide:no;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:.5in;
+ mso-pagination:widow-orphan;
+ font-size:12.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";}
+p.msochpdefault, li.msochpdefault, div.msochpdefault
+ {mso-style-name:msochpdefault;
+ mso-style-noshow:yes;
+ mso-style-priority:99;
+ mso-style-unhide:no;
+ mso-margin-top-alt:auto;
+ margin-right:0in;
+ mso-margin-bottom-alt:auto;
+ margin-left:0in;
+ mso-pagination:widow-orphan;
+ font-size:10.0pt;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";}
+span.CodeChar
+ {mso-style-name:"Code Char";
+ mso-style-noshow:yes;
+ mso-style-priority:99;
+ mso-style-unhide:no;
+ mso-style-locked:yes;
+ mso-style-parent:"Normal \(Web\) Char";
+ mso-style-link:Code;
+ mso-ansi-font-size:11.0pt;
+ mso-bidi-font-size:11.0pt;
+ font-family:"Courier New";
+ mso-ascii-font-family:"Courier New";
+ mso-fareast-font-family:"Times New Roman";
+ mso-fareast-theme-font:minor-fareast;
+ mso-hansi-font-family:"Courier New";
+ mso-bidi-font-family:"Courier New";
+ color:black;
+ mso-bidi-font-style:italic;}
+p.Code, li.Code, div.Code
+ {mso-style-name:Code;
+ mso-style-update:auto;
+ mso-style-noshow:yes;
+ mso-style-priority:99;
+ mso-style-unhide:no;
+ mso-style-qformat:yes;
+ mso-style-parent:"Normal \(Web\)";
+ mso-style-link:"Code Char";
+ mso-style-next:Normal;
+ margin-top:0in;
+ margin-right:0in;
+ margin-bottom:0in;
+ margin-left:.5in;
+ margin-bottom:.0001pt;
+ mso-pagination:widow-orphan;
+ tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;
+ font-size:11.0pt;
+ font-family:"Courier New";
+ mso-fareast-font-family:"Times New Roman";
+ mso-bidi-font-style:italic;}
+span.SpellE
+ {mso-style-name:"";
+ mso-spl-e:yes;}
+span.GramE
+ {mso-style-name:"";
+ mso-gram-e:yes;}
+.MsoChpDefault
+ {mso-style-type:export-only;
+ mso-default-props:yes;
+ font-size:10.0pt;
+ mso-ansi-font-size:10.0pt;
+ mso-bidi-font-size:10.0pt;}
+@page WordSection1
+ {size:8.5in 11.0in;
+ margin:1.0in 1.0in 1.0in 1.0in;
+ mso-header-margin:.5in;
+ mso-footer-margin:.5in;
+ mso-paper-source:0;}
+div.WordSection1
+ {page:WordSection1;}
+ /* List Definitions */
+ @list l0
+ {mso-list-id:123081263;
+ mso-list-type:hybrid;
+ mso-list-template-ids:1895480452 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
+@list l0:level1
+ {mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l0:level2
+ {mso-level-tab-stop:1.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l0:level3
+ {mso-level-tab-stop:1.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l0:level4
+ {mso-level-tab-stop:2.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l0:level5
+ {mso-level-tab-stop:2.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l0:level6
+ {mso-level-tab-stop:3.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l0:level7
+ {mso-level-tab-stop:3.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l0:level8
+ {mso-level-tab-stop:4.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l0:level9
+ {mso-level-tab-stop:4.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l1
+ {mso-list-id:353769049;
+ mso-list-type:hybrid;
+ mso-list-template-ids:1155662276 -808390566 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
+@list l1:level1
+ {mso-level-start-at:0;
+ mso-level-number-format:bullet;
+ mso-level-text:-;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Times New Roman","serif";
+ mso-fareast-font-family:"Times New Roman";}
+@list l1:level2
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l1:level3
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l1:level4
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l1:level5
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l1:level6
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l1:level7
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l1:level8
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l1:level9
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l2
+ {mso-list-id:538591595;
+ mso-list-type:hybrid;
+ mso-list-template-ids:-263824488 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
+@list l2:level1
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l2:level2
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l2:level3
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l2:level4
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l2:level5
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l2:level6
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l2:level7
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l2:level8
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l2:level9
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l3
+ {mso-list-id:656424215;
+ mso-list-type:hybrid;
+ mso-list-template-ids:-1122056928 67698691 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
+@list l3:level1
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l3:level2
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l3:level3
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l3:level4
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l3:level5
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l3:level6
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l3:level7
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l3:level8
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l3:level9
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l4
+ {mso-list-id:822896956;
+ mso-list-type:hybrid;
+ mso-list-template-ids:-265523238 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
+@list l4:level1
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ margin-left:.25in;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l4:level2
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ margin-left:.75in;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l4:level3
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ margin-left:1.25in;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l4:level4
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ margin-left:1.75in;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l4:level5
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ margin-left:2.25in;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l4:level6
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ margin-left:2.75in;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l4:level7
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ margin-left:3.25in;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l4:level8
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ margin-left:3.75in;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l4:level9
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ margin-left:4.25in;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l5
+ {mso-list-id:847646372;
+ mso-list-type:hybrid;
+ mso-list-template-ids:766823076 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
+@list l5:level1
+ {mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l5:level2
+ {mso-level-number-format:alpha-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l5:level3
+ {mso-level-number-format:roman-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:right;
+ text-indent:-9.0pt;}
+@list l5:level4
+ {mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l5:level5
+ {mso-level-number-format:alpha-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l5:level6
+ {mso-level-number-format:roman-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:right;
+ text-indent:-9.0pt;}
+@list l5:level7
+ {mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l5:level8
+ {mso-level-number-format:alpha-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l5:level9
+ {mso-level-number-format:roman-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:right;
+ text-indent:-9.0pt;}
+@list l6
+ {mso-list-id:875699675;
+ mso-list-type:hybrid;
+ mso-list-template-ids:679874930 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
+@list l6:level1
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ margin-left:1.0in;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l6:level2
+ {mso-level-tab-stop:1.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l6:level3
+ {mso-level-tab-stop:1.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l6:level4
+ {mso-level-tab-stop:2.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l6:level5
+ {mso-level-tab-stop:2.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l6:level6
+ {mso-level-tab-stop:3.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l6:level7
+ {mso-level-tab-stop:3.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l6:level8
+ {mso-level-tab-stop:4.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l6:level9
+ {mso-level-tab-stop:4.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l7
+ {mso-list-id:892732566;
+ mso-list-type:hybrid;
+ mso-list-template-ids:771816730 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
+@list l7:level1
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ margin-left:1.0in;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l7:level2
+ {mso-level-tab-stop:1.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l7:level3
+ {mso-level-tab-stop:1.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l7:level4
+ {mso-level-tab-stop:2.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l7:level5
+ {mso-level-tab-stop:2.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l7:level6
+ {mso-level-tab-stop:3.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l7:level7
+ {mso-level-tab-stop:3.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l7:level8
+ {mso-level-tab-stop:4.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l7:level9
+ {mso-level-tab-stop:4.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l8
+ {mso-list-id:1183400259;
+ mso-list-type:hybrid;
+ mso-list-template-ids:-318178610 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
+@list l8:level1
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ margin-left:1.0in;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l8:level2
+ {mso-level-tab-stop:1.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l8:level3
+ {mso-level-tab-stop:1.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l8:level4
+ {mso-level-tab-stop:2.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l8:level5
+ {mso-level-tab-stop:2.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l8:level6
+ {mso-level-tab-stop:3.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l8:level7
+ {mso-level-tab-stop:3.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l8:level8
+ {mso-level-tab-stop:4.0in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l8:level9
+ {mso-level-tab-stop:4.5in;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l9
+ {mso-list-id:1398086136;
+ mso-list-type:hybrid;
+ mso-list-template-ids:-458853048 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
+@list l9:level1
+ {mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l9:level2
+ {mso-level-number-format:alpha-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l9:level3
+ {mso-level-number-format:roman-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:right;
+ text-indent:-9.0pt;}
+@list l9:level4
+ {mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l9:level5
+ {mso-level-number-format:alpha-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l9:level6
+ {mso-level-number-format:roman-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:right;
+ text-indent:-9.0pt;}
+@list l9:level7
+ {mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l9:level8
+ {mso-level-number-format:alpha-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l9:level9
+ {mso-level-number-format:roman-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:right;
+ text-indent:-9.0pt;}
+@list l10
+ {mso-list-id:1784762355;
+ mso-list-type:hybrid;
+ mso-list-template-ids:439810870 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
+@list l10:level1
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l10:level2
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l10:level3
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l10:level4
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l10:level5
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l10:level6
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l10:level7
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0B7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Symbol;}
+@list l10:level8
+ {mso-level-number-format:bullet;
+ mso-level-text:o;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:"Courier New";}
+@list l10:level9
+ {mso-level-number-format:bullet;
+ mso-level-text:\F0A7;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;
+ font-family:Wingdings;}
+@list l11
+ {mso-list-id:1808859173;
+ mso-list-type:hybrid;
+ mso-list-template-ids:1566222360 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
+@list l11:level1
+ {mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l11:level2
+ {mso-level-number-format:alpha-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l11:level3
+ {mso-level-number-format:roman-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:right;
+ text-indent:-9.0pt;}
+@list l11:level4
+ {mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l11:level5
+ {mso-level-number-format:alpha-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l11:level6
+ {mso-level-number-format:roman-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:right;
+ text-indent:-9.0pt;}
+@list l11:level7
+ {mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l11:level8
+ {mso-level-number-format:alpha-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:left;
+ text-indent:-.25in;}
+@list l11:level9
+ {mso-level-number-format:roman-lower;
+ mso-level-tab-stop:none;
+ mso-level-number-position:right;
+ text-indent:-9.0pt;}
+-->
+</style>
+<!--[if gte mso 10]>
+<style>
+ /* Style Definitions */
+ table.MsoNormalTable
+ {mso-style-name:"Table Normal";
+ mso-tstyle-rowband-size:0;
+ mso-tstyle-colband-size:0;
+ mso-style-noshow:yes;
+ mso-style-priority:99;
+ mso-style-parent:"";
+ mso-padding-alt:0in 5.4pt 0in 5.4pt;
+ mso-para-margin:0in;
+ mso-para-margin-bottom:.0001pt;
+ mso-pagination:widow-orphan;
+ font-size:10.0pt;
+ font-family:"Times New Roman","serif";}
+table.MsoTableGrid
+ {mso-style-name:"Table Grid";
+ mso-tstyle-rowband-size:0;
+ mso-tstyle-colband-size:0;
+ mso-style-priority:59;
+ mso-style-unhide:no;
+ border:solid windowtext 1.0pt;
+ mso-border-alt:solid windowtext .5pt;
+ mso-padding-alt:0in 5.4pt 0in 5.4pt;
+ mso-border-insideh:.5pt solid windowtext;
+ mso-border-insidev:.5pt solid windowtext;
+ mso-para-margin:0in;
+ mso-para-margin-bottom:.0001pt;
+ mso-pagination:widow-orphan;
+ font-size:10.0pt;
+ font-family:"Times New Roman","serif";}
+</style>
+<![endif]-->
+<meta name=CREATED content="0;0">
+<meta name=CHANGED content="20120624;22041132">
+<meta name=CHANGEDBY content=" ">
+<meta name=CHANGEDBY content=" ">
+<meta name=CHANGEDBY content=" ">
+<meta name=CHANGEDBY content=" ">
+<meta name=CHANGEDBY content=" ">
+<!--[if gte mso 9]><xml>
+ <o:shapedefaults v:ext="edit" spidmax="1026"/>
+</xml><![endif]--><!--[if gte mso 9]><xml>
+ <o:shapelayout v:ext="edit">
+ <o:idmap v:ext="edit" data="1"/>
+ </o:shapelayout></xml><![endif]-->
+</head>
+
+<body lang=EN-US link=blue vlink=blue style='tab-interval:.5in'>
+
+<div class=WordSection1>
+
+<div style='mso-element:para-border-div;border:none;border-bottom:solid #4F81BD 1.0pt;
+mso-border-bottom-themecolor:accent1;padding:0in 0in 4.0pt 0in'>
+
+<p class=MsoTitle>SystemML Language Reference</p>
+
+</div>
+
+<p class=MsoSubtitle><a name=mozTocId182601></a>August 2014<o:p></o:p></p>
+
+<w:sdt sdtdocpart="t" docparttype="Table of Contents" docpartunique="t" id="472146699">
+
+<p class=MsoTocHeading>Table of Contents<w:sdtpr></w:sdtpr></p>
+
+<p class=MsoToc1 style='tab-stops:right dotted 467.5pt'><!--[if supportFields]><span
+style='mso-element:field-begin'></span><span
+style='mso-spacerun:yes'>�</span>TOC \o "1-3" \h \z \u <span
+style='mso-element:field-separator'></span><![endif]--><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846316">Introduction<span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846316 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>2<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300310036000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc1 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846317">Variables<span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846317 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>3<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300310037000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc2 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846318">Identifier
+Names<span style='color:windowtext;display:none;mso-hide:screen;text-decoration:
+none;text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846318 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>3<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300310038000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc2 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846319">Data
+Types<span style='color:windowtext;display:none;mso-hide:screen;text-decoration:
+none;text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846319 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>3<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300310039000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc2 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846320">Comments<span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846320 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>4<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300320030000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc1 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846321">Expressions<span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846321 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>4<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300320031000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc2 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846322">Operators<span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846322 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>4<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300320032000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc2 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846323">Matrix-Vector
+operations<span style='color:windowtext;display:none;mso-hide:screen;
+text-decoration:none;text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846323 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>5<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300320033000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc2 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846324">Matrix
+Indexing<span style='color:windowtext;display:none;mso-hide:screen;text-decoration:
+none;text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846324 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>6<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300320034000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc1 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846325">Statements<span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846325 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>7<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300320035000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc2 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846326">Assignment
+Statement<span style='color:windowtext;display:none;mso-hide:screen;text-decoration:
+none;text-underline:none'><span style='mso-tab-count:1 dotted'> </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846326 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>7<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300320036000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc2 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846327">Control
+Statements<span style='color:windowtext;display:none;mso-hide:screen;
+text-decoration:none;text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846327 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>7<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300320037000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc3 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846328">While
+Statement<span style='color:windowtext;display:none;mso-hide:screen;text-decoration:
+none;text-underline:none'><span style='mso-tab-count:1 dotted'> </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846328 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>7<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300320038000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc3 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846329">If
+Statement<span style='color:windowtext;display:none;mso-hide:screen;text-decoration:
+none;text-underline:none'><span style='mso-tab-count:1 dotted'> </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846329 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>8<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300320039000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc3 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846330">For
+Statement<span style='color:windowtext;display:none;mso-hide:screen;text-decoration:
+none;text-underline:none'><span style='mso-tab-count:1 dotted'> </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846330 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>9<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300330030000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc3 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846331">ParFor
+Statement<span style='color:windowtext;display:none;mso-hide:screen;text-decoration:
+none;text-underline:none'><span style='mso-tab-count:1 dotted'> </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846331 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>9<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300330031000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc2 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846332">User-defined
+Function (UDF)<span style='color:windowtext;display:none;mso-hide:screen;
+text-decoration:none;text-underline:none'><span style='mso-tab-count:1 dotted'> </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846332 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>10<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300330032000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc1 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846333">Variable
+Scoping<span style='color:windowtext;display:none;mso-hide:screen;text-decoration:
+none;text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846333 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>12<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300330033000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc1 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846334">Command-line
+arguments<span style='color:windowtext;display:none;mso-hide:screen;text-decoration:
+none;text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846334 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>13<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300330034000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc1 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846335">Built-in
+Functions<span style='color:windowtext;display:none;mso-hide:screen;text-decoration:
+none;text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846335 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>15<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054006F0063003400320035003800340036003300330035000000</w:data>
+</xml><![endif]--></span><!--[if supportFields]><span style='color:windowtext;
+display:none;mso-hide:screen;text-decoration:none;text-underline:none'><span
+style='mso-element:field-end'></span></span><![endif]--></a></span></span><span
+style='font-size:11.0pt;font-family:"Calibri","sans-serif";mso-ascii-theme-font:
+minor-latin;mso-fareast-font-family:"Times New Roman";mso-fareast-theme-font:
+minor-fareast;mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times New Roman";
+mso-bidi-theme-font:minor-bidi;mso-no-proof:yes'><o:p></o:p></span></p>
+
+<p class=MsoToc3 style='tab-stops:right dotted 467.5pt'><span
+class=MsoHyperlink><span style='mso-no-proof:yes'><a href="#_Toc425846336">Matrix
+Construction, Manipulation, and Aggregation Built-In Functions<span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-tab-count:1 dotted'>. </span></span><!--[if supportFields]><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'><span style='mso-element:field-begin'></span> PAGEREF
+_Toc425846336 \h <span style='mso-element:field-separator'></span></span><![endif]--><span
+style='color:windowtext;display:none;mso-hide:screen;text-decoration:none;
+text-underline:none'>16<!--[if gte mso 9]><xml>
+ <w:data>08D0C9EA79F9BACE118C8200AA004BA90B02000000080000000E0000005F0054
<TRUNCATED>
[24/47] incubator-systemml git commit: Add Quick Start Guide to
Documentation dropdown menu
Posted by du...@apache.org.
Add Quick Start Guide to Documentation dropdown menu
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/f1eb4289
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/f1eb4289
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/f1eb4289
Branch: refs/heads/gh-pages
Commit: f1eb4289b782a6dafa209c8ee320dc86d453cb7e
Parents: c0d0fde
Author: Christian Kadner <ck...@us.ibm.com>
Authored: Thu Sep 10 18:59:12 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Thu Sep 10 18:59:12 2015 -0700
----------------------------------------------------------------------
_layouts/global.html | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/f1eb4289/_layouts/global.html
----------------------------------------------------------------------
diff --git a/_layouts/global.html b/_layouts/global.html
index 6499efb..2a75531 100644
--- a/_layouts/global.html
+++ b/_layouts/global.html
@@ -35,6 +35,7 @@
<ul class="dropdown-menu">
<li><a href="http://www.github.com/SparkTC/systemml">SystemML GitHub README</a></li>
+ <li><a href="quick-start-guide.html">Quick Start Guide</a></li>
<li><a href="algorithms-reference.html">Algorithms Reference</a></li>
<li><a href="dml-language-reference.html">DML Language Reference</a></li>
<li class="divider"></li>
[02/47] incubator-systemml git commit: Add Algorithms and Language
References
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/Kmeans.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/Kmeans.tex b/Algorithms Reference/Kmeans.tex
new file mode 100644
index 0000000..0b8cbcd
--- /dev/null
+++ b/Algorithms Reference/Kmeans.tex
@@ -0,0 +1,350 @@
+\subsection{K-Means Clustering}
+
+\noindent{\bf Description}
+\smallskip
+
+Given a collection of $n$ records with a pairwise similarity measure,
+the goal of clustering is to assign a category label to each record so that
+similar records tend to get the same label. In contrast to multinomial
+logistic regression, clustering is an \emph{unsupervised}\/ learning problem
+with neither category assignments nor label interpretations given in advance.
+In $k$-means clustering, the records $x_1, x_2, \ldots, x_n$ are numerical
+feature vectors of $\dim x_i = m$ with the squared Euclidean distance
+$\|x_i - x_{i'}\|_2^2$ as the similarity measure. We want to partition
+$\{x_1, \ldots, x_n\}$ into $k$ clusters $\{S_1, \ldots, S_k\}$ so that
+the aggregated squared distance from records to their cluster means is
+minimized:
+\begin{equation}
+\textrm{WCSS}\,\,=\,\, \sum_{i=1}^n \,\big\|x_i - \mean(S_j: x_i\in S_j)\big\|_2^2 \,\,\to\,\,\min
+\label{eqn:WCSS}
+\end{equation}
+The aggregated distance measure in~(\ref{eqn:WCSS}) is called the
+\emph{within-cluster sum of squares}~(WCSS). It can be viewed as a measure
+of residual variance that remains in the data after the clustering assignment,
+conceptually similar to the residual sum of squares~(RSS) in linear regression.
+However, unlike for the RSS, the minimization of~(\ref{eqn:WCSS}) is an NP-hard
+problem~\cite{AloiseDHP2009:kmeans}.
+
+Rather than searching for the global optimum in~(\ref{eqn:WCSS}), a heuristic algorithm
+called Lloyd's algorithm is typically used. This iterative algorithm maintains
+and updates a set of $k$~\emph{centroids} $\{c_1, \ldots, c_k\}$, one centroid per cluster.
+It defines each cluster $S_j$ as the set of all records closer to~$c_j$ than
+to any other centroid. Each iteration of the algorithm reduces the WCSS in two steps:
+\begin{Enumerate}
+\item Assign each record to the closest centroid, making $\mean(S_j)\neq c_j$;
+\label{step:kmeans:recluster}
+\item Reset each centroid to its cluster's mean: $c_j := \mean(S_j)$.
+\label{step:kmeans:recenter}
+\end{Enumerate}
+After Step~\ref{step:kmeans:recluster} the centroids are generally different from the cluster
+means, so we can compute another ``within-cluster sum of squares'' based on the centroids:
+\begin{equation}
+\textrm{WCSS\_C}\,\,=\,\, \sum_{i=1}^n \,\big\|x_i - \mathop{\textrm{centroid}}(S_j: x_i\in S_j)\big\|_2^2
+\label{eqn:WCSS:C}
+\end{equation}
+This WCSS\_C after Step~\ref{step:kmeans:recluster} is less than the means-based WCSS
+before Step~\ref{step:kmeans:recluster} (or equal if convergence achieved), and in
+Step~\ref{step:kmeans:recenter} the WCSS cannot exceed the WCSS\_C for \emph{the same}
+clustering; hence the WCSS reduction.
+
+Exact convergence is reached when each record becomes closer to its
+cluster's mean than to any other cluster's mean, so there are no more re-assignments
+and the centroids coincide with the means. In practice, iterations may be stopped
+when the reduction in WCSS (or in WCSS\_C) falls below a minimum threshold, or upon
+reaching the maximum number of iterations. The initialization of the centroids is also
+an important part of the algorithm. The smallest WCSS obtained by the algorithm is not
+the global minimum and varies depending on the initial centroids. We implement multiple
+parallel runs with different initial centroids and report the best result.
+
+\Paragraph{Scoring}
+Our scoring script evaluates the clustering output by comparing it with a known category
+assignment. Since cluster labels have no prior correspondence to the categories, we
+cannot count ``correct'' and ``wrong'' cluster assignments. Instead, we quantify them in
+two ways:
+\begin{Enumerate}
+\item Count how many same-category and different-category pairs of records end up in the
+same cluster or in different clusters;
+\item For each category, count the prevalence of its most common cluster; for each
+cluster, count the prevalence of its most common category.
+\end{Enumerate}
+The number of categories and the number of clusters ($k$) do not have to be equal.
+A same-category pair of records clustered into the same cluster is viewed as a
+``true positive,'' a different-category pair clustered together is a ``false positive,''
+a same-category pair clustered apart is a ``false negative''~etc.
+
+
+\smallskip
+\noindent{\bf Usage: K-means Script}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+{\tt{}-f }path/\/{\tt{}Kmeans.dml}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} C=}path/file
+{\tt{} k=}int
+{\tt{} runs=}int
+{\tt{} maxi=}int
+{\tt{} tol=}double
+{\tt{} samp=}int
+{\tt{} isY=}int
+{\tt{} Y=}path/file
+{\tt{} fmt=}format
+{\tt{} verb=}int
+
+}
+
+\smallskip
+\noindent{\bf Usage: K-means Scoring/Prediction}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+{\tt{}-f }path/\/{\tt{}Kmeans-predict.dml}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} C=}path/file
+{\tt{} spY=}path/file
+{\tt{} prY=}path/file
+{\tt{} fmt=}format
+{\tt{} O=}path/file
+
+}
+
+\smallskip
+\noindent{\bf Arguments}
+\begin{Description}
+\item[{\tt X}:]
+Location to read matrix $X$ with the input data records as rows
+\item[{\tt C}:] (default:\mbox{ }{\tt "C.mtx"})
+Location to store the output matrix with the best available cluster centroids as rows
+\item[{\tt k}:]
+Number of clusters (and centroids)
+\item[{\tt runs}:] (default:\mbox{ }{\tt 10})
+Number of parallel runs, each run with different initial centroids
+\item[{\tt maxi}:] (default:\mbox{ }{\tt 1000})
+Maximum number of iterations per run
+\item[{\tt tol}:] (default:\mbox{ }{\tt 0.000001})
+Tolerance (epsilon) for single-iteration WCSS\_C change ratio
+\item[{\tt samp}:] (default:\mbox{ }{\tt 50})
+Average number of records per centroid in data samples used in the centroid
+initialization procedure
+\item[{\tt Y}:] (default:\mbox{ }{\tt "Y.mtx"})
+Location to store the one-column matrix $Y$ with the best available mapping of
+records to clusters (defined by the output centroids)
+\item[{\tt isY}:] (default:\mbox{ }{\tt 0})
+{\tt 0} = do not write matrix~$Y$, {\tt 1} = write~$Y$
+\item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+see read/write functions in SystemML Language Reference for details.
+\item[{\tt verb}:] (default:\mbox{ }{\tt 0})
+{\tt 0} = do not print per-iteration statistics for each run, {\tt 1} = print them
+(the ``verbose'' option)
+\end{Description}
+\smallskip
+\noindent{\bf Arguments --- Scoring/Prediction}
+\begin{Description}
+\item[{\tt X}:] (default:\mbox{ }{\tt " "})
+Location to read matrix $X$ with the input data records as rows,
+optional when {\tt prY} input is provided
+\item[{\tt C}:] (default:\mbox{ }{\tt " "})
+Location to read matrix $C$ with cluster centroids as rows, optional
+when {\tt prY} input is provided; NOTE: if both {\tt X} and {\tt C} are
+provided, {\tt prY} is an output, not input
+\item[{\tt spY}:] (default:\mbox{ }{\tt " "})
+Location to read a one-column matrix with the externally specified ``true''
+assignment of records (rows) to categories, optional for prediction without
+scoring
+\item[{\tt prY}:] (default:\mbox{ }{\tt " "})
+Location to read (or write, if {\tt X} and {\tt C} are present) a
+column-vector with the predicted assignment of rows to clusters;
+NOTE: No prior correspondence is assumed between the predicted
+cluster labels and the externally specified categories
+\item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+Matrix file output format for {\tt prY}, such as {\tt text}, {\tt mm},
+or {\tt csv}; see read/write functions in SystemML Language Reference
+for details
+\item[{\tt O}:] (default:\mbox{ }{\tt " "})
+Location to write the output statistics defined in
+Table~\ref{table:kmeans:predict:stats}, by default print them to the
+standard output
+\end{Description}
+
+
+\begin{table}[t]\small\centerline{%
+\begin{tabular}{|lcl|}
+\hline
+Name & CID & Meaning \\
+\hline
+{\tt TSS} & & Total Sum of Squares (from the total mean) \\
+{\tt WCSS\_M} & & Within-Cluster Sum of Squares (means as centers) \\
+{\tt WCSS\_M\_PC} & & Within-Cluster Sum of Squares (means), in \% of TSS \\
+{\tt BCSS\_M} & & Between-Cluster Sum of Squares (means as centers) \\
+{\tt BCSS\_M\_PC} & & Between-Cluster Sum of Squares (means), in \% of TSS \\
+\hline
+{\tt WCSS\_C} & & Within-Cluster Sum of Squares (centroids as centers) \\
+{\tt WCSS\_C\_PC} & & Within-Cluster Sum of Squares (centroids), \% of TSS \\
+{\tt BCSS\_C} & & Between-Cluster Sum of Squares (centroids as centers) \\
+{\tt BCSS\_C\_PC} & & Between-Cluster Sum of Squares (centroids), \% of TSS \\
+\hline
+{\tt TRUE\_SAME\_CT} & & Same-category pairs predicted as Same-cluster, count \\
+{\tt TRUE\_SAME\_PC} & & Same-category pairs predicted as Same-cluster, \% \\
+{\tt TRUE\_DIFF\_CT} & & Diff-category pairs predicted as Diff-cluster, count \\
+{\tt TRUE\_DIFF\_PC} & & Diff-category pairs predicted as Diff-cluster, \% \\
+{\tt FALSE\_SAME\_CT} & & Diff-category pairs predicted as Same-cluster, count \\
+{\tt FALSE\_SAME\_PC} & & Diff-category pairs predicted as Same-cluster, \% \\
+{\tt FALSE\_DIFF\_CT} & & Same-category pairs predicted as Diff-cluster, count \\
+{\tt FALSE\_DIFF\_PC} & & Same-category pairs predicted as Diff-cluster, \% \\
+\hline
+{\tt SPEC\_TO\_PRED} & $+$ & For specified category, the best predicted cluster id \\
+{\tt SPEC\_FULL\_CT} & $+$ & For specified category, its full count \\
+{\tt SPEC\_MATCH\_CT} & $+$ & For specified category, best-cluster matching count \\
+{\tt SPEC\_MATCH\_PC} & $+$ & For specified category, \% of matching to full count \\
+{\tt PRED\_TO\_SPEC} & $+$ & For predicted cluster, the best specified category id \\
+{\tt PRED\_FULL\_CT} & $+$ & For predicted cluster, its full count \\
+{\tt PRED\_MATCH\_CT} & $+$ & For predicted cluster, best-category matching count \\
+{\tt PRED\_MATCH\_PC} & $+$ & For predicted cluster, \% of matching to full count \\
+\hline
+\end{tabular}}
+\caption{The {\tt O}-file for {\tt Kmeans-predict} provides the output statistics
+in CSV format, one per line, in the following format: (NAME, [CID], VALUE). Note:
+the 1st group statistics are given if {\tt X} input is available;
+the 2nd group statistics are given if {\tt X} and {\tt C} inputs are available;
+the 3rd and 4th group statistics are given if {\tt spY} input is available;
+only the 4th group statistics contain a nonempty CID value;
+when present, CID contains either the specified category label or the
+predicted cluster label.}
+\label{table:kmeans:predict:stats}
+\end{table}
+
+
+\noindent{\bf Details}
+\smallskip
+
+Our clustering script proceeds in 3~stages: centroid initialization,
+parallel $k$-means iterations, and the best-available output generation.
+Centroids are initialized at random from the input records (the rows of~$X$),
+biased towards being chosen far apart from each other. The initialization
+method is based on the {\tt k-means++} heuristic from~\cite{ArthurVassilvitskii2007:kmeans},
+with one important difference: to reduce the number of passes through~$X$,
+we take a small sample of $X$ and run the {\tt k-means++} heuristic over
+this sample. Here is, conceptually, our centroid initialization algorithm
+for one clustering run:
+\begin{Enumerate}
+\item Sample the rows of~$X$ uniformly at random, picking each row with probability
+$p = ks / n$ where
+\begin{Itemize}
+\item $k$~is the number of centroids,
+\item $n$~is the number of records, and
+\item $s$~is the {\tt samp} input parameter.
+\end{Itemize}
+If $ks \geq n$, the entire $X$ is used in place of its sample.
+\item Choose the first centroid uniformly at random from the sampled rows.
+\item Choose each subsequent centroid from the sampled rows, at random, with
+probability proportional to the squared Euclidean distance between the row and
+the nearest already-chosen centroid.
+\end{Enumerate}
+The sampling of $X$ and the selection of centroids are performed independently
+and in parallel for each run of the $k$-means algorithm. When we sample the
+rows of~$X$, rather than tossing a random coin for each row, we compute the
+number of rows to skip until the next sampled row as $\lceil \log(u) / \log(1 - p) \rceil$
+where $u\in (0, 1)$ is uniformly random. This time-saving trick works because
+\begin{equation*}
+\Prob [k-1 < \log_{1-p}(u) < k] \,\,=\,\, p(1-p)^{k-1} \,\,=\,\,
+\Prob [\textrm{skip $k-1$ rows}]
+\end{equation*}
+However, it requires us to estimate the maximum sample size, which we set
+near~$ks + 10\sqrt{ks}$ to make it generous enough.
+
+Once we selected the initial centroid sets, we start the $k$-means iterations
+independently in parallel for all clustering runs. The number of clustering runs
+is given as the {\tt runs} input parameter. Each iteration of each clustering run
+performs the following steps:
+\begin{Itemize}
+\item Compute the centroid-dependent part of squared Euclidean distances from
+all records (rows of~$X$) to each of the $k$~centroids using matrix product;
+\item Take the minimum of the above for each record;
+\item Update the current within-cluster sum of squares (WCSS) value, with centroids
+substituted instead of the means for efficiency;
+\item Check the convergence criterion:\hfil
+$\textrm{WCSS}_{\mathrm{old}} - \textrm{WCSS}_{\mathrm{new}} < \eps\cdot\textrm{WCSS}_{\mathrm{new}}$\linebreak
+as well as the number of iterations limit;
+\item Find the closest centroid for each record, sharing equally any records with multiple
+closest centroids;
+\item Compute the number of records closest to each centroid, checking for ``runaway''
+centroids with no records left (in which case the run fails);
+\item Compute the new centroids by averaging the records in their clusters.
+\end{Itemize}
+When a termination condition is satisfied, we store the centroids and the WCSS value
+and exit this run. A run has to satisfy the WCSS convergence criterion to be considered
+successful. Upon the termination of all runs, we select the smallest WCSS value among
+the successful runs, and write out this run's centroids. If requested, we also compute
+the cluster assignment of all records in~$X$, using integers from 1 to~$k$ as the cluster
+labels. The scoring script can then be used to compare the cluster assignment with
+an externally specified category assignment.
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+We output the $k$ centroids for the best available clustering, i.~e.\ whose WCSS
+is the smallest of all successful runs.
+The centroids are written as the rows of the $k\,{\times}\,m$-matrix into the output
+file whose path/name was provided as the ``{\tt C}'' input argument. If the input
+parameter ``{\tt isY}'' was set to~{\tt 1}, we also output the one-column matrix with
+the cluster assignment for all the records. This assignment is written into the
+file whose path/name was provided as the ``{\tt Y}'' input argument.
+The best WCSS value, as well as some information about the performance of the other
+runs, is printed during the script execution. The scoring script {\tt Kmeans-predict}
+prints all its results in a self-explanatory manner, as defined in
+Table~\ref{table:kmeans:predict:stats}.
+
+
+\smallskip
+\noindent{\bf Examples}
+\smallskip
+
+{\hangindent=\parindent\noindent\tt
+\hml -f Kmeans.dml -nvargs X=/user/biadmin/X.mtx k=5 C=/user/biadmin/centroids.mtx fmt=csv
+
+}
+
+{\hangindent=\parindent\noindent\tt
+\hml -f Kmeans.dml -nvargs X=/user/biadmin/X.mtx k=5 runs=100 maxi=5000
+tol=0.00000001 samp=20 C=/user/biadmin/centroids.mtx isY=1 Y=/user/biadmin/Yout.mtx verb=1
+
+}
+\noindent To predict {\tt Y} given {\tt X} and {\tt C}:
+
+{\hangindent=\parindent\noindent\tt
+\hml -f Kmeans-predict.dml -nvargs X=/user/biadmin/X.mtx
+ C=/user/biadmin/C.mtx prY=/user/biadmin/PredY.mtx O=/user/biadmin/stats.csv
+
+}
+\noindent To compare ``actual'' labels {\tt spY} with ``predicted'' labels given {\tt X} and {\tt C}:
+
+{\hangindent=\parindent\noindent\tt
+\hml -f Kmeans-predict.dml -nvargs X=/user/biadmin/X.mtx
+ C=/user/biadmin/C.mtx spY=/user/biadmin/Y.mtx O=/user/biadmin/stats.csv
+
+}
+\noindent To compare ``actual'' labels {\tt spY} with given ``predicted'' labels {\tt prY}:
+
+{\hangindent=\parindent\noindent\tt
+\hml -f Kmeans-predict.dml -nvargs spY=/user/biadmin/Y.mtx prY=/user/biadmin/PredY.mtx O=/user/biadmin/stats.csv
+
+}
+
+\smallskip
+\noindent{\bf References}
+\begin{itemize}
+\item
+D.~Aloise, A.~Deshpande, P.~Hansen, and P.~Popat.
+\newblock {NP}-hardness of {E}uclidean sum-of-squares clustering.
+\newblock {\em Machine Learning}, 75(2):245--248, May 2009.
+\item
+D.~Arthur and S.~Vassilvitskii.
+\newblock {\tt k-means++}: The advantages of careful seeding.
+\newblock In {\em Proceedings of the 18th Annual {ACM-SIAM} Symposium on
+ Discrete Algorithms ({SODA}~2007)}, pages 1027--1035, New Orleans~{LA},
+ {USA}, January 7--9 2007.
+\end{itemize}
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/LinReg.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/LinReg.tex b/Algorithms Reference/LinReg.tex
new file mode 100644
index 0000000..ad0dab8
--- /dev/null
+++ b/Algorithms Reference/LinReg.tex
@@ -0,0 +1,306 @@
+\subsection{Linear Regression}
+
+\noindent{\bf Description}
+\smallskip
+
+Linear Regression scripts are used to model the relationship between one numerical
+response variable and one or more explanatory (feature) variables.
+The scripts are given a dataset $(X, Y) = (x_i, y_i)_{i=1}^n$ where $x_i$ is a
+numerical vector of feature variables and $y_i$ is a numerical response value for
+each training data record. The feature vectors are provided as a matrix $X$ of size
+$n\,{\times}\,m$, where $n$ is the number of records and $m$ is the number of features.
+The observed response values are provided as a 1-column matrix~$Y$, with a numerical
+value $y_i$ for each~$x_i$ in the corresponding row of matrix~$X$.
+
+In linear regression, we predict the distribution of the response~$y_i$ based on
+a fixed linear combination of the features in~$x_i$. We assume that
+there exist constant regression coefficients $\beta_0, \beta_1, \ldots, \beta_m$
+and a constant residual variance~$\sigma^2$ such that
+\begin{equation}
+y_i \sim \Normal(\mu_i, \sigma^2) \,\,\,\,\textrm{where}\,\,\,\,
+\mu_i \,=\, \beta_0 + \beta_1 x_{i,1} + \ldots + \beta_m x_{i,m}
+\label{eqn:linregdef}
+\end{equation}
+Distribution $y_i \sim \Normal(\mu_i, \sigma^2)$ models the ``unexplained'' residual
+noise and is assumed independent across different records.
+
+The goal is to estimate the regression coefficients and the residual variance.
+Once they are accurately estimated, we can make predictions about $y_i$ given~$x_i$
+in new records. We can also use the $\beta_j$'s to analyze the influence of individual
+features on the response value, and assess the quality of this model by comparing
+residual variance in the response, left after prediction, with its total variance.
+
+There are two scripts in our library, both doing the same estimation, but using different
+computational methods. Depending on the size and the sparsity of the feature matrix~$X$,
+one or the other script may be more efficient. The ``direct solve'' script
+{\tt LinearRegDS} is more efficient when the number of features $m$ is relatively small
+($m \sim 1000$ or less) and matrix~$X$ is either tall or fairly dense
+(has~${\gg}\:m^2$ nonzeros); otherwise, the ``conjugate gradient'' script {\tt LinearRegCG}
+is more efficient. If $m > 50000$, use only {\tt LinearRegCG}.
+
+\smallskip
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+{\tt{}-f }path/\/{\tt{}LinearRegDS.dml}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} Y=}path/file
+{\tt{} B=}path/file
+{\tt{} O=}path/file
+{\tt{} icpt=}int
+{\tt{} reg=}double
+{\tt{} fmt=}format
+
+}\smallskip
+{\hangindent=\parindent\noindent\it%
+{\tt{}-f }path/\/{\tt{}LinearRegCG.dml}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} Y=}path/file
+{\tt{} B=}path/file
+{\tt{} O=}path/file
+{\tt{} Log=}path/file
+{\tt{} icpt=}int
+{\tt{} reg=}double
+{\tt{} tol=}double
+{\tt{} maxi=}int
+{\tt{} fmt=}format
+
+}
+
+\smallskip
+\noindent{\bf Arguments}
+\begin{Description}
+\item[{\tt X}:]
+Location (on HDFS) to read the matrix of feature vectors, each row constitutes
+one feature vector
+\item[{\tt Y}:]
+Location to read the 1-column matrix of response values
+\item[{\tt B}:]
+Location to store the estimated regression parameters (the $\beta_j$'s), with the
+intercept parameter~$\beta_0$ at position {\tt B[}$m\,{+}\,1$, {\tt 1]} if available
+\item[{\tt O}:] (default:\mbox{ }{\tt " "})
+Location to store the CSV-file of summary statistics defined in
+Table~\ref{table:linreg:stats}, the default is to print it to the standard output
+\item[{\tt Log}:] (default:\mbox{ }{\tt " "}, {\tt LinearRegCG} only)
+Location to store iteration-specific variables for monitoring and debugging purposes,
+see Table~\ref{table:linreg:log} for details.
+\item[{\tt icpt}:] (default:\mbox{ }{\tt 0})
+Intercept presence and shifting/rescaling the features in~$X$:\\
+{\tt 0} = no intercept (hence no~$\beta_0$), no shifting or rescaling of the features;\\
+{\tt 1} = add intercept, but do not shift/rescale the features in~$X$;\\
+{\tt 2} = add intercept, shift/rescale the features in~$X$ to mean~0, variance~1
+\item[{\tt reg}:] (default:\mbox{ }{\tt 0.000001})
+L2-regularization parameter~\mbox{$\lambda\geq 0$}; set to nonzero for highly dependent,
+sparse, or numerous ($m \gtrsim n/10$) features
+\item[{\tt tol}:] (default:\mbox{ }{\tt 0.000001}, {\tt LinearRegCG} only)
+Tolerance \mbox{$\eps\geq 0$} used in the convergence criterion: we terminate conjugate
+gradient iterations when the $\beta$-residual reduces in L2-norm by this factor
+\item[{\tt maxi}:] (default:\mbox{ }{\tt 0}, {\tt LinearRegCG} only)
+Maximum number of conjugate gradient iterations, or~0 if no maximum
+limit provided
+\item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+see read/write functions in SystemML Language Reference for details.
+\end{Description}
+
+
+\begin{table}[t]\small\centerline{%
+\begin{tabular}{|ll|}
+\hline
+Name & Meaning \\
+\hline
+{\tt AVG\_TOT\_Y} & Average of the response value $Y$ \\
+{\tt STDEV\_TOT\_Y} & Standard Deviation of the response value $Y$ \\
+{\tt AVG\_RES\_Y} & Average of the residual $Y - \mathop{\mathrm{pred}}(Y|X)$, i.e.\ residual bias \\
+{\tt STDEV\_RES\_Y} & Standard Deviation of the residual $Y - \mathop{\mathrm{pred}}(Y|X)$ \\
+{\tt DISPERSION} & GLM-style dispersion, i.e.\ residual sum of squares / \#deg.\ fr. \\
+{\tt PLAIN\_R2} & Plain $R^2$ of residual with bias included vs.\ total average \\
+{\tt ADJUSTED\_R2} & Adjusted $R^2$ of residual with bias included vs.\ total average \\
+{\tt PLAIN\_R2\_NOBIAS} & Plain $R^2$ of residual with bias subtracted vs.\ total average \\
+{\tt ADJUSTED\_R2\_NOBIAS} & Adjusted $R^2$ of residual with bias subtracted vs.\ total average \\
+{\tt PLAIN\_R2\_VS\_0} & ${}^*$Plain $R^2$ of residual with bias included vs.\ zero constant \\
+{\tt ADJUSTED\_R2\_VS\_0} & ${}^*$Adjusted $R^2$ of residual with bias included vs.\ zero constant \\
+\hline
+\multicolumn{2}{r}{${}^{*\mathstrut}$ The last two statistics are only printed if there is no intercept ({\tt icpt=0})} \\
+\end{tabular}}
+\caption{Besides~$\beta$, linear regression scripts compute a few summary statistics
+listed above. The statistics are provided in CSV format, one comma-separated name-value
+pair per each line.}
+\label{table:linreg:stats}
+\end{table}
+
+\begin{table}[t]\small\centerline{%
+\begin{tabular}{|ll|}
+\hline
+Name & Meaning \\
+\hline
+{\tt CG\_RESIDUAL\_NORM} & L2-norm of conjug.\ grad.\ residual, which is $A \pxp \beta - t(X) \pxp y$ \\
+ & where $A = t(X) \pxp X + \diag (\lambda)$, or a similar quantity \\
+{\tt CG\_RESIDUAL\_RATIO} & Ratio of current L2-norm of conjug.\ grad.\ residual over the initial \\
+\hline
+\end{tabular}}
+\caption{
+The {\tt Log} file for {\tt{}LinearRegCG} script contains the above \mbox{per-}iteration
+variables in CSV format, each line containing triple (Name, Iteration\#, Value) with
+Iteration\# being~0 for initial values.}
+\label{table:linreg:log}
+\end{table}
+
+
+\noindent{\bf Details}
+\smallskip
+
+To solve a linear regression problem over feature matrix~$X$ and response vector~$Y$,
+we can find coefficients $\beta_0, \beta_1, \ldots, \beta_m$ and $\sigma^2$ that maximize
+the joint likelihood of all $y_i$ for $i=1\ldots n$, defined by the assumed statistical
+model~(\ref{eqn:linregdef}). Since the joint likelihood of the independent
+$y_i \sim \Normal(\mu_i, \sigma^2)$ is proportional to the product of
+$\exp\big({-}\,(y_i - \mu_i)^2 / (2\sigma^2)\big)$, we can take the logarithm of this
+product, then multiply by $-2\sigma^2 < 0$ to obtain a least squares problem:
+\begin{equation}
+\sum_{i=1}^n \, (y_i - \mu_i)^2 \,\,=\,\,
+\sum_{i=1}^n \Big(y_i - \beta_0 - \sum_{j=1}^m \beta_j x_{i,j}\Big)^2
+\,\,\to\,\,\min
+\label{eqn:linregls}
+\end{equation}
+This may not be enough, however. The minimum may sometimes be attained over infinitely many
+$\beta$-vectors, for example if $X$ has an all-0 column, or has linearly dependent columns,
+or has fewer rows than columns~\mbox{($n < m$)}. Even if~(\ref{eqn:linregls}) has a unique
+solution, other $\beta$-vectors may be just a little suboptimal\footnote{Smaller likelihood
+difference between two models suggests less statistical evidence to pick one model over the
+other.}, yet give significantly different predictions for new feature vectors. This results
+in \emph{overfitting}: prediction error for the training data ($X$ and~$Y$) is much smaller
+than for the test data (new records).
+
+Overfitting and degeneracy in the data is commonly mitigated by adding a regularization penalty
+term to the least squares function:
+\begin{equation}
+\sum_{i=1}^n \Big(y_i - \beta_0 - \sum_{j=1}^m \beta_j x_{i,j}\Big)^2
+\,+\,\, \lambda \sum_{j=1}^m \beta_j^2
+\,\,\to\,\,\min
+\label{eqn:linreglsreg}
+\end{equation}
+The choice of $\lambda>0$, the regularization constant, typically involves cross-validation
+where the dataset is repeatedly split into a training part (to estimate the~$\beta_j$'s) and
+a test part (to evaluate prediction accuracy), with the goal of maximizing the test accuracy.
+In our scripts, $\lambda$~is provided as input parameter~{\tt reg}.
+
+The solution to least squares problem~(\ref{eqn:linreglsreg}), through taking the derivative
+and setting it to~0, has the matrix linear equation form
+\begin{equation}
+A\left[\textstyle\beta_{1:m}\atop\textstyle\beta_0\right] \,=\, \big[X,\,1\big]^T Y,\,\,\,
+\textrm{where}\,\,\,
+A \,=\, \big[X,\,1\big]^T \big[X,\,1\big]\,+\,\hspace{0.5pt} \diag(\hspace{0.5pt}
+\underbrace{\raisebox{0pt}[0pt][0.5pt]{$\lambda,\ldots, \lambda$}}_{\raisebox{2pt}{$\scriptstyle m$}}
+\hspace{0.5pt}, 0)
+\label{eqn:linregeq}
+\end{equation}
+where $[X,\,1]$ is $X$~with an extra column of~1s appended on the right, and the
+diagonal matrix of $\lambda$'s has a zero to keep the intercept~$\beta_0$ unregularized.
+If the intercept is disabled by setting {\tt icpt=0}, the equation is simply
+\mbox{$X^T X \beta = X^T Y$}.
+
+We implemented two scripts for solving equation~(\ref{eqn:linregeq}): one is a ``direct solver''
+that computes $A$ and then solves $A\beta = [X,\,1]^T Y$ by calling an external package,
+the other performs linear conjugate gradient~(CG) iterations without ever materializing~$A$.
+The CG~algorithm closely follows Algorithm~5.2 in Chapter~5 of~\cite{Nocedal2006:Optimization}.
+Each step in the CG~algorithm computes a matrix-vector multiplication $q = Ap$ by first computing
+$[X,\,1]\, p$ and then $[X,\,1]^T [X,\,1]\, p$. Usually the number of such multiplications,
+one per CG iteration, is much smaller than~$m$. The user can put a hard bound on it with input
+parameter~{\tt maxi}, or use the default maximum of~$m+1$ (or~$m$ if no intercept) by
+having {\tt maxi=0}. The CG~iterations terminate when the L2-norm of vector
+$r = A\beta - [X,\,1]^T Y$ decreases from its initial value (for~$\beta=0$) by the tolerance
+factor specified in input parameter~{\tt tol}.
+
+The CG algorithm is more efficient if computing
+$[X,\,1]^T \big([X,\,1]\, p\big)$ is much faster than materializing $A$,
+an $(m\,{+}\,1)\times(m\,{+}\,1)$ matrix. The Direct Solver~(DS) is more efficient if
+$X$ takes up a lot more memory than $A$ (i.e.\ $X$~has a lot more nonzeros than~$m^2$)
+and if $m^2$ is small enough for the external solver ($m \lesssim 50000$). A more precise
+determination between CG and~DS is subject to further research.
+
+In addition to the $\beta$-vector, the scripts estimate the residual standard
+deviation~$\sigma$ and the~$R^2$, the ratio of ``explained'' variance to the total
+variance of the response variable. These statistics only make sense if the number
+of degrees of freedom $n\,{-}\,m\,{-}\,1$ is positive and the regularization constant
+$\lambda$ is negligible or zero. The formulas for $\sigma$ and $R^2$~are:
+\begin{equation*}
+R^2_{\textrm{plain}} = 1 - \frac{\mathrm{RSS}}{\mathrm{TSS}},\quad
+\sigma \,=\, \sqrt{\frac{\mathrm{RSS}}{n - m - 1}},\quad
+R^2_{\textrm{adj.}} = 1 - \frac{\sigma^2 (n-1)}{\mathrm{TSS}}
+\end{equation*}
+where
+\begin{equation*}
+\mathrm{RSS} \,=\, \sum_{i=1}^n \Big(y_i - \hat{\mu}_i -
+\frac{1}{n} \sum_{i'=1}^n \,(y_{i'} - \hat{\mu}_{i'})\Big)^2; \quad
+\mathrm{TSS} \,=\, \sum_{i=1}^n \Big(y_i - \frac{1}{n} \sum_{i'=1}^n y_{i'}\Big)^2
+\end{equation*}
+Here $\hat{\mu}_i$ are the predicted means for $y_i$ based on the estimated
+regression coefficients and the feature vectors. They may be biased when no
+intercept is present, hence the RSS formula subtracts the bias.
+
+Lastly, note that by choosing the input option {\tt icpt=2} the user can shift
+and rescale the columns of~$X$ to have zero average and the variance of~1.
+This is particularly important when using regularization over highly disbalanced
+features, because regularization tends to penalize small-variance columns (which
+need large~$\beta_j$'s) more than large-variance columns (with small~$\beta_j$'s).
+At the end, the estimated regression coefficients are shifted and rescaled to
+apply to the original features.
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+The estimated regression coefficients (the $\hat{\beta}_j$'s) are populated into
+a matrix and written to an HDFS file whose path/name was provided as the ``{\tt B}''
+input argument. What this matrix contains, and its size, depends on the input
+argument {\tt icpt}, which specifies the user's intercept and rescaling choice:
+\begin{Description}
+\item[{\tt icpt=0}:] No intercept, matrix~$B$ has size $m\,{\times}\,1$, with
+$B[j, 1] = \hat{\beta}_j$ for each $j$ from 1 to~$m = {}$ncol$(X)$.
+\item[{\tt icpt=1}:] There is intercept, but no shifting/rescaling of~$X$; matrix~$B$
+has size $(m\,{+}\,1) \times 1$, with $B[j, 1] = \hat{\beta}_j$ for $j$ from 1 to~$m$,
+and $B[m\,{+}\,1, 1] = \hat{\beta}_0$, the estimated intercept coefficient.
+\item[{\tt icpt=2}:] There is intercept, and the features in~$X$ are shifted to
+mean${} = 0$ and rescaled to variance${} = 1$; then there are two versions of
+the~$\hat{\beta}_j$'s, one for the original features and another for the
+shifted/rescaled features. Now matrix~$B$ has size $(m\,{+}\,1) \times 2$, with
+$B[\cdot, 1]$ for the original features and $B[\cdot, 2]$ for the shifted/rescaled
+features, in the above format. Note that $B[\cdot, 2]$ are iteratively estimated
+and $B[\cdot, 1]$ are obtained from $B[\cdot, 2]$ by complementary shifting and
+rescaling.
+\end{Description}
+The estimated summary statistics, including residual standard deviation~$\sigma$ and
+the~$R^2$, are printed out or sent into a file (if specified) in CSV format as
+defined in Table~\ref{table:linreg:stats}. For conjugate gradient iterations,
+a log file with monitoring variables can also be made available, see
+Table~\ref{table:linreg:log}.
+
+\smallskip
+\noindent{\bf Examples}
+\smallskip
+
+{\hangindent=\parindent\noindent\tt
+\hml -f LinearRegCG.dml -nvargs X=/user/biadmin/X.mtx Y=/user/biadmin/Y.mtx
+ B=/user/biadmin/B.mtx fmt=csv O=/user/biadmin/stats.csv
+ icpt=2 reg=1.0 tol=0.00000001 maxi=100 Log=/user/biadmin/log.csv
+
+}
+{\hangindent=\parindent\noindent\tt
+\hml -f LinearRegDS.dml -nvargs X=/user/biadmin/X.mtx Y=/user/biadmin/Y.mtx
+ B=/user/biadmin/B.mtx fmt=csv O=/user/biadmin/stats.csv
+ icpt=2 reg=1.0
+
+}
+
+% \smallskip
+% \noindent{\bf See Also}
+% \smallskip
+%
+% In case of binary classification problems, please consider using L2-SVM or
+% binary logistic regression; for multiclass classification, use multiclass~SVM
+% or multinomial logistic regression. For more complex distributions of the
+% response variable use the Generalized Linear Models script.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/LogReg.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/LogReg.tex b/Algorithms Reference/LogReg.tex
new file mode 100644
index 0000000..6693a01
--- /dev/null
+++ b/Algorithms Reference/LogReg.tex
@@ -0,0 +1,266 @@
+\subsection{Multinomial Logistic Regression}
+
+\noindent{\bf Description}
+\smallskip
+
+Our logistic regression script performs both binomial and multinomial logistic regression.
+The script is given a dataset $(X, Y)$ where matrix $X$ has $m$~columns and matrix $Y$ has
+one column; both $X$ and~$Y$ have $n$~rows. The rows of $X$ and~$Y$ are viewed as a collection
+of records: $(X, Y) = (x_i, y_i)_{i=1}^n$ where $x_i$ is a numerical vector of explanatory
+(feature) variables and $y_i$ is a categorical response variable.
+Each row~$x_i$ in~$X$ has size~\mbox{$\dim x_i = m$}, while its corresponding $y_i$ is an
+integer that represents the observed response value for record~$i$.
+
+The goal of logistic regression is to learn a linear model over the feature vector
+$x_i$ that can be used to predict how likely each categorical label is expected to
+be observed as the actual~$y_i$.
+Note that logistic regression predicts more than a label: it predicts the probability
+for every possible label. The binomial case allows only two possible labels, the
+multinomial case has no such restriction.
+
+Just as linear regression estimates the mean value $\mu_i$ of a numerical response
+variable, logistic regression does the same for category label probabilities.
+In linear regression, the mean of $y_i$ is estimated as a linear combination of the features:
+$\mu_i = \beta_0 + \beta_1 x_{i,1} + \ldots + \beta_m x_{i,m} = \beta_0 + x_i\beta_{1:m}$.
+In logistic regression, the
+label probability has to lie between 0 and~1, so a link function is applied to connect
+it to $\beta_0 + x_i\beta_{1:m}$. If there are just two possible category labels, for example
+0~and~1, the logistic link looks as follows:
+\begin{equation*}
+\Prob[y_i\,{=}\,1\mid x_i; \beta] \,=\,
+\frac{e^{\,\beta_0 + x_i\beta_{1:m}}}{1 + e^{\,\beta_0 + x_i\beta_{1:m}}};
+\quad
+\Prob[y_i\,{=}\,0\mid x_i; \beta] \,=\,
+\frac{1}{1 + e^{\,\beta_0 + x_i\beta_{1:m}}}
+\end{equation*}
+Here category label~0 serves as the \emph{baseline}, and function
+$\exp(\beta_0 + x_i\beta_{1:m})$
+shows how likely we expect to see ``$y_i = 1$'' in comparison to the baseline.
+Like in a loaded coin, the predicted odds of seeing 1~versus~0 are
+$\exp(\beta_0 + x_i\beta_{1:m})$ to~1,
+with each feature $x_{i,j}$ multiplying its own factor $\exp(\beta_j x_{i,j})$ to the odds.
+Given a large collection of pairs $(x_i, y_i)$, $i=1\ldots n$, logistic regression seeks
+to find the $\beta_j$'s that maximize the product of probabilities
+\hbox{$\Prob[y_i\mid x_i; \beta]$}
+for actually observed $y_i$-labels (assuming no regularization).
+
+Multinomial logistic regression~\cite{Agresti2002:CDA} extends this link to $k \geq 3$ possible
+categories. Again we identify one category as the baseline, for example the $k$-th category.
+Instead of a coin, here we have a loaded multisided die, one side per category. Each non-baseline
+category $l = 1\ldots k\,{-}\,1$ has its own vector $(\beta_{0,l}, \beta_{1,l}, \ldots, \beta_{m,l})$
+of regression parameters with the intercept, making up a matrix $B$ of size
+$(m\,{+}\,1)\times(k\,{-}\,1)$. The predicted odds of seeing non-baseline category~$l$ versus
+the baseline~$k$ are $\exp\big(\beta_{0,l} + \sum\nolimits_{j=1}^m x_{i,j}\beta_{j,l}\big)$
+to~1, and the predicted probabilities are:
+\begin{align}
+l < k:\quad\Prob[y_i\,{=}\,\makebox[0.5em][c]{$l$}\mid x_i; B] \,\,\,{=}\,\,\,&
+\frac{\exp\big(\beta_{0,l} + \sum\nolimits_{j=1}^m x_{i,j}\beta_{j,l}\big)}%
+{1 \,+\, \sum_{l'=1}^{k-1}\exp\big(\beta_{0,l'} + \sum\nolimits_{j=1}^m x_{i,j}\beta_{j,l'}\big)};
+\label{eqn:mlogreg:nonbaseprob}\\
+\Prob[y_i\,{=}\,\makebox[0.5em][c]{$k$}\mid x_i; B] \,\,\,{=}\,\,\,& \frac{1}%
+{1 \,+\, \sum_{l'=1}^{k-1}\exp\big(\beta_{0,l'} + \sum\nolimits_{j=1}^m x_{i,j}\beta_{j,l'}\big)}.
+\label{eqn:mlogreg:baseprob}
+\end{align}
+The goal of the regression is to estimate the parameter matrix~$B$ from the provided dataset
+$(X, Y) = (x_i, y_i)_{i=1}^n$ by maximizing the product of \hbox{$\Prob[y_i\mid x_i; B]$}
+over the observed labels~$y_i$. Taking its logarithm, negating, and adding a regularization term
+gives us a minimization objective:
+\begin{equation}
+f(B; X, Y) \,\,=\,\,
+-\sum_{i=1}^n \,\log \Prob[y_i\mid x_i; B] \,+\,
+\frac{\lambda}{2} \sum_{j=1}^m \sum_{l=1}^{k-1} |\beta_{j,l}|^2
+\,\,\to\,\,\min
+\label{eqn:mlogreg:loss}
+\end{equation}
+The optional regularization term is added to mitigate overfitting and degeneracy in the data;
+to reduce bias, the intercepts $\beta_{0,l}$ are not regularized. Once the~$\beta_{j,l}$'s
+are accurately estimated, we can make predictions about the category label~$y$ for a new
+feature vector~$x$ using Eqs.~(\ref{eqn:mlogreg:nonbaseprob}) and~(\ref{eqn:mlogreg:baseprob}).
+
+\smallskip
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+{\tt{}-f }path/\/{\tt{}MultiLogReg.dml}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} Y=}path/file
+{\tt{} B=}path/file
+{\tt{} Log=}path/file
+{\tt{} icpt=}int
+{\tt{} reg=}double
+{\tt{} tol=}double
+{\tt{} moi=}int
+{\tt{} mii=}int
+{\tt{} fmt=}format
+
+}
+
+
+\smallskip
+\noindent{\bf Arguments}
+\begin{Description}
+\item[{\tt X}:]
+Location (on HDFS) to read the input matrix of feature vectors; each row constitutes
+one feature vector.
+\item[{\tt Y}:]
+Location to read the input one-column matrix of category labels that correspond to
+feature vectors in~{\tt X}. Note the following:\\
+-- Each non-baseline category label must be a positive integer.\\
+-- If all labels are positive, the largest represents the baseline category.\\
+-- If non-positive labels such as $-1$ or~$0$ are present, then they represent the (same)
+baseline category and are converted to label $\max(\texttt{Y})\,{+}\,1$.
+\item[{\tt B}:]
+Location to store the matrix of estimated regression parameters (the $\beta_{j, l}$'s),
+with the intercept parameters~$\beta_{0, l}$ at position {\tt B[}$m\,{+}\,1$, $l${\tt ]}
+if available. The size of {\tt B} is $(m\,{+}\,1)\times (k\,{-}\,1)$ with the intercepts
+or $m \times (k\,{-}\,1)$ without the intercepts, one column per non-baseline category
+and one row per feature.
+\item[{\tt Log}:] (default:\mbox{ }{\tt " "})
+Location to store iteration-specific variables for monitoring and debugging purposes,
+see Table~\ref{table:mlogreg:log} for details.
+\item[{\tt icpt}:] (default:\mbox{ }{\tt 0})
+Intercept and shifting/rescaling of the features in~$X$:\\
+{\tt 0} = no intercept (hence no~$\beta_0$), no shifting/rescaling of the features;\\
+{\tt 1} = add intercept, but do not shift/rescale the features in~$X$;\\
+{\tt 2} = add intercept, shift/rescale the features in~$X$ to mean~0, variance~1
+\item[{\tt reg}:] (default:\mbox{ }{\tt 0.0})
+L2-regularization parameter (lambda)
+\item[{\tt tol}:] (default:\mbox{ }{\tt 0.000001})
+Tolerance (epsilon) used in the convergence criterion
+\item[{\tt moi}:] (default:\mbox{ }{\tt 100})
+Maximum number of outer (Fisher scoring) iterations
+\item[{\tt mii}:] (default:\mbox{ }{\tt 0})
+Maximum number of inner (conjugate gradient) iterations, or~0 if no maximum
+limit provided
+\item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+see read/write functions in SystemML Language Reference for details.
+\end{Description}
+
+
+\begin{table}[t]\small\centerline{%
+\begin{tabular}{|ll|}
+\hline
+Name & Meaning \\
+\hline
+{\tt LINEAR\_TERM\_MIN} & The minimum value of $X \pxp B$, used to check for overflows \\
+{\tt LINEAR\_TERM\_MAX} & The maximum value of $X \pxp B$, used to check for overflows \\
+{\tt NUM\_CG\_ITERS} & Number of inner (Conj.\ Gradient) iterations in this outer iteration \\
+{\tt IS\_TRUST\_REACHED} & $1 = {}$trust region boundary was reached, $0 = {}$otherwise \\
+{\tt POINT\_STEP\_NORM} & L2-norm of iteration step from old point (matrix $B$) to new point \\
+{\tt OBJECTIVE} & The loss function we minimize (negative regularized log-likelihood) \\
+{\tt OBJ\_DROP\_REAL} & Reduction in the objective during this iteration, actual value \\
+{\tt OBJ\_DROP\_PRED} & Reduction in the objective predicted by a quadratic approximation \\
+{\tt OBJ\_DROP\_RATIO} & Actual-to-predicted reduction ratio, used to update the trust region \\
+{\tt IS\_POINT\_UPDATED} & $1 = {}$new point accepted; $0 = {}$new point rejected, old point restored \\
+{\tt GRADIENT\_NORM} & L2-norm of the loss function gradient (omitted if point is rejected) \\
+{\tt TRUST\_DELTA} & Updated trust region size, the ``delta'' \\
+\hline
+\end{tabular}}
+\caption{
+The {\tt Log} file for multinomial logistic regression contains the above \mbox{per-}iteration
+variables in CSV format, each line containing triple (Name, Iteration\#, Value) with Iteration\#
+being~0 for initial values.}
+\label{table:mlogreg:log}
+\end{table}
+
+
+\noindent{\bf Details}
+\smallskip
+
+We estimate the logistic regression parameters via L2-regularized negative
+log-likelihood minimization~(\ref{eqn:mlogreg:loss}).
+The optimization method used in the script closely follows the trust region
+Newton method for logistic regression described in~\cite{Lin2008:logistic}.
+For convenience, let us make some changes in notation:
+\begin{Itemize}
+\item Convert the input vector of observed category labels into an indicator matrix $Y$
+of size $n \times k$ such that $Y_{i, l} = 1$ if the $i$-th category label is~$l$ and
+$Y_{i, l} = 0$ otherwise;
+\item Append an extra column of all ones, i.e.\ $(1, 1, \ldots, 1)^T$, as the
+$m\,{+}\,1$-st column to the feature matrix $X$ to represent the intercept;
+\item Append an all-zero column as the $k$-th column to $B$, the matrix of regression
+parameters, to represent the baseline category;
+\item Convert the regularization constant $\lambda$ into matrix $\Lambda$ of the same
+size as $B$, placing 0's into the $m\,{+}\,1$-st row to disable intercept regularization,
+and placing $\lambda$'s everywhere else.
+\end{Itemize}
+Now the ($n\,{\times}\,k$)-matrix of predicted probabilities given
+by (\ref{eqn:mlogreg:nonbaseprob}) and~(\ref{eqn:mlogreg:baseprob})
+and the objective function $f$ in~(\ref{eqn:mlogreg:loss}) have the matrix form
+\begin{align*}
+P \,\,&=\,\, \exp(XB) \,\,/\,\, \big(\exp(XB)\,1_{k\times k}\big)\\
+f \,\,&=\,\, - \,\,{\textstyle\sum} \,\,Y \cdot (X B)\, + \,
+{\textstyle\sum}\,\log\big(\exp(XB)\,1_{k\times 1}\big) \,+ \,
+(1/2)\,\, {\textstyle\sum} \,\,\Lambda \cdot B \cdot B
+\end{align*}
+where operations $\cdot\,$, $/$, $\exp$, and $\log$ are applied cellwise,
+and $\textstyle\sum$ denotes the sum of all cells in a matrix.
+The gradient of~$f$ with respect to~$B$ can be represented as a matrix too:
+\begin{equation*}
+\nabla f \,\,=\,\, X^T (P - Y) \,+\, \Lambda \cdot B
+\end{equation*}
+The Hessian $\mathcal{H}$ of~$f$ is a tensor, but, fortunately, the conjugate
+gradient inner loop of the trust region algorithm in~\cite{Lin2008:logistic}
+does not need to instantiate it. We only need to multiply $\mathcal{H}$ by
+ordinary matrices of the same size as $B$ and $\nabla f$, and this can be done
+in matrix form:
+\begin{equation*}
+\mathcal{H}V \,\,=\,\, X^T \big( Q \,-\, P \cdot (Q\,1_{k\times k}) \big) \,+\,
+\Lambda \cdot V, \,\,\,\,\textrm{where}\,\,\,\,Q \,=\, P \cdot (XV)
+\end{equation*}
+At each Newton iteration (the \emph{outer} iteration) the minimization algorithm
+approximates the difference $\varDelta f(S; B) = f(B + S; X, Y) \,-\, f(B; X, Y)$
+attained in the objective function after a step $B \mapsto B\,{+}\,S$ by a
+second-degree formula
+\begin{equation*}
+\varDelta f(S; B) \,\,\,\approx\,\,\, (1/2)\,\,{\textstyle\sum}\,\,S \cdot \mathcal{H}S
+ \,+\, {\textstyle\sum}\,\,S\cdot \nabla f
+\end{equation*}
+This approximation is then minimized by trust-region conjugate gradient iterations
+(the \emph{inner} iterations) subject to the constraint $\|S\|_2 \leq \delta$.
+The trust region size $\delta$ is initialized as $0.5\sqrt{m}\,/ \max\nolimits_i \|x_i\|_2$
+and updated as described in~\cite{Lin2008:logistic}.
+Users can specify the maximum number of the outer and the inner iterations with
+input parameters {\tt moi} and {\tt mii}, respectively. The iterative minimizer
+terminates successfully if $\|\nabla f\|_2 < \eps\,\|\nabla f_{B=0}\|_2$,
+where $\eps > 0$ is a tolerance supplied by the user via input parameter~{\tt tol}.
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+The estimated regression parameters (the $\hat{\beta}_{j, l}$) are populated into
+a matrix and written to an HDFS file whose path/name was provided as the ``{\tt B}''
+input argument. Only the non-baseline categories ($1\leq l \leq k\,{-}\,1$) have
+their $\hat{\beta}_{j, l}$ in the output; to add the baseline category, just append
+a column of zeros. If {\tt icpt=0} in the input command line, no intercepts are used
+and {\tt B} has size $m\times (k\,{-}\,1)$; otherwise {\tt B} has size
+$(m\,{+}\,1)\times (k\,{-}\,1)$
+and the intercepts are in the $m\,{+}\,1$-st row. If {\tt icpt=2}, then initially
+the feature columns in~$X$ are shifted to mean${} = 0$ and rescaled to variance${} = 1$.
+After the iterations converge, the $\hat{\beta}_{j, l}$'s are rescaled and shifted
+to work with the original features.
+
+
+\smallskip
+\noindent{\bf Examples}
+\smallskip
+
+{\hangindent=\parindent\noindent\tt
+\hml -f MultiLogReg.dml -nvargs X=/user/biadmin/X.mtx
+ Y=/user/biadmin/Y.mtx B=/user/biadmin/B.mtx fmt=csv
+ icpt=2 reg=1.0 tol=0.0001 moi=100 mii=10 Log=/user/biadmin/log.csv
+
+}
+
+
+\smallskip
+\noindent{\bf References}
+\begin{itemize}
+\item A.~Agresti.
+\newblock {\em Categorical Data Analysis}.
+\newblock Wiley Series in Probability and Statistics. Wiley-Interscience, second edition, 2002.
+\end{itemize}
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/MultiSVM.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/MultiSVM.tex b/Algorithms Reference/MultiSVM.tex
new file mode 100644
index 0000000..7c844f7
--- /dev/null
+++ b/Algorithms Reference/MultiSVM.tex
@@ -0,0 +1,153 @@
+\subsubsection{Multi-class Support Vector Machines}
+\label{msvm}
+
+\noindent{\bf Description}
+
+Support Vector Machines are used to model the relationship between a categorical
+dependent variable y and one or more explanatory variables denoted X. This
+implementation supports dependent variables that have domain size greater or
+equal to 2 and hence is not restricted to binary class labels.
+\\
+
+\noindent{\bf Usage}
+
+\begin{tabbing}
+\texttt{-f} \textit{path}/\texttt{m-svm.dml -nvargs}
+\=\texttt{X=}\textit{path}/\textit{file}
+ \texttt{Y=}\textit{path}/\textit{file}
+ \texttt{icpt=}\textit{int}\\
+\>\texttt{tol=}\textit{double}
+ \texttt{reg=}\textit{double}
+ \texttt{maxiter=}\textit{int}
+ \texttt{model=}\textit{path}/\textit{file}\\
+\>\texttt{Log=}\textit{path}/\textit{file}
+ \texttt{fmt=}\textit{csv}$\vert$\textit{text}
+\end{tabbing}
+
+\begin{tabbing}
+\texttt{-f} \textit{path}/\texttt{m-svm-predict.dml -nvargs}
+\=\texttt{X=}\textit{path}/\textit{file}
+ \texttt{Y=}\textit{path}/\textit{file}
+ \texttt{icpt=}\textit{int}
+ \texttt{model=}\textit{path}/\textit{file}\\
+\>\texttt{scores=}\textit{path}/\textit{file}
+ \texttt{accuracy=}\textit{path}/\textit{file}\\
+\>\texttt{confusion=}\textit{path}/\textit{file}
+ \texttt{fmt=}\textit{csv}$\vert$\textit{text}
+\end{tabbing}
+
+\noindent{\bf Arguments}
+
+\begin{itemize}
+\item X: Location (on HDFS) containing the explanatory variables
+in a matrix. Each row constitutes an example.
+\item Y: Location (on HDFS) containing a 1-column matrix specifying
+the categorical dependent variable (label). Labels are assumed to be
+contiguously numbered from 1 $\ldots$ \#classes. Note that, this
+argument is optional for prediction.
+\item icpt (default: {\tt 0}): If set to 1 then a constant bias column
+is added to X.
+\item tol (default: {\tt 0.001}): Procedure terminates early if the reduction
+in objective function value is less than tolerance times the initial objective
+function value.
+\item reg (default: {\tt 1}): Regularization constant. See details to find
+out where lambda appears in the objective function. If one were interested
+in drawing an analogy with C-SVM, then C = 2/lambda. Usually, cross validation
+is employed to determine the optimum value of lambda.
+\item maxiter (default: {\tt 100}): The maximum number of iterations.
+\item model: Location (on HDFS) that contains the learnt weights.
+\item Log: Location (on HDFS) to collect various metrics (e.g., objective
+function value etc.) that depict progress across iterations while training.
+\item fmt (default: {\tt text}): Specifies the output format. Choice of
+comma-separated values (csv) or as a sparse-matrix (text).
+\item scores: Location (on HDFS) to store scores for a held-out test set.
+Note that, this is an optional argument.
+\item accuracy: Location (on HDFS) to store the accuracy computed on a
+held-out test set. Note that, this is an optional argument.
+\item confusion: Location (on HDFS) to store the confusion matrix
+computed using a held-out test set. Note that, this is an optional
+argument.
+\end{itemize}
+
+\noindent{\bf Details}
+
+Support vector machines learn a classification function by solving the
+following optimization problem ($L_2$-SVM):
+\begin{eqnarray*}
+&\textrm{argmin}_w& \frac{\lambda}{2} ||w||_2^2 + \sum_i \xi_i^2\\
+&\textrm{subject to:}& y_i w^{\top} x_i \geq 1 - \xi_i ~ \forall i
+\end{eqnarray*}
+where $x_i$ is an example from the training set with its label given by $y_i$,
+$w$ is the vector of parameters and $\lambda$ is the regularization constant
+specified by the user.
+
+To extend the above formulation (binary class SVM) to the multiclass setting,
+one standard approache is to learn one binary class SVM per class that
+separates data belonging to that class from the rest of the training data
+(one-against-the-rest SVM, see C. Scholkopf, 1995).
+
+To account for the missing bias term, one may augment the data with a column
+of constants which is achieved by setting intercept argument to 1 (C-J Hsieh
+et al, 2008).
+
+This implementation optimizes the primal directly (Chapelle, 2007). It uses
+nonlinear conjugate gradient descent to minimize the objective function
+coupled with choosing step-sizes by performing one-dimensional Newton
+minimization in the direction of the gradient.
+\\
+
+\noindent{\bf Returns}
+
+The learnt weights produced by m-svm.dml are populated into a matrix that
+has as many columns as there are classes in the training data, and written
+to file provided on HDFS (see model in section Arguments). The number of rows
+in this matrix is ncol(X) if intercept was set to 0 during invocation and ncol(X) + 1
+otherwise. The bias terms, if used, are placed in the last row. Depending on what
+arguments are provided during invocation, m-svm-predict.dml may compute one or more
+of scores, accuracy and confusion matrix in the output format specified.
+\\
+
+%%\noindent{\bf See Also}
+%%
+%%In case of binary classification problems, please consider using a binary class classifier
+%%learning algorithm, e.g., binary class $L_2$-SVM (see Section \ref{l2svm}) or logistic regression
+%%(see Section \ref{logreg}). To model the relationship between a scalar dependent variable
+%%y and one or more explanatory variables X, consider Linear Regression instead (see Section
+%%\ref{linreg-solver} or Section \ref{linreg-iterative}).
+%%\\
+%%
+\noindent{\bf Examples}
+\begin{verbatim}
+hadoop jar SystemML.jar -f m-svm.dml -nvargs X=/user/biadmin/X.mtx
+ Y=/user/biadmin/y.mtx
+ icpt=0 tol=0.001
+ reg=1.0 maxiter=100 fmt=csv
+ model=/user/biadmin/weights.csv
+ Log=/user/biadmin/Log.csv
+\end{verbatim}
+
+\begin{verbatim}
+hadoop jar SystemML.jar -f m-svm-predict.dml -nvargs X=/user/biadmin/X.mtx
+ Y=/user/biadmin/y.mtx
+ icpt=0 fmt=csv
+ model=/user/biadmin/weights.csv
+ scores=/user/biadmin/scores.csv
+ accuracy=/user/biadmin/accuracy.csv
+ confusion=/user/biadmin/confusion.csv
+\end{verbatim}
+
+\noindent{\bf References}
+
+\begin{itemize}
+\item W. T. Vetterling and B. P. Flannery. \newblock{\em Conjugate Gradient Methods in Multidimensions in
+Numerical Recipes in C - The Art in Scientific Computing.} \newblock W. H. Press and S. A. Teukolsky
+(eds.), Cambridge University Press, 1992.
+\item J. Nocedal and S. J. Wright. \newblock{\em Numerical Optimization.} \newblock Springer-Verlag, 1999.
+\item C-J Hsieh, K-W Chang, C-J Lin, S. S. Keerthi and S. Sundararajan. \newblock {\em A Dual Coordinate
+Descent Method for Large-scale Linear SVM.} \newblock International Conference of Machine Learning
+(ICML), 2008.
+\item Olivier Chapelle. \newblock{\em Training a Support Vector Machine in the Primal.} \newblock Neural
+Computation, 2007.
+\item B. Scholkopf, C. Burges and V. Vapnik. \newblock{\em Extracting Support Data for a Given Task.} \newblock International Conference on Knowledge Discovery and Data Mining (ICDM), 1995.
+\end{itemize}
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/NaiveBayes.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/NaiveBayes.tex b/Algorithms Reference/NaiveBayes.tex
new file mode 100644
index 0000000..0f99c46
--- /dev/null
+++ b/Algorithms Reference/NaiveBayes.tex
@@ -0,0 +1,134 @@
+\subsection{Naive Bayes}
+\label{naive_bayes}
+
+\noindent{\bf Description}
+
+Naive Bayes is very simple generative model used for classifying data.
+This implementation learns a multinomial naive Bayes classifier which
+is applicable when all features are counts of categorical values.
+\\
+
+\noindent{\bf Usage}
+
+\begin{tabbing}
+\texttt{-f} \textit{path}/\texttt{naive-bayes.dml -nvargs}
+\=\texttt{X=}\textit{path}/\textit{file}
+ \texttt{Y=}\textit{path}/\textit{file}
+ \texttt{laplace=}\textit{double}\\
+\>\texttt{prior=}\textit{path}/\textit{file}
+ \texttt{conditionals=}\textit{path}/\textit{file}\\
+\>\texttt{accuracy=}\textit{path}/\textit{file}
+ \texttt{fmt=}\textit{csv}$\vert$\textit{text}
+\end{tabbing}
+
+\begin{tabbing}
+\texttt{-f} \textit{path}/\texttt{naive-bayes-predict.dml -nvargs}
+\=\texttt{X=}\textit{path}/\textit{file}
+ \texttt{Y=}\textit{path}/\textit{file}
+ \texttt{prior=}\textit{path}/\textit{file}\\
+\>\texttt{conditionals=}\textit{path}/\textit{file}
+ \texttt{fmt=}\textit{csv}$\vert$\textit{text}\\
+\>\texttt{accuracy=}\textit{path}/\textit{file}
+ \texttt{confusion=}\textit{path}/\textit{file}\\
+\>\texttt{probabilities=}\textit{path}/\textit{file}
+\end{tabbing}
+
+\noindent{\bf Arguments}
+
+\begin{itemize}
+\item X: Location (on HDFS) to read the matrix of feature vectors;
+each row constitutes one feature vector.
+\item Y: Location (on HDFS) to read the one-column matrix of (categorical)
+labels that correspond to feature vectors in X. Classes are assumed to be
+contiguously labeled beginning from 1. Note that, this argument is optional
+for prediction.
+\item laplace (default: {\tt 1}): Laplace smoothing specified by the
+user to avoid creation of 0 probabilities.
+\item prior: Location (on HDFS) that contains the class prior probabilites.
+\item conditionals: Location (on HDFS) that contains the class conditional
+feature distributions.
+\item fmt (default: {\tt text}): Specifies the output format. Choice of
+comma-separated values (csv) or as a sparse-matrix (text).
+\item probabilities: Location (on HDFS) to store class membership probabilities
+for a held-out test set. Note that, this is an optional argument.
+\item accuracy: Location (on HDFS) to store the training accuracy during
+learning and testing accuracy from a held-out test set during prediction.
+Note that, this is an optional argument for prediction.
+\item confusion: Location (on HDFS) to store the confusion matrix
+computed using a held-out test set. Note that, this is an optional
+argument.
+\end{itemize}
+
+\noindent{\bf Details}
+
+Naive Bayes is a very simple generative classification model. It posits that
+given the class label, features can be generated independently of each other.
+More precisely, the (multinomial) naive Bayes model uses the following
+equation to estimate the joint probability of a feature vector $x$ belonging
+to class $y$:
+\begin{equation*}
+\text{Prob}(y, x) = \pi_y \prod_{i \in x} \theta_{iy}^{n(i,x)}
+\end{equation*}
+where $\pi_y$ denotes the prior probability of class $y$, $i$ denotes a feature
+present in $x$ with $n(i,x)$ denoting its count and $\theta_{iy}$ denotes the
+class conditional probability of feature $i$ in class $y$. The usual
+constraints hold on $\pi$ and $\theta$:
+\begin{eqnarray*}
+&& \pi_y \geq 0, ~ \sum_{y \in \mathcal{C}} \pi_y = 1\\
+\forall y \in \mathcal{C}: && \theta_{iy} \geq 0, ~ \sum_i \theta_{iy} = 1
+\end{eqnarray*}
+where $\mathcal{C}$ is the set of classes.
+
+Given a fully labeled training dataset, it is possible to learn a naive Bayes
+model using simple counting (group-by aggregates). To compute the class conditional
+probabilities, it is usually advisable to avoid setting $\theta_{iy}$ to 0. One way to
+achieve this is using additive smoothing or Laplace smoothing. Some authors have argued
+that this should in fact be add-one smoothing. This implementation uses add-one smoothing
+by default but lets the user specify her/his own constant, if required.
+
+This implementation is sometimes referred to as \emph{multinomial} naive Bayes. Other
+flavours of naive Bayes are also popular.
+\\
+
+\noindent{\bf Returns}
+
+The learnt model produced by naive-bayes.dml is stored in two separate files.
+The first file stores the class prior (a single-column matrix). The second file
+stores the class conditional probabilities organized into a matrix with as many
+rows as there are class labels and as many columns as there are features.
+Depending on what arguments are provided during invocation, naive-bayes-predict.dml
+may compute one or more of probabilities, accuracy and confusion matrix in the
+output format specified.
+\\
+
+\noindent{\bf Examples}
+
+\begin{verbatim}
+hadoop jar SystemML.jar -f naive-bayes.dml -nvargs
+ X=/user/biadmin/X.mtx
+ Y=/user/biadmin/y.mtx
+ laplace=1 fmt=csv
+ prior=/user/biadmin/prior.csv
+ conditionals=/user/biadmin/conditionals.csv
+ accuracy=/user/biadmin/accuracy.csv
+\end{verbatim}
+
+\begin{verbatim}
+hadoop jar SystemML.jar -f naive-bayes-predict.dml -nvargs
+ X=/user/biadmin/X.mtx
+ Y=/user/biadmin/y.mtx
+ prior=/user/biadmin/prior.csv
+ conditionals=/user/biadmin/conditionals.csv
+ fmt=csv
+ accuracy=/user/biadmin/accuracy.csv
+ probabilities=/user/biadmin/probabilities.csv
+ confusion=/user/biadmin/confusion.csv
+\end{verbatim}
+
+\noindent{\bf References}
+
+\begin{itemize}
+\item S. Russell and P. Norvig. \newblock{\em Artificial Intelligence: A Modern Approach.} Prentice Hall, 2009.
+\item A. McCallum and K. Nigam. \newblock{\em A comparison of event models for naive bayes text classification.}
+\newblock AAAI-98 workshop on learning for text categorization, 1998.
+\end{itemize}
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/PCA.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/PCA.tex b/Algorithms Reference/PCA.tex
new file mode 100644
index 0000000..f85879d
--- /dev/null
+++ b/Algorithms Reference/PCA.tex
@@ -0,0 +1,121 @@
+\subsection{Principle Component Analysis}
+\label{pca}
+
+\noindent{\bf Description}
+
+Principle Component Analysis (PCA) is a simple, non-parametric method to transform the given data set with possibly correlated columns into a set of linearly uncorrelated or orthogonal columns, called {\em principle components}. The principle components are ordered in such a way that the first component accounts for the largest possible variance, followed by remaining principle components in the decreasing order of the amount of variance captured from the data. PCA is often used as a dimensionality reduction technique, where the original data is projected or rotated onto a low-dimensional space with basis vectors defined by top-$K$ (for a given value of $K$) principle components.
+\\
+
+\noindent{\bf Usage}
+
+\begin{tabbing}
+\texttt{-f} \textit{path}/\texttt{PCA.dml -nvargs}
+\=\texttt{INPUT=}\textit{path}/\textit{file}
+ \texttt{K=}\textit{int} \\
+\>\texttt{CENTER=}\textit{0/1}
+ \texttt{SCALE=}\textit{0/1}\\
+\>\texttt{PROJDATA=}\textit{0/1}
+ \texttt{OFMT=}\textit{csv}/\textit{text}\\
+\>\texttt{MODEL=}\textit{path}$\vert$\textit{file}
+ \texttt{OUTPUT=}\textit{path}/\textit{file}
+\end{tabbing}
+
+\noindent{\bf Arguments}
+
+\begin{itemize}
+\item INPUT: Location (on HDFS) to read the input matrix.
+\item K: Indicates dimension of the new vector space constructed from $K$ principle components. It must be a value between $1$ and the number of columns in the input data.
+\item CENTER (default: {\tt 0}): Indicates whether or not to {\em center} input data prior to the computation of principle components.
+\item SCALE (default: {\tt 0}): Indicates whether or not to {\em scale} input data prior to the computation of principle components.
+\item PROJDATA: Indicates whether or not the input data must be projected on to new vector space defined over principle components.
+\item OFMT (default: {\tt csv}): Specifies the output format. Choice of comma-separated values (csv) or as a sparse-matrix (text).
+\item MODEL: Either the location (on HDFS) where the computed model is stored; or the location of an existing model.
+\item OUTPUT: Location (on HDFS) to store the data rotated on to the new vector space.
+\end{itemize}
+
+\noindent{\bf Details}
+
+Principle Component Analysis (PCA) is a non-parametric procedure for orthogonal linear transformation of the input data to a new coordinate system, such that the greatest variance by some projection of the data comes to lie on the first coordinate (called the first principal component), the second greatest variance on the second coordinate, and so on. In other words, PCA first selects a normalized direction in $m$-dimensional space ($m$ is the number of columns in the input data) along which the variance in input data is maximized -- this is referred to as the first principle component. It then repeatedly finds other directions (principle components) in which the variance is maximized. At every step, PCA restricts the search for only those directions that are perpendicular to all previously selected directions. By doing so, PCA aims to reduce the redundancy among input variables. To understand the notion of redundancy, consider an extreme scenario with a data set comprising of two v
ariables, where the first one denotes some quantity expressed in meters, and the other variable represents the same quantity but in inches. Both these variables evidently capture redundant information, and hence one of them can be removed. In a general scenario, keeping solely the linear combination of input variables would both express the data more concisely and reduce the number of variables. This is why PCA is often used as a dimensionality reduction technique.
+
+The specific method to compute such a new coordinate system is as follows -- compute a covariance matrix $C$ that measures the strength of correlation among all pairs of variables in the input data; factorize $C$ according to eigen decomposition to calculate its eigenvalues and eigenvectors; and finally, order eigenvectors in the decreasing order of their corresponding eigenvalue. The computed eigenvectors (also known as {\em loadings}) define the new coordinate system and the square root of eigen values provide the amount of variance in the input data explained by each coordinate or eigenvector.
+\\
+
+%As an example, consider the data in Table~\ref{tab:pca_data}.
+\begin{comment}
+\begin{table}
+\parbox{.35\linewidth}{
+\centering
+\begin{tabular}{cc}
+ \hline
+ x & y \\
+ \hline
+ 2.5 & 2.4 \\
+ 0.5 & 0.7 \\
+ 2.2 & 2.9 \\
+ 1.9 & 2.2 \\
+ 3.1 & 3.0 \\
+ 2.3 & 2.7 \\
+ 2 & 1.6 \\
+ 1 & 1.1 \\
+ 1.5 & 1.6 \\
+ 1.1 & 0.9 \\
+ \hline
+\end{tabular}
+\caption{Input Data}
+\label{tab:pca_data}
+}
+\hfill
+\parbox{.55\linewidth}{
+\centering
+\begin{tabular}{cc}
+ \hline
+ x & y \\
+ \hline
+ .69 & .49 \\
+ -1.31 & -1.21 \\
+ .39 & .99 \\
+ .09 & .29 \\
+ 1.29 & 1.09 \\
+ .49 & .79 \\
+ .19 & -.31 \\
+ -.81 & -.81 \\
+ -.31 & -.31 \\
+ -.71 & -1.01 \\
+ \hline
+\end{tabular}
+\caption{Data after centering and scaling}
+\label{tab:pca_scaled_data}
+}
+\end{table}
+\end{comment}
+
+\noindent{\bf Returns}
+When MODEL is not provided, PCA procedure is applied on INPUT data to generate MODEL as well as the rotated data OUTPUT (if PROJDATA is set to $1$) in the new coordinate system.
+The produced model consists of basis vectors MODEL$/dominant.eigen.vectors$ for the new coordinate system; eigen values MODEL$/dominant.eigen.values$; and the standard deviation MODEL$/dominant.eigen.standard.deviations$ of principle components.
+When MODEL is provided, INPUT data is rotated according to the coordinate system defined by MODEL$/dominant.eigen.vectors$. The resulting data is stored at location OUTPUT.
+\\
+
+\noindent{\bf Examples}
+
+\begin{verbatim}
+hadoop jar SystemML.jar -f PCA.dml -nvargs
+ INPUT=/user/biuser/input.mtx K=10
+ CENTER=1 SCALE=1
+ OFMT=csv PROJDATA=1
+ # location to store model and rotated data
+ OUTPUT=/user/biuser/pca_output/
+\end{verbatim}
+
+\begin{verbatim}
+hadoop jar SystemML.jar -f PCA.dml -nvargs
+ INPUT=/user/biuser/test_input.mtx K=10
+ CENTER=1 SCALE=1
+ OFMT=csv PROJDATA=1
+ # location of an existing model
+ MODEL=/user/biuser/pca_output/
+ # location of rotated data
+ OUTPUT=/user/biuser/test_output.mtx
+\end{verbatim}
+
+
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/SystemML_Algorithms_Reference.bib
----------------------------------------------------------------------
diff --git a/Algorithms Reference/SystemML_Algorithms_Reference.bib b/Algorithms Reference/SystemML_Algorithms_Reference.bib
new file mode 100644
index 0000000..8dce564
--- /dev/null
+++ b/Algorithms Reference/SystemML_Algorithms_Reference.bib
@@ -0,0 +1,138 @@
+
+@article {Lin2008:logistic,
+ author = {Chih-Jen Lin and Ruby C.\ Weng and S.\ Sathiya Keerthi},
+ title = {Trust Region {N}ewton Method for Large-Scale Logistic Regression},
+ journal = {Journal of Machine Learning Research},
+ month = {April},
+ year = {2008},
+ volume = {9},
+ pages = {627--650}
+}
+
+@book {Agresti2002:CDA,
+ author = {Alan Agresti},
+ title = {Categorical Data Analysis},
+ edition = {Second},
+ series = {Wiley Series in Probability and Statistics},
+ publisher = {Wiley-Interscience},
+ year = {2002},
+ pages = {710}
+}
+
+@article {Nelder1972:GLM,
+ author = {John Ashworth Nelder and Robert William Maclagan Wedderburn},
+ title = {Generalized Linear Models},
+ journal = {Journal of the Royal Statistical Society, Series~A (General)},
+ year = {1972},
+ volume = {135},
+ number = {3},
+ pages = {370--384}
+}
+
+@book {McCullagh1989:GLM,
+ author = {Peter McCullagh and John Ashworth Nelder},
+ title = {Generalized Linear Models},
+ edition = {Second},
+ series = {Monographs on Statistics and Applied Probability},
+ number = {37},
+ year = {1989},
+ publisher = {Chapman~\&~Hall/CRC},
+ pages = {532}
+}
+
+@book {Gill2000:GLM,
+ author = {Jeff Gill},
+ title = {Generalized Linear Models: A Unified Approach},
+ series = {Sage University Papers Series on Quantitative Applications in the Social Sciences},
+ number = {07-134},
+ year = {2000},
+ publisher = {Sage Publications},
+ pages = {101}
+}
+
+@inproceedings {AgrawalKSX2002:hippocratic,
+ author = {Rakesh Agrawal and Jerry Kiernan and Ramakrishnan Srikant and Yirong Xu},
+ title = {Hippocratic Databases},
+ booktitle = {Proceedings of the 28-th International Conference on Very Large Data Bases ({VLDB} 2002)},
+ address = {Hong Kong, China},
+ month = {August 20--23},
+ year = {2002},
+ pages = {143--154}
+}
+
+@book {Nocedal2006:Optimization,
+ title = {Numerical Optimization},
+ author = {Jorge Nocedal and Stephen Wright},
+ series = {Springer Series in Operations Research and Financial Engineering},
+ pages = {664},
+ edition = {Second},
+ publisher = {Springer},
+ year = {2006}
+}
+
+@book {Hartigan1975:clustering,
+ author = {John A.\ Hartigan},
+ title = {Clustering Algorithms},
+ publisher = {John Wiley~\&~Sons Inc.},
+ series = {Probability and Mathematical Statistics},
+ month = {April},
+ year = {1975},
+ pages = {365}
+}
+
+@inproceedings {ArthurVassilvitskii2007:kmeans,
+ title = {{\tt k-means++}: The Advantages of Careful Seeding},
+ author = {David Arthur and Sergei Vassilvitskii},
+ booktitle = {Proceedings of the 18th Annual {ACM-SIAM} Symposium on Discrete Algorithms ({SODA}~2007)},
+ month = {January 7--9},
+ year = {2007},
+ address = {New Orleans~{LA}, {USA}},
+ pages = {1027--1035}
+}
+
+@article {AloiseDHP2009:kmeans,
+ author = {Daniel Aloise and Amit Deshpande and Pierre Hansen and Preyas Popat},
+ title = {{NP}-hardness of {E}uclidean Sum-of-squares Clustering},
+ journal = {Machine Learning},
+ publisher = {Kluwer Academic Publishers},
+ volume = {75},
+ number = {2},
+ month = {May},
+ year = {2009},
+ pages = {245--248}
+}
+
+@article {Cochran1954:chisq,
+ author = {William G.\ Cochran},
+ title = {Some Methods for Strengthening the Common $\chi^2$ Tests},
+ journal = {Biometrics},
+ volume = {10},
+ number = {4},
+ month = {December},
+ year = {1954},
+ pages = {417--451}
+}
+
+@article {AcockStavig1979:CramersV,
+ author = {Alan C.\ Acock and Gordon R.\ Stavig},
+ title = {A Measure of Association for Nonparametric Statistics},
+ journal = {Social Forces},
+ publisher = {Oxford University Press},
+ volume = {57},
+ number = {4},
+ month = {June},
+ year = {1979},
+ pages = {1381--1386}
+}
+
+@article {Stevens1946:scales,
+ author = {Stanley Smith Stevens},
+ title = {On the Theory of Scales of Measurement},
+ journal = {Science},
+ month = {June 7},
+ year = {1946},
+ volume = {103},
+ number = {2684},
+ pages = {677--680}
+}
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/SystemML_Algorithms_Reference.pdf
----------------------------------------------------------------------
diff --git a/Algorithms Reference/SystemML_Algorithms_Reference.pdf b/Algorithms Reference/SystemML_Algorithms_Reference.pdf
new file mode 100644
index 0000000..4d4ea6a
Binary files /dev/null and b/Algorithms Reference/SystemML_Algorithms_Reference.pdf differ
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/SystemML_Algorithms_Reference.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/SystemML_Algorithms_Reference.tex b/Algorithms Reference/SystemML_Algorithms_Reference.tex
new file mode 100644
index 0000000..7dccad0
--- /dev/null
+++ b/Algorithms Reference/SystemML_Algorithms_Reference.tex
@@ -0,0 +1,142 @@
+\documentclass[letter]{article}
+\usepackage{graphicx,amsmath,amssymb,amsthm,subfigure,color,url,multirow,rotating,comment}
+%\usepackage{tikz}
+\usepackage[normalem]{ulem}
+\usepackage[np,autolanguage]{numprint}
+
+\usepackage[]{hyperref}
+\hypersetup{
+ unicode=false, % non-Latin characters in Acrobat’s bookmarks
+ pdftoolbar=true, % show Acrobat’s toolbar?
+ pdfmenubar=true, % show Acrobat’s menu?
+ pdffitwindow=true, % window fit to page when opened
+ pdfstartview={FitV}, % fits the width of the page to the window
+ pdftitle={SystemML Algorithms Reference}, % title
+ pdfauthor={SystemML Team}, % author
+ pdfsubject={Documentation}, % subject of the document
+ pdfkeywords={}, % list of keywords
+ pdfnewwindow=true, % links in new window
+ bookmarksnumbered=true, % put section numbers in bookmarks
+ bookmarksopen=true, % open up bookmark tree
+ bookmarksopenlevel=1, % \maxdimen level to which bookmarks are open
+ colorlinks=true, % false: boxed links; true: colored links
+ linkcolor=black, % color of internal links
+ citecolor=blue, % color of links to bibliography
+ filecolor=black, % color of file links
+ urlcolor=black % color of external links
+}
+
+
+\newtheorem{definition}{Definition}
+\newtheorem{example}{Example}
+
+\newcommand{\Paragraph}[1]{\vspace*{1ex} \noindent {\bf #1} \hspace*{1ex}}
+\newenvironment{Itemize}{\vspace{-0.5ex}\begin{itemize}\setlength{\itemsep}{-0.2ex}
+}{\end{itemize}\vspace{-0.5ex}}
+\newenvironment{Enumerate}{\vspace{-0.5ex}\begin{enumerate}\setlength{\itemsep}{-0.2ex}
+}{\end{enumerate}\vspace{-0.5ex}}
+\newenvironment{Description}{\vspace{-0.5ex}\begin{description}\setlength{\itemsep}{-0.2ex}
+}{\end{description}\vspace{-0.5ex}}
+
+
+\newcommand{\SystemML}{\texttt{SystemML} }
+\newcommand{\hml}{\texttt{hadoop jar SystemML.jar} }
+\newcommand{\pxp}{\mathbin{\texttt{\%\textasteriskcentered\%}}}
+\newcommand{\todo}[1]{{{\color{red}TODO: #1}}}
+\newcommand{\Normal}{\ensuremath{\mathop{\mathrm{Normal}}\nolimits}}
+\newcommand{\Prob}{\ensuremath{\mathop{\mathrm{Prob}\hspace{0.5pt}}\nolimits}}
+\newcommand{\E}{\ensuremath{\mathop{\mathrm{E}}\nolimits}}
+\newcommand{\mean}{\ensuremath{\mathop{\mathrm{mean}}\nolimits}}
+\newcommand{\Var}{\ensuremath{\mathop{\mathrm{Var}}\nolimits}}
+\newcommand{\Cov}{\ensuremath{\mathop{\mathrm{Cov}}\nolimits}}
+\newcommand{\stdev}{\ensuremath{\mathop{\mathrm{st.dev}}\nolimits}}
+\newcommand{\atan}{\ensuremath{\mathop{\mathrm{arctan}}\nolimits}}
+\newcommand{\diag}{\ensuremath{\mathop{\mathrm{diag}}\nolimits}}
+\newcommand{\const}{\ensuremath{\mathop{\mathrm{const}}\nolimits}}
+\newcommand{\eps}{\varepsilon}
+
+\sloppy
+
+%%%%%%%%%%%%%%%%%%%%%
+% header
+%%%%%%%%%%%%%%%%%%%%%
+
+\title{\LARGE{{\SystemML Algorithms Reference}}}
+\date{\today}
+
+%%%%%%%%%%%%%%%%%%%%%
+% document start
+%%%%%%%%%%%%%%%%%%%%%
+\begin{document}
+
+%\pagenumbering{roman}
+\maketitle
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\section{Descriptive Statistics}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\input{DescriptiveStats}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\section{Classification}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\input{LogReg}
+
+\subsection{Support Vector Machines}
+
+\input{BinarySVM}
+
+\input{MultiSVM}
+
+\input{NaiveBayes}
+
+%\input{DecisionTrees}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\section{Clustering}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\input{Kmeans}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\section{Regression}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\input{LinReg}
+
+\newpage
+
+\input{GLM}
+
+\newpage
+
+\input{GLMpredict.tex}
+
+\newpage
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\section{Matrix Factorization}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\input{pca}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%{\color{red}\section{Sequence Mining}}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%\section{Matrix Factorization}
+
+%%{\color{red}\subsection{GNMF}}
+
+\bibliographystyle{abbrv}
+
+\bibliography{SystemML_ALgorithms_Reference}
+
+
+%%%%%%%%%%%%%%%%%%%%%
+% document end
+%%%%%%%%%%%%%%%%%%%%%
+\end{document}
+
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Language Reference/PyDML Language Reference.doc
----------------------------------------------------------------------
diff --git a/Language Reference/PyDML Language Reference.doc b/Language Reference/PyDML Language Reference.doc
new file mode 100644
index 0000000..b43b6db
Binary files /dev/null and b/Language Reference/PyDML Language Reference.doc differ
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Language Reference/Python syntax for DML.doc
----------------------------------------------------------------------
diff --git a/Language Reference/Python syntax for DML.doc b/Language Reference/Python syntax for DML.doc
new file mode 100644
index 0000000..ee43a6b
Binary files /dev/null and b/Language Reference/Python syntax for DML.doc differ
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Language Reference/README.txt
----------------------------------------------------------------------
diff --git a/Language Reference/README.txt b/Language Reference/README.txt
new file mode 100644
index 0000000..0f22aa6
--- /dev/null
+++ b/Language Reference/README.txt
@@ -0,0 +1,87 @@
+Usage
+-----
+The machine learning algorithms described in
+$BIGINSIGHTS_HOME/machine-learning/docs/SystemML_Algorithms_Reference.pdf can be invoked
+from the hadoop command line using the described, algorithm-specific parameters.
+
+Generic command line arguments arguments are provided by the help command below.
+
+ hadoop jar SystemML.jar -? or -help
+
+
+Recommended configurations
+--------------------------
+1) JVM Heap Sizes:
+We recommend an equal-sized JVM configuration for clients, mappers, and reducers. For the client
+process this can be done via
+
+ export HADOOP_CLIENT_OPTS="-Xmx2048m -Xms2048m -Xmn256m"
+
+where Xmx specifies the maximum heap size, Xms the initial heap size, and Xmn is size of the young
+generation. For Xmn values of equal or less than 15% of the max heap size, we guarantee the memory budget.
+
+The above option may also be set through BigR setting the "ml.jvm" option, e.g.
+ bigr.set.server.option("jaql.fence.jvm.parameters", "-Xmx2g -Xms2g -Xmn256m")
+
+For mapper or reducer JVM configurations, the following properties can be specified in mapred-site.xml,
+where 'child' refers to both mapper and reducer. If map and reduce are specified individually, they take
+precedence over the generic property.
+
+ <property>
+ <name>mapreduce.child.java.opts</name> <!-- synonym: mapred.child.java.opts -->
+ <value>-Xmx2048m -Xms2048m -Xmn256m</value>
+ </property>
+ <property>
+ <name>mapreduce.map.java.opts</name> <!-- synonym: mapred.map.java.opts -->
+ <value>-Xmx2048m -Xms2048m -Xmn256m</value>
+ </property>
+ <property>
+ <name>mapreduce.reduce.java.opts</name> <!-- synonym: mapred.reduce.java.opts -->
+ <value>-Xmx2048m -Xms2048m -Xmn256m</value>
+ </property>
+
+
+2) CP Memory Limitation:
+There exist size limitations for in-memory matrices. Dense in-memory matrices are limited to 16GB
+independent of their dimension. Sparse in-memory matrices are limited to 2G rows and 2G columns
+but the overall matrix can be larger. These limitations do only apply to in-memory matrices but
+NOT in HDFS or involved in MR computations. Setting HADOOP_CLIENT_OPTS below those limitations
+prevents runtime errors.
+
+3) Transparent Huge Pages (on Red Hat Enterprise Linux 6):
+Hadoop workloads might show very high System CPU utilization if THP is enabled. In case of such
+behavior, we recommend to disable THP with
+
+ echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
+
+4) JVM Reuse:
+Performance benefits from JVM reuse because data sets that fit into the mapper memory budget are
+reused across tasks per slot. However, Hadoop 1.0.3 JVM Reuse is incompatible with security (when
+using the LinuxTaskController). The workaround is to use the DefaultTaskController. SystemML provides
+a configuration property in $BIGINSIGHTS_HOME/machine-learning/SystemML-config.xml to enable JVM reuse
+on a per job level without changing the global cluster configuration.
+
+ <jvmreuse>false</jvmreuse>
+
+5) Number of Reducers:
+The number of reducers can have significant impact on performance. SystemML provides a configuration
+property to set the default number of reducers per job without changing the global cluster configuration.
+In general, we recommend a setting of twice the number of nodes. Smaller numbers create less intermediate
+files, larger numbers increase the degree of parallelism for compute and parallel write. In
+$BIGINSIGHTS_HOME/machine-learning/SystemML-config.xml, set:
+
+ <!-- default number of reduce tasks per MR job, default: 2 x number of nodes -->
+ <numreducers>12</numreducers>
+
+6) SystemML temporary directories:
+SystemML uses temporary directories in two different locations: (1) on local file system for temping from
+the client process, and (2) on HDFS for intermediate results between different MR jobs and between MR jobs
+and in-memory operations. Locations of these directories can be configured in
+$BIGINSIGHTS_HOME/machine-learning/SystemML-config.xml with the following properties
+
+ <!-- local fs tmp working directory-->
+ <localtmpdir>/tmp/systemml</localtmpdir>
+
+ <!-- hdfs tmp working directory-->
+ <scratch>scratch_space</scratch>
+
\ No newline at end of file
[37/47] incubator-systemml git commit: [SYSML-326] MLContext
Programming Guide
Posted by du...@apache.org.
[SYSML-326] MLContext Programming Guide
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/b053ce6b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/b053ce6b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/b053ce6b
Branch: refs/heads/gh-pages
Commit: b053ce6b449b045293c1c2e2ae766e6ca7445ff1
Parents: 53e814f
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Wed Oct 14 18:00:09 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Wed Oct 14 18:00:09 2015 -0700
----------------------------------------------------------------------
_layouts/global.html | 1 +
.../2AZ2AQ12B.tar.gz | Bin 0 -> 7242 bytes
...elin-notebook-systemml-linear-regression.png | Bin 0 -> 158819 bytes
.../zeppelin-notebook.png | Bin 0 -> 152132 bytes
index.md | 1 +
mlcontext-programming-guide.md | 978 +++++++++++++++++++
6 files changed, 980 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b053ce6b/_layouts/global.html
----------------------------------------------------------------------
diff --git a/_layouts/global.html b/_layouts/global.html
index 6d50bb1..06d8f17 100644
--- a/_layouts/global.html
+++ b/_layouts/global.html
@@ -37,6 +37,7 @@
<li><a href="http://www.github.com/SparkTC/systemml">SystemML GitHub README</a></li>
<li><a href="quick-start-guide.html">Quick Start Guide</a></li>
<li><a href="dml-and-pydml-programming-guide.html">DML and PyDML Programming Guide</a></li>
+ <li><a href="mlcontext-programming-guide.html">MLContext Programming Guide</a></li>
<li><a href="algorithms-reference.html">Algorithms Reference</a></li>
<li><a href="dml-language-reference.html">DML Language Reference</a></li>
<li class="divider"></li>
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b053ce6b/files/mlcontext-programming-guide/zeppelin-notebook-linear-regression/2AZ2AQ12B.tar.gz
----------------------------------------------------------------------
diff --git a/files/mlcontext-programming-guide/zeppelin-notebook-linear-regression/2AZ2AQ12B.tar.gz b/files/mlcontext-programming-guide/zeppelin-notebook-linear-regression/2AZ2AQ12B.tar.gz
new file mode 100644
index 0000000..560f455
Binary files /dev/null and b/files/mlcontext-programming-guide/zeppelin-notebook-linear-regression/2AZ2AQ12B.tar.gz differ
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b053ce6b/img/mlcontext-programming-guide/zeppelin-notebook-systemml-linear-regression.png
----------------------------------------------------------------------
diff --git a/img/mlcontext-programming-guide/zeppelin-notebook-systemml-linear-regression.png b/img/mlcontext-programming-guide/zeppelin-notebook-systemml-linear-regression.png
new file mode 100644
index 0000000..bfadbc4
Binary files /dev/null and b/img/mlcontext-programming-guide/zeppelin-notebook-systemml-linear-regression.png differ
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b053ce6b/img/mlcontext-programming-guide/zeppelin-notebook.png
----------------------------------------------------------------------
diff --git a/img/mlcontext-programming-guide/zeppelin-notebook.png b/img/mlcontext-programming-guide/zeppelin-notebook.png
new file mode 100644
index 0000000..0e10cd1
Binary files /dev/null and b/img/mlcontext-programming-guide/zeppelin-notebook.png differ
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b053ce6b/index.md
----------------------------------------------------------------------
diff --git a/index.md b/index.md
index 4851630..21af2b0 100644
--- a/index.md
+++ b/index.md
@@ -17,6 +17,7 @@ For more information about SystemML, please consult the following references:
* [SystemML GitHub README](http://www.github.com/SparkTC/systemml)
* [Quick Start Guide](quick-start-guide.html)
* [DML and PyDML Programming Guide](dml-and-pydml-programming-guide.html)
+* [MLContext Programming Guide](mlcontext-programming-guide.html)
* [Algorithms Reference](algorithms-reference.html)
* [DML (R-like Declarative Machine Learning) Language Reference](dml-language-reference.html)
* PyDML (Python-Like Declarative Machine Learning) Language Reference - **Coming Soon**
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b053ce6b/mlcontext-programming-guide.md
----------------------------------------------------------------------
diff --git a/mlcontext-programming-guide.md b/mlcontext-programming-guide.md
new file mode 100644
index 0000000..5f3155a
--- /dev/null
+++ b/mlcontext-programming-guide.md
@@ -0,0 +1,978 @@
+---
+layout: global
+title: MLContext Programming Guide
+description: MLContext Programming Guide
+---
+
+* This will become a table of contents (this text will be scraped).
+{:toc}
+
+<br/>
+
+
+# Overview
+
+The `MLContext` API offers a programmatic interface for interacting with SystemML from languages
+such as Scala and Java. When interacting with `MLContext` from Spark, `DataFrame`s and `RDD`s can be passed
+to SystemML. These data representations are converted to a
+binary-block data format, allowing for SystemML's optimizations to be performed.
+
+
+# Spark Shell Example
+
+## Start Spark Shell with SystemML
+
+To use SystemML with the Spark Shell, the SystemML jar can be referenced using the Spark Shell's `--jars` option.
+Instructions to build the SystemML jar can be found in the [SystemML GitHub README](http://www.github.com/SparkTC/systemml).
+
+{% highlight bash %}
+./bin/spark-shell --executor-memory 4G --driver-memory 4G --jars system-ml-5.2-SNAPSHOT.jar
+{% endhighlight %}
+
+Here is an example of Spark Shell with SystemML and YARN.
+
+{% highlight bash %}
+./bin/spark-shell --master yarn-client --num-executors 3 --driver-memory 5G --executor-memory 5G --executor-cores 4 --jars system-ml-5.2-SNAPSHOT.jar
+{% endhighlight %}
+
+
+## Create MLContext
+
+An `MLContext` object can be created by passing its constructor a reference to the `SparkContext`.
+
+<div class="codetabs">
+
+<div data-lang="Spark Shell" markdown="1">
+{% highlight scala %}
+scala>import com.ibm.bi.dml.api.MLContext
+import com.ibm.bi.dml.api.MLContext
+
+scala> val ml = new MLContext(sc)
+ml: com.ibm.bi.dml.api.MLContext = com.ibm.bi.dml.api.MLContext@33e38c6b
+{% endhighlight %}
+</div>
+
+<div data-lang="Statements" markdown="1">
+{% highlight scala %}
+import com.ibm.bi.dml.api.MLContext
+val ml = new MLContext(sc)
+{% endhighlight %}
+</div>
+
+</div>
+
+
+## Create DataFrame
+
+For demonstration purposes, we'll create a `DataFrame` consisting of 100,000 rows and 1,000 columns
+of random `double`s.
+
+<div class="codetabs">
+
+<div data-lang="Spark Shell" markdown="1">
+{% highlight scala %}
+scala> import org.apache.spark.sql._
+import org.apache.spark.sql._
+
+scala> import org.apache.spark.sql.types.{StructType,StructField,DoubleType}
+import org.apache.spark.sql.types.{StructType, StructField, DoubleType}
+
+scala> import scala.util.Random
+import scala.util.Random
+
+scala> val numRows = 100000
+numRows: Int = 100000
+
+scala> val numCols = 1000
+numCols: Int = 1000
+
+scala> val data = sc.parallelize(0 to numRows-1).map { _ => Row.fromSeq(Seq.fill(numCols)(Random.nextDouble)) }
+data: org.apache.spark.rdd.RDD[org.apache.spark.sql.Row] = MapPartitionsRDD[1] at map at <console>:33
+
+scala> val schema = StructType((0 to numCols-1).map { i => StructField("C" + i, DoubleType, true) } )
+schema: org.apache.spark.sql.types.StructType = StructType(StructField(C0,DoubleType,true), StructField(C1,DoubleType,true), StructField(C2,DoubleType,true), StructField(C3,DoubleType,true), StructField(C4,DoubleType,true), StructField(C5,DoubleType,true), StructField(C6,DoubleType,true), StructField(C7,DoubleType,true), StructField(C8,DoubleType,true), StructField(C9,DoubleType,true), StructField(C10,DoubleType,true), StructField(C11,DoubleType,true), StructField(C12,DoubleType,true), StructField(C13,DoubleType,true), StructField(C14,DoubleType,true), StructField(C15,DoubleType,true), StructField(C16,DoubleType,true), StructField(C17,DoubleType,true), StructField(C18,DoubleType,true), StructField(C19,DoubleType,true), StructField(C20,DoubleType,true), StructField(C21,DoubleType,true), ...
+
+scala> val df = sqlContext.createDataFrame(data, schema)
+df: org.apache.spark.sql.DataFrame = [C0: double, C1: double, C2: double, C3: double, C4: double, C5: double, C6: double, C7: double, C8: double, C9: double, C10: double, C11: double, C12: double, C13: double, C14: double, C15: double, C16: double, C17: double, C18: double, C19: double, C20: double, C21: double, C22: double, C23: double, C24: double, C25: double, C26: double, C27: double, C28: double, C29: double, C30: double, C31: double, C32: double, C33: double, C34: double, C35: double, C36: double, C37: double, C38: double, C39: double, C40: double, C41: double, C42: double, C43: double, C44: double, C45: double, C46: double, C47: double, C48: double, C49: double, C50: double, C51: double, C52: double, C53: double, C54: double, C55: double, C56: double, C57: double, C58: double, C5...
+
+{% endhighlight %}
+</div>
+
+<div data-lang="Statements" markdown="1">
+{% highlight scala %}
+import org.apache.spark.sql._
+import org.apache.spark.sql.types.{StructType,StructField,DoubleType}
+import scala.util.Random
+val numRows = 100000
+val numCols = 1000
+val data = sc.parallelize(0 to numRows-1).map { _ => Row.fromSeq(Seq.fill(numCols)(Random.nextDouble)) }
+val schema = StructType((0 to numCols-1).map { i => StructField("C" + i, DoubleType, true) } )
+val df = sqlContext.createDataFrame(data, schema)
+{% endhighlight %}
+</div>
+
+</div>
+
+
+## Helper Methods
+
+For convenience, we'll create some helper methods. The SystemML output data is encapsulated in
+an `MLOutput` object. The `getScalar()` method extracts a scalar value from a `DataFrame` returned by
+`MLOutput`. The `getScalarDouble()` method returns such a value as a `Double`, and the
+`getScalarInt()` method returns such a value as an `Int`.
+
+<div class="codetabs">
+
+<div data-lang="Spark Shell" markdown="1">
+{% highlight scala %}
+scala> import com.ibm.bi.dml.api.MLOutput
+import com.ibm.bi.dml.api.MLOutput
+
+scala> def getScalar(outputs: MLOutput, symbol: String): Any =
+ | outputs.getDF(sqlContext, symbol).first()(1)
+getScalar: (outputs: com.ibm.bi.dml.api.MLOutput, symbol: String)Any
+
+scala> def getScalarDouble(outputs: MLOutput, symbol: String): Double =
+ | getScalar(outputs, symbol).asInstanceOf[Double]
+getScalarDouble: (outputs: com.ibm.bi.dml.api.MLOutput, symbol: String)Double
+
+scala> def getScalarInt(outputs: MLOutput, symbol: String): Int =
+ | getScalarDouble(outputs, symbol).toInt
+getScalarInt: (outputs: com.ibm.bi.dml.api.MLOutput, symbol: String)Int
+
+{% endhighlight %}
+</div>
+
+<div data-lang="Statements" markdown="1">
+{% highlight scala %}
+import com.ibm.bi.dml.api.MLOutput
+def getScalar(outputs: MLOutput, symbol: String): Any =
+outputs.getDF(sqlContext, symbol).first()(1)
+def getScalarDouble(outputs: MLOutput, symbol: String): Double =
+getScalar(outputs, symbol).asInstanceOf[Double]
+def getScalarInt(outputs: MLOutput, symbol: String): Int =
+getScalarDouble(outputs, symbol).toInt
+
+{% endhighlight %}
+</div>
+
+</div>
+
+
+## Convert DataFrame to Binary-Block Matrix
+
+SystemML is optimized to operate on a binary-block format for matrix representation. For large
+datasets, conversion from DataFrame to binary-block can require a significant quantity of time.
+Explicit DataFrame to binary-block conversion allows algorithm performance to be measured separately
+from data conversion time.
+
+The SystemML binary-block matrix representation can be thought of as a two-dimensional array of blocks, where each block
+consists of a number of rows and columns. In this example, we specify a matrix consisting
+of blocks of size 1000x1000. The experimental `dataFrameToBinaryBlock()` method of `RDDConverterUtilsExt` is used
+to convert the `DataFrame df` to a SystemML binary-block matrix, which is represented by the datatype
+`JavaPairRDD[MatrixIndexes, MatrixBlock]`.
+
+<div class="codetabs">
+
+<div data-lang="Spark Shell" markdown="1">
+{% highlight scala %}
+scala> import com.ibm.bi.dml.runtime.instructions.spark.utils.{RDDConverterUtilsExt => RDDConverterUtils}
+import com.ibm.bi.dml.runtime.instructions.spark.utils.{RDDConverterUtilsExt=>RDDConverterUtils}
+
+scala> import com.ibm.bi.dml.runtime.matrix.MatrixCharacteristics;
+import com.ibm.bi.dml.runtime.matrix.MatrixCharacteristics
+
+scala> val numRowsPerBlock = 1000
+numRowsPerBlock: Int = 1000
+
+scala> val numColsPerBlock = 1000
+numColsPerBlock: Int = 1000
+
+scala> val mc = new MatrixCharacteristics(numRows, numCols, numRowsPerBlock, numColsPerBlock)
+mc: com.ibm.bi.dml.runtime.matrix.MatrixCharacteristics = [100000 x 1000, nnz=-1, blocks (1000 x 1000)]
+
+scala> val sysMlMatrix = RDDConverterUtils.dataFrameToBinaryBlock(sc, df, mc, false)
+sysMlMatrix: org.apache.spark.api.java.JavaPairRDD[com.ibm.bi.dml.runtime.matrix.data.MatrixIndexes,com.ibm.bi.dml.runtime.matrix.data.MatrixBlock] = org.apache.spark.api.java.JavaPairRDD@2bce3248
+
+{% endhighlight %}
+</div>
+
+<div data-lang="Statements" markdown="1">
+{% highlight scala %}
+import com.ibm.bi.dml.runtime.instructions.spark.utils.{RDDConverterUtilsExt => RDDConverterUtils}
+import com.ibm.bi.dml.runtime.matrix.MatrixCharacteristics;
+val numRowsPerBlock = 1000
+val numColsPerBlock = 1000
+val mc = new MatrixCharacteristics(numRows, numCols, numRowsPerBlock, numColsPerBlock)
+val sysMlMatrix = RDDConverterUtils.dataFrameToBinaryBlock(sc, df, mc, false)
+
+{% endhighlight %}
+</div>
+
+</div>
+
+
+## DML Script
+
+For this example, we will utilize the following DML Script called `shape.dml` that reads in a matrix and outputs the number of rows and the
+number of columns, each represented as a matrix.
+
+{% highlight r %}
+X = read($Xin)
+m = matrix(nrow(X), rows=1, cols=1)
+n = matrix(ncol(X), rows=1, cols=1)
+write(m, $Mout)
+write(n, $Nout)
+{% endhighlight %}
+
+
+## Execute Script
+
+Let's execute our DML script, as shown in the example below. The call to `reset()` of `MLContext` is not necessary here, but this method should
+be called if you need to reset inputs and outputs or if you would like to call `execute()` with a different script.
+
+An example of registering the `DataFrame df` as an input to the `X` variable is shown but commented out. If a DataFrame is registered directly,
+it will implicitly be converted to SystemML's binary-block format. However, since we've already explicitly converted the DataFrame to the
+binary-block fixed variable `systemMlMatrix`, we will register this input to the `X` variable. We register the `m` and `n` variables
+as outputs.
+
+When SystemML is executed via `DMLScript` (such as in Standalone Mode), inputs are supplied as either command-line named arguments
+or positional argument. These inputs are specified in DML scripts by prepending them with a `$`. Values are read from or written
+to files using `read`/`write` (DML) and `load`/`save` (PyDML) statements. When utilizing the `MLContext` API,
+inputs and outputs can be other data representations, such as `DataFrame`s. The input and output data are bound to DML variables.
+The named arguments in the `shape.dml` script do not have default values set for them, so we create a `Map` to map the required named
+arguments to blank `String`s so that the script can pass validation.
+
+The `shape.dml` script is executed by the call to `execute()`, where we supply the `Map` of required named arguments. The
+execution results are returned as the `MLOutput` fixed variable `outputs`. The number of rows is obtained by calling the `getStaticInt()`
+helper method with the `outputs` object and `"m"`. The number of columns is retrieved by calling `getStaticInt()` with
+`outputs` and `"n"`.
+
+<div class="codetabs">
+
+<div data-lang="Spark Shell" markdown="1">
+{% highlight scala %}
+scala> ml.reset()
+
+scala> //ml.registerInput("X", df) // implicit conversion of DataFrame to binary-block
+
+scala> ml.registerInput("X", sysMlMatrix, numRows, numCols)
+
+scala> ml.registerOutput("m")
+
+scala> ml.registerOutput("n")
+
+scala> val nargs = Map("Xin" -> " ", "Mout" -> " ", "Nout" -> " ")
+nargs: scala.collection.immutable.Map[String,String] = Map(Xin -> " ", Mout -> " ", Nout -> " ")
+
+scala> val outputs = ml.execute("shape.dml", nargs)
+15/10/12 16:29:15 WARN : Your hostname, derons-mbp.usca.ibm.com resolves to a loopback/non-reachable address: 127.0.0.1, but we couldn't find any external IP address!
+15/10/12 16:29:15 WARN OptimizerUtils: Auto-disable multi-threaded text read for 'text' and 'csv' due to thread contention on JRE < 1.8 (java.version=1.7.0_80).
+outputs: com.ibm.bi.dml.api.MLOutput = com.ibm.bi.dml.api.MLOutput@4d424743
+
+scala> val m = getScalarInt(outputs, "m")
+m: Int = 100000
+
+scala> val n = getScalarInt(outputs, "n")
+n: Int = 1000
+
+{% endhighlight %}
+</div>
+
+<div data-lang="Statements" markdown="1">
+{% highlight scala %}
+ml.reset()
+//ml.registerInput("X", df) // implicit conversion of DataFrame to binary-block
+ml.registerInput("X", sysMlMatrix, numRows, numCols)
+ml.registerOutput("m")
+ml.registerOutput("n")
+val nargs = Map("Xin" -> " ", "Mout" -> " ", "Nout" -> " ")
+val outputs = ml.execute("shape.dml", nargs)
+val m = getScalarInt(outputs, "m")
+val n = getScalarInt(outputs, "n")
+
+{% endhighlight %}
+</div>
+
+</div>
+
+
+## DML Script as String
+
+The `MLContext` API allows a DML script to be specified
+as a `String`. Here, we specify a DML script as a fixed `String` variable called `minMaxMeanScript`.
+This DML will find the minimum, maximum, and mean value of a matrix.
+
+<div class="codetabs">
+
+<div data-lang="Spark Shell" markdown="1">
+{% highlight scala %}
+scala> val minMaxMeanScript: String =
+ | """
+ | Xin = read(" ")
+ | minOut = matrix(min(Xin), rows=1, cols=1)
+ | maxOut = matrix(max(Xin), rows=1, cols=1)
+ | meanOut = matrix(mean(Xin), rows=1, cols=1)
+ | write(minOut, " ")
+ | write(maxOut, " ")
+ | write(meanOut, " ")
+ | """
+minMaxMeanScript: String =
+"
+Xin = read(" ")
+minOut = matrix(min(Xin), rows=1, cols=1)
+maxOut = matrix(max(Xin), rows=1, cols=1)
+meanOut = matrix(mean(Xin), rows=1, cols=1)
+write(minOut, " ")
+write(maxOut, " ")
+write(meanOut, " ")
+"
+
+{% endhighlight %}
+</div>
+
+<div data-lang="Statements" markdown="1">
+{% highlight scala %}
+val minMaxMeanScript: String =
+"""
+Xin = read(" ")
+minOut = matrix(min(Xin), rows=1, cols=1)
+maxOut = matrix(max(Xin), rows=1, cols=1)
+meanOut = matrix(mean(Xin), rows=1, cols=1)
+write(minOut, " ")
+write(maxOut, " ")
+write(meanOut, " ")
+"""
+
+{% endhighlight %}
+</div>
+
+</div>
+
+## Scala Wrapper for DML
+
+We can create a Scala wrapper for our invocation of the `minMaxMeanScript` DML `String`. The `minMaxMean()` method
+takes a `JavaPairRDD[MatrixIndexes, MatrixBlock]` parameter, which is a SystemML binary-block matrix representation.
+It also takes a `rows` parameter indicating the number of rows in the matrix, a `cols` parameter indicating the number
+of columns in the matrix, and an `MLContext` parameter. The `minMaxMean()` method
+returns a tuple consisting of the minimum value in the matrix, the maximum value in the matrix, and the computed
+mean value of the matrix.
+
+<div class="codetabs">
+
+<div data-lang="Spark Shell" markdown="1">
+{% highlight scala %}
+scala> import com.ibm.bi.dml.runtime.matrix.data.MatrixIndexes
+import com.ibm.bi.dml.runtime.matrix.data.MatrixIndexes
+
+scala> import com.ibm.bi.dml.runtime.matrix.data.MatrixBlock
+import com.ibm.bi.dml.runtime.matrix.data.MatrixBlock
+
+scala> import org.apache.spark.api.java.JavaPairRDD
+import org.apache.spark.api.java.JavaPairRDD
+
+scala> def minMaxMean(mat: JavaPairRDD[MatrixIndexes, MatrixBlock], rows: Int, cols: Int, ml: MLContext): (Double, Double, Double) = {
+ | ml.reset()
+ | ml.registerInput("Xin", mat, rows, cols)
+ | ml.registerOutput("minOut")
+ | ml.registerOutput("maxOut")
+ | ml.registerOutput("meanOut")
+ | val outputs = ml.executeScript(minMaxMeanScript)
+ | val minOut = getScalarDouble(outputs, "minOut")
+ | val maxOut = getScalarDouble(outputs, "maxOut")
+ | val meanOut = getScalarDouble(outputs, "meanOut")
+ | (minOut, maxOut, meanOut)
+ | }
+minMaxMean: (mat: org.apache.spark.api.java.JavaPairRDD[com.ibm.bi.dml.runtime.matrix.data.MatrixIndexes,com.ibm.bi.dml.runtime.matrix.data.MatrixBlock], rows: Int, cols: Int, ml: com.ibm.bi.dml.api.MLContext)(Double, Double, Double)
+
+{% endhighlight %}
+</div>
+
+<div data-lang="Statements" markdown="1">
+{% highlight scala %}
+import com.ibm.bi.dml.runtime.matrix.data.MatrixIndexes
+import com.ibm.bi.dml.runtime.matrix.data.MatrixBlock
+import org.apache.spark.api.java.JavaPairRDD
+def minMaxMean(mat: JavaPairRDD[MatrixIndexes, MatrixBlock], rows: Int, cols: Int, ml: MLContext): (Double, Double, Double) = {
+ml.reset()
+ml.registerInput("Xin", mat, rows, cols)
+ml.registerOutput("minOut")
+ml.registerOutput("maxOut")
+ml.registerOutput("meanOut")
+val outputs = ml.executeScript(minMaxMeanScript)
+val minOut = getScalarDouble(outputs, "minOut")
+val maxOut = getScalarDouble(outputs, "maxOut")
+val meanOut = getScalarDouble(outputs, "meanOut")
+(minOut, maxOut, meanOut)
+}
+
+{% endhighlight %}
+</div>
+
+</div>
+
+
+## Invoking DML via Scala Wrapper
+
+Here, we invoke `minMaxMeanScript` using our `minMaxMean()` Scala wrapper method. It returns a tuple
+consisting of the minimum value in the matrix, the maximum value in the matrix, and the mean value of the matrix.
+
+<div class="codetabs">
+
+<div data-lang="Spark Shell" markdown="1">
+{% highlight scala %}
+scala> val (min, max, mean) = minMaxMean(sysMlMatrix, numRows, numCols, ml)
+15/10/13 14:33:11 WARN OptimizerUtils: Auto-disable multi-threaded text read for 'text' and 'csv' due to thread contention on JRE < 1.8 (java.version=1.7.0_80).
+min: Double = 5.378949397005783E-9
+max: Double = 0.9999999934660398
+mean: Double = 0.499988222338507
+
+{% endhighlight %}
+</div>
+
+<div data-lang="Statements" markdown="1">
+{% highlight scala %}
+val (min, max, mean) = minMaxMean(sysMlMatrix, numRows, numCols, ml)
+
+{% endhighlight %}
+</div>
+
+</div>
+
+
+* * *
+
+# Java Example
+
+Next, let's consider a Java example. The `MLContextExample` class creates an `MLContext` object from a `JavaSparkContext`.
+Next, it reads in a matrix CSV file as a `JavaRDD<String>` object. It registers this as input `X`. It registers
+two outputs, `m` and `n`. A `HashMap` maps the expected command-line arguments of the `shape.dml` script to spaces so that
+it passes validation. The `shape.dml` script is executed, and the number of rows and columns in the matrix are output
+to standard output.
+
+
+{% highlight java %}
+package com.ibm.bi.dml;
+
+import java.util.HashMap;
+
+import org.apache.spark.SparkConf;
+import org.apache.spark.api.java.JavaRDD;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.apache.spark.sql.DataFrame;
+import org.apache.spark.sql.SQLContext;
+
+import com.ibm.bi.dml.api.MLContext;
+import com.ibm.bi.dml.api.MLOutput;
+
+public class MLContextExample {
+
+ public static void main(String[] args) throws Exception {
+
+ SparkConf conf = new SparkConf().setAppName("MLContextExample").setMaster("local");
+ JavaSparkContext sc = new JavaSparkContext(conf);
+ SQLContext sqlContext = new SQLContext(sc);
+ MLContext ml = new MLContext(sc);
+
+ JavaRDD<String> csv = sc.textFile("A.csv");
+ ml.registerInput("X", csv, "csv");
+ ml.registerOutput("m");
+ ml.registerOutput("n");
+ HashMap<String, String> cmdLineArgs = new HashMap<String, String>();
+ cmdLineArgs.put("X", " ");
+ cmdLineArgs.put("m", " ");
+ cmdLineArgs.put("n", " ");
+ MLOutput output = ml.execute("shape.dml", cmdLineArgs);
+ DataFrame mDf = output.getDF(sqlContext, "m");
+ DataFrame nDf = output.getDF(sqlContext, "n");
+ System.out.println("rows:" + mDf.first().getDouble(1));
+ System.out.println("cols:" + nDf.first().getDouble(1));
+ }
+
+}
+
+
+{% endhighlight %}
+
+
+* * *
+
+# Zeppelin Notebook Example - Linear Regression Algorithm
+
+Next, we'll consider an example of a SystemML linear regression algorithm run from Spark through an Apache Zeppelin notebook.
+Instructions to clone and build Zeppelin can be found at the [GitHub Apache Zeppelin](https://github.com/apache/incubator-zeppelin)
+site. This example also will look at the Spark ML linear regression algorithm.
+
+This Zeppelin notebook example can be downloaded [here](files/mlcontext-programming-guide/zeppelin-notebook-linear-regression/2AZ2AQ12B.tar.gz).
+Once downloaded and unzipped, place the folder in the Zeppelin `notebook` directory.
+
+A `conf/zeppelin-env.sh` file is created based on `conf/zeppelin-env.sh.template`. For
+this demonstration, it features `SPARK_HOME`, `SPARK_SUBMIT_OPTIONS`, and `ZEPPELIN_SPARK_USEHIVECONTEXT`
+environment variables:
+
+ export SPARK_HOME=/Users/example/spark-1.5.1-bin-hadoop2.6
+ export SPARK_SUBMIT_OPTIONS="--jars $/Users/example/systemml/system-ml/target/system-ml-5.2-SNAPSHOT.jar"
+ export ZEPPELIN_SPARK_USEHIVECONTEXT=false
+
+Start Zeppelin using the `zeppelin.sh` script:
+
+ bin/zeppelin.sh
+
+After opening Zeppelin in a brower, we see the "SystemML - Linear Regression" note in the list of available
+Zeppelin notes.
+
+
+
+If we go to the "SystemML - Linear Regression" note, we see that the note consists of several cells of code.
+
+
+
+Let's briefly consider these cells.
+
+## Trigger Spark Startup
+
+This cell triggers Spark to initialize by calling the `SparkContext` `sc` object. Information regarding these startup operations can be viewed in the
+console window in which `zeppelin.sh` is running.
+
+**Cell:**
+{% highlight scala %}
+// Trigger Spark Startup
+sc
+{% endhighlight %}
+
+**Output:**
+{% highlight scala %}
+res8: org.apache.spark.SparkContext = org.apache.spark.SparkContext@6ce70bf3
+{% endhighlight %}
+
+
+## Generate Linear Regression Test Data
+
+The Spark `LinearDataGenerator` is used to generate test data for the Spark ML and SystemML linear regression algorithms.
+
+**Cell:**
+{% highlight scala %}
+// Generate data
+import org.apache.spark.mllib.util.LinearDataGenerator
+
+val numRows = 10000
+val numCols = 1000
+val rawData = LinearDataGenerator.generateLinearRDD(sc, numRows, numCols, 1).toDF()
+
+// Repartition into a more parallelism-friendly number of partitions
+val data = rawData.repartition(64).cache()
+{% endhighlight %}
+
+**Output:**
+{% highlight scala %}
+import org.apache.spark.mllib.util.LinearDataGenerator
+numRows: Int = 10000
+numCols: Int = 1000
+rawData: org.apache.spark.sql.DataFrame = [label: double, features: vector]
+data: org.apache.spark.sql.DataFrame = [label: double, features: vector]
+{% endhighlight %}
+
+
+## Train using Spark ML Linear Regression Algorithm for Comparison
+
+For purpose of comparison, we can train a model using the Spark ML linear regression
+algorithm.
+
+**Cell:**
+{% highlight scala %}
+// Spark ML
+import org.apache.spark.ml.regression.LinearRegression
+
+// Model Settings
+val maxIters = 100
+val reg = 0
+val elasticNetParam = 0 // L2 reg
+
+// Fit the model
+val lr = new LinearRegression()
+ .setMaxIter(maxIters)
+ .setRegParam(reg)
+ .setElasticNetParam(elasticNetParam)
+val start = System.currentTimeMillis()
+val model = lr.fit(data)
+val trainingTime = (System.currentTimeMillis() - start).toDouble / 1000.0
+
+// Summarize the model over the training set and gather some metrics
+val trainingSummary = model.summary
+val r2 = trainingSummary.r2
+val iters = trainingSummary.totalIterations
+val trainingTimePerIter = trainingTime / iters
+{% endhighlight %}
+
+**Output:**
+{% highlight scala %}
+import org.apache.spark.ml.regression.LinearRegression
+maxIters: Int = 100
+reg: Int = 0
+elasticNetParam: Int = 0
+lr: org.apache.spark.ml.regression.LinearRegression = linReg_a7f51d676562
+start: Long = 1444672044647
+model: org.apache.spark.ml.regression.LinearRegressionModel = linReg_a7f51d676562
+trainingTime: Double = 12.985
+trainingSummary: org.apache.spark.ml.regression.LinearRegressionTrainingSummary = org.apache.spark.ml.regression.LinearRegressionTrainingSummary@227ba28b
+r2: Double = 0.9677118209276552
+iters: Int = 17
+trainingTimePerIter: Double = 0.7638235294117647
+{% endhighlight %}
+
+
+## Spark ML Linear Regression Summary Statistics
+
+Summary statistics for the Spark ML linear regression algorithm are displayed by this cell.
+
+**Cell:**
+{% highlight scala %}
+// Print statistics
+println(s"R2: ${r2}")
+println(s"Iterations: ${iters}")
+println(s"Training time per iter: ${trainingTimePerIter} seconds")
+{% endhighlight %}
+
+**Output:**
+{% highlight scala %}
+R2: 0.9677118209276552
+Iterations: 17
+Training time per iter: 0.7638235294117647 seconds
+{% endhighlight %}
+
+
+## SystemML Linear Regression Algorithm
+
+The `linearReg` fixed `String` variable is set to
+a linear regression algorithm written in DML, SystemML's Declarative Machine Learning language.
+
+
+
+**Cell:**
+{% highlight scala %}
+// SystemML kernels
+val linearReg =
+"""
+#
+# THIS SCRIPT SOLVES LINEAR REGRESSION USING THE CONJUGATE GRADIENT ALGORITHM
+#
+# INPUT PARAMETERS:
+# --------------------------------------------------------------------------------------------
+# NAME TYPE DEFAULT MEANING
+# --------------------------------------------------------------------------------------------
+# X String --- Matrix X of feature vectors
+# Y String --- 1-column Matrix Y of response values
+# icpt Int 0 Intercept presence, shifting and rescaling the columns of X:
+# 0 = no intercept, no shifting, no rescaling;
+# 1 = add intercept, but neither shift nor rescale X;
+# 2 = add intercept, shift & rescale X columns to mean = 0, variance = 1
+# reg Double 0.000001 Regularization constant (lambda) for L2-regularization; set to nonzero
+# for highly dependend/sparse/numerous features
+# tol Double 0.000001 Tolerance (epsilon); conjugate graduent procedure terminates early if
+# L2 norm of the beta-residual is less than tolerance * its initial norm
+# maxi Int 0 Maximum number of conjugate gradient iterations, 0 = no maximum
+# --------------------------------------------------------------------------------------------
+#
+# OUTPUT:
+# B Estimated regression parameters (the betas) to store
+#
+# Note: Matrix of regression parameters (the betas) and its size depend on icpt input value:
+# OUTPUT SIZE: OUTPUT CONTENTS: HOW TO PREDICT Y FROM X AND B:
+# icpt=0: ncol(X) x 1 Betas for X only Y ~ X %*% B[1:ncol(X), 1], or just X %*% B
+# icpt=1: ncol(X)+1 x 1 Betas for X and intercept Y ~ X %*% B[1:ncol(X), 1] + B[ncol(X)+1, 1]
+# icpt=2: ncol(X)+1 x 2 Col.1: betas for X & intercept Y ~ X %*% B[1:ncol(X), 1] + B[ncol(X)+1, 1]
+# Col.2: betas for shifted/rescaled X and intercept
+#
+
+fileX = "";
+fileY = "";
+fileB = "";
+
+intercept_status = ifdef ($icpt, 0); # $icpt=0;
+tolerance = ifdef ($tol, 0.000001); # $tol=0.000001;
+max_iteration = ifdef ($maxi, 0); # $maxi=0;
+regularization = ifdef ($reg, 0.000001); # $reg=0.000001;
+
+X = read (fileX);
+y = read (fileY);
+
+n = nrow (X);
+m = ncol (X);
+ones_n = matrix (1, rows = n, cols = 1);
+zero_cell = matrix (0, rows = 1, cols = 1);
+
+# Introduce the intercept, shift and rescale the columns of X if needed
+
+m_ext = m;
+if (intercept_status == 1 | intercept_status == 2) # add the intercept column
+{
+ X = append (X, ones_n);
+ m_ext = ncol (X);
+}
+
+scale_lambda = matrix (1, rows = m_ext, cols = 1);
+if (intercept_status == 1 | intercept_status == 2)
+{
+ scale_lambda [m_ext, 1] = 0;
+}
+
+if (intercept_status == 2) # scale-&-shift X columns to mean 0, variance 1
+{ # Important assumption: X [, m_ext] = ones_n
+ avg_X_cols = t(colSums(X)) / n;
+ var_X_cols = (t(colSums (X ^ 2)) - n * (avg_X_cols ^ 2)) / (n - 1);
+ is_unsafe = ppred (var_X_cols, 0.0, "<=");
+ scale_X = 1.0 / sqrt (var_X_cols * (1 - is_unsafe) + is_unsafe);
+ scale_X [m_ext, 1] = 1;
+ shift_X = - avg_X_cols * scale_X;
+ shift_X [m_ext, 1] = 0;
+} else {
+ scale_X = matrix (1, rows = m_ext, cols = 1);
+ shift_X = matrix (0, rows = m_ext, cols = 1);
+}
+
+# Henceforth, if intercept_status == 2, we use "X %*% (SHIFT/SCALE TRANSFORM)"
+# instead of "X". However, in order to preserve the sparsity of X,
+# we apply the transform associatively to some other part of the expression
+# in which it occurs. To avoid materializing a large matrix, we rewrite it:
+#
+# ssX_A = (SHIFT/SCALE TRANSFORM) %*% A --- is rewritten as:
+# ssX_A = diag (scale_X) %*% A;
+# ssX_A [m_ext, ] = ssX_A [m_ext, ] + t(shift_X) %*% A;
+#
+# tssX_A = t(SHIFT/SCALE TRANSFORM) %*% A --- is rewritten as:
+# tssX_A = diag (scale_X) %*% A + shift_X %*% A [m_ext, ];
+
+lambda = scale_lambda * regularization;
+beta_unscaled = matrix (0, rows = m_ext, cols = 1);
+
+if (max_iteration == 0) {
+ max_iteration = m_ext;
+}
+i = 0;
+
+# BEGIN THE CONJUGATE GRADIENT ALGORITHM
+r = - t(X) %*% y;
+
+if (intercept_status == 2) {
+ r = scale_X * r + shift_X %*% r [m_ext, ];
+}
+
+p = - r;
+norm_r2 = sum (r ^ 2);
+norm_r2_initial = norm_r2;
+norm_r2_target = norm_r2_initial * tolerance ^ 2;
+
+while (i < max_iteration & norm_r2 > norm_r2_target)
+{
+ if (intercept_status == 2) {
+ ssX_p = scale_X * p;
+ ssX_p [m_ext, ] = ssX_p [m_ext, ] + t(shift_X) %*% p;
+ } else {
+ ssX_p = p;
+ }
+
+ q = t(X) %*% (X %*% ssX_p);
+
+ if (intercept_status == 2) {
+ q = scale_X * q + shift_X %*% q [m_ext, ];
+ }
+
+ q = q + lambda * p;
+ a = norm_r2 / sum (p * q);
+ beta_unscaled = beta_unscaled + a * p;
+ r = r + a * q;
+ old_norm_r2 = norm_r2;
+ norm_r2 = sum (r ^ 2);
+ p = -r + (norm_r2 / old_norm_r2) * p;
+ i = i + 1;
+}
+# END THE CONJUGATE GRADIENT ALGORITHM
+
+if (intercept_status == 2) {
+ beta = scale_X * beta_unscaled;
+ beta [m_ext, ] = beta [m_ext, ] + t(shift_X) %*% beta_unscaled;
+} else {
+ beta = beta_unscaled;
+}
+
+# Output statistics
+avg_tot = sum (y) / n;
+ss_tot = sum (y ^ 2);
+ss_avg_tot = ss_tot - n * avg_tot ^ 2;
+var_tot = ss_avg_tot / (n - 1);
+y_residual = y - X %*% beta;
+avg_res = sum (y_residual) / n;
+ss_res = sum (y_residual ^ 2);
+ss_avg_res = ss_res - n * avg_res ^ 2;
+
+R2_temp = 1 - ss_res / ss_avg_tot
+R2 = matrix(R2_temp, rows=1, cols=1)
+write(R2, "")
+
+totalIters = matrix(i, rows=1, cols=1)
+write(totalIters, "")
+
+# Prepare the output matrix
+if (intercept_status == 2) {
+ beta_out = append (beta, beta_unscaled);
+} else {
+ beta_out = beta;
+}
+
+write (beta_out, fileB);
+"""
+{% endhighlight %}
+
+**Output:**
+
+None
+
+
+## Helper Methods
+
+This cell contains helper methods to return `Double` and `Int` values from output generated by the `MLContext` API.
+
+**Cell:**
+{% highlight scala %}
+// Helper functions
+import com.ibm.bi.dml.api.MLOutput
+
+def getScalar(outputs: MLOutput, symbol: String): Any =
+ outputs.getDF(sqlContext, symbol).first()(1)
+
+def getScalarDouble(outputs: MLOutput, symbol: String): Double =
+ getScalar(outputs, symbol).asInstanceOf[Double]
+
+def getScalarInt(outputs: MLOutput, symbol: String): Int =
+ getScalarDouble(outputs, symbol).toInt
+{% endhighlight %}
+
+**Output:**
+{% highlight scala %}
+import com.ibm.bi.dml.api.MLOutput
+getScalar: (outputs: com.ibm.bi.dml.api.MLOutput, symbol: String)Any
+getScalarDouble: (outputs: com.ibm.bi.dml.api.MLOutput, symbol: String)Double
+getScalarInt: (outputs: com.ibm.bi.dml.api.MLOutput, symbol: String)Int
+{% endhighlight %}
+
+
+## Convert DataFrame to Binary-Block Format
+
+SystemML uses a binary-block format for matrix data representation. This cell
+explicitly converts the `DataFrame` `data` object to a binary-block `features` matrix
+and single-column `label` matrix, both represented by the
+`JavaPairRDD[MatrixIndexes, MatrixBlock]` datatype.
+
+
+**Cell:**
+{% highlight scala %}
+// Imports
+import com.ibm.bi.dml.api.MLContext
+import com.ibm.bi.dml.runtime.instructions.spark.utils.{RDDConverterUtilsExt => RDDConverterUtils}
+import com.ibm.bi.dml.runtime.matrix.MatrixCharacteristics;
+
+// Create SystemML context
+val ml = new MLContext(sc)
+
+// Convert data to proper format
+val mcX = new MatrixCharacteristics(numRows, numCols, 1000, 1000)
+val mcY = new MatrixCharacteristics(numRows, 1, 1000, 1000)
+val X = RDDConverterUtils.vectorDataFrameToBinaryBlock(sc, data, mcX, false, "features")
+val y = RDDConverterUtils.dataFrameToBinaryBlock(sc, data.select("label"), mcY, false)
+// val y = data.select("label")
+
+// Cache
+val X2 = X.cache()
+val y2 = y.cache()
+val cnt1 = X2.count()
+val cnt2 = y2.count()
+{% endhighlight %}
+
+**Output:**
+{% highlight scala %}
+import com.ibm.bi.dml.api.MLContext
+import com.ibm.bi.dml.runtime.instructions.spark.utils.{RDDConverterUtilsExt=>RDDConverterUtils}
+import com.ibm.bi.dml.runtime.matrix.MatrixCharacteristics
+ml: com.ibm.bi.dml.api.MLContext = com.ibm.bi.dml.api.MLContext@38d59245
+mcX: com.ibm.bi.dml.runtime.matrix.MatrixCharacteristics = [10000 x 1000, nnz=-1, blocks (1000 x 1000)]
+mcY: com.ibm.bi.dml.runtime.matrix.MatrixCharacteristics = [10000 x 1, nnz=-1, blocks (1000 x 1000)]
+X: org.apache.spark.api.java.JavaPairRDD[com.ibm.bi.dml.runtime.matrix.data.MatrixIndexes,com.ibm.bi.dml.runtime.matrix.data.MatrixBlock] = org.apache.spark.api.java.JavaPairRDD@b5a86e3
+y: org.apache.spark.api.java.JavaPairRDD[com.ibm.bi.dml.runtime.matrix.data.MatrixIndexes,com.ibm.bi.dml.runtime.matrix.data.MatrixBlock] = org.apache.spark.api.java.JavaPairRDD@56377665
+X2: org.apache.spark.api.java.JavaPairRDD[com.ibm.bi.dml.runtime.matrix.data.MatrixIndexes,com.ibm.bi.dml.runtime.matrix.data.MatrixBlock] = org.apache.spark.api.java.JavaPairRDD@650f29d2
+y2: org.apache.spark.api.java.JavaPairRDD[com.ibm.bi.dml.runtime.matrix.data.MatrixIndexes,com.ibm.bi.dml.runtime.matrix.data.MatrixBlock] = org.apache.spark.api.java.JavaPairRDD@334857a8
+cnt1: Long = 10
+cnt2: Long = 10
+{% endhighlight %}
+
+
+## Train using SystemML Linear Regression Algorithm
+
+Now, we can train our model using the SystemML linear regression algorithm. We register the features matrix `X` and the label matrix `y` as inputs. We register the `beta_out` matrix,
+`R2`, and `totalIters` as outputs.
+
+**Cell:**
+{% highlight scala %}
+// Register inputs & outputs
+ml.reset()
+ml.registerInput("X", X, numRows, numCols)
+ml.registerInput("y", y, numRows, 1)
+// ml.registerInput("y", y)
+ml.registerOutput("beta_out")
+ml.registerOutput("R2")
+ml.registerOutput("totalIters")
+
+// Run the script
+val start = System.currentTimeMillis()
+val outputs = ml.executeScript(linearReg)
+val trainingTime = (System.currentTimeMillis() - start).toDouble / 1000.0
+
+// Get outputs
+val B = outputs.getDF(sqlContext, "beta_out").sort("ID").drop("ID")
+val r2 = getScalarDouble(outputs, "R2")
+val iters = getScalarInt(outputs, "totalIters")
+val trainingTimePerIter = trainingTime / iters
+{% endhighlight %}
+
+**Output:**
+{% highlight scala %}
+start: Long = 1444672090620
+outputs: com.ibm.bi.dml.api.MLOutput = com.ibm.bi.dml.api.MLOutput@5d2c22d0
+trainingTime: Double = 1.176
+B: org.apache.spark.sql.DataFrame = [C1: double]
+r2: Double = 0.9677079547216473
+iters: Int = 12
+trainingTimePerIter: Double = 0.09799999999999999
+{% endhighlight %}
+
+
+## SystemML Linear Regression Summary Statistics
+
+SystemML linear regression summary statistics are displayed by this cell.
+
+**Cell:**
+{% highlight scala %}
+// Print statistics
+println(s"R2: ${r2}")
+println(s"Iterations: ${iters}")
+println(s"Training time per iter: ${trainingTimePerIter} seconds")
+B.describe().show()
+{% endhighlight %}
+
+**Output:**
+{% highlight scala %}
+R2: 0.9677079547216473
+Iterations: 12
+Training time per iter: 0.2334166666666667 seconds
++-------+-------------------+
+|summary| C1|
++-------+-------------------+
+| count| 1000|
+| mean| 0.0184500840658385|
+| stddev| 0.2764750319432085|
+| min|-0.5426068958986378|
+| max| 0.5225309861616542|
++-------+-------------------+
+{% endhighlight %}
+
+
+
[38/47] incubator-systemml git commit: [SYSML-328] Markdown for
additional algorithms
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2a2520bd/algorithms-matrix-factorization.md
----------------------------------------------------------------------
diff --git a/algorithms-matrix-factorization.md b/algorithms-matrix-factorization.md
index 2fedfce..a46a2cd 100644
--- a/algorithms-matrix-factorization.md
+++ b/algorithms-matrix-factorization.md
@@ -3,7 +3,6 @@ layout: global
title: SystemML Algorithms Reference - Matrix Factorization
displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
---
-
# 5 Matrix Factorization
@@ -27,14 +26,14 @@ top-$K$ (for a given value of $K$) principle components.
### Usage
hadoop jar SystemML.jar -f PCA.dml
- -nvargs INPUT=file
- K=int
- CENTER=int
- SCALE=int
- PROJDATA=int
- OFMT=format
- MODEL=file
- OUTPUT=file
+ -nvargs INPUT=<file>
+ K=<int>
+ CENTER=[int]
+ SCALE=[int]
+ PROJDATA=<int>
+ OFMT=[format]
+ MODEL=<file>
+ OUTPUT=<file>
#### Arguments
@@ -42,22 +41,23 @@ top-$K$ (for a given value of $K$) principle components.
**INPUT**: Location (on HDFS) to read the input matrix.
**K**: Indicates dimension of the new vector space constructed from $K$
- principle components. It must be a value between $1$ and the number
+ principle components. It must be a value between `1` and the number
of columns in the input data.
-**CENTER**: (default: 0) 0 or 1. Indicates whether or not to
+**CENTER**: (default: `0`) `0` or `1`. Indicates whether or not to
*center* input data prior to the computation of
principle components.
-**SCALE**: (default: 0) 0 or 1. Indicates whether or not to
+**SCALE**: (default: `0`) `0` or `1`. Indicates whether or not to
*scale* input data prior to the computation of
principle components.
-**PROJDATA**: 0 or 1. Indicates whether or not the input data must be projected
+**PROJDATA**: `0` or `1`. Indicates whether or not the input data must be projected
on to new vector space defined over principle components.
-**OFMT**: (default: `"csv"`) Specifies the output format.
- Choice of comma-separated values (`csv`) or as a sparse-matrix (`text`).
+**OFMT**: (default: `"csv"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
**MODEL**: Either the location (on HDFS) where the computed model is
stored; or the location of an existing model.
@@ -70,23 +70,23 @@ top-$K$ (for a given value of $K$) principle components.
#### Examples
hadoop jar SystemML.jar -f PCA.dml
- -nvargs INPUT=/user/ml/input.mtx
- K=10
- CENTER=1
- SCALE=1O
- FMT=csv
- PROJDATA=1
- OUTPUT=/user/ml/pca_output/
+ -nvargs INPUT=/user/ml/input.mtx
+ K=10
+ CENTER=1
+ SCALE=1O
+ FMT=csv
+ PROJDATA=1
+ OUTPUT=/user/ml/pca_output/
- hadoop jar SystemML.jar -f PCA.dml
- -nvargs INPUT=/user/ml/test_input.mtx
- K=10
+ hadoop jar SystemML.jar -f PCA.dml
+ -nvargs INPUT=/user/ml/test_input.mtx
+ K=10
CENTER=1
- SCALE=1O
- FMT=csv
+ SCALE=1O
+ FMT=csv
PROJDATA=1
- MODEL=/user/ml/pca_output/
- OUTPUT=/user/ml/test_output.mtx
+ MODEL=/user/ml/pca_output/
+ OUTPUT=/user/ml/test_output.mtx
@@ -141,3 +141,290 @@ stored at location OUTPUT.
* * *
+## 5.2 Matrix Completion via Alternating Minimizations
+
+### Description
+
+Low-rank matrix completion is an effective technique for statistical
+data analysis widely used in the data mining and machine learning
+applications. Matrix completion is a variant of low-rank matrix
+factorization with the goal of recovering a partially observed and
+potentially noisy matrix from a subset of its revealed entries. Perhaps
+the most popular applications in which matrix completion has been
+successfully applied is in the context of collaborative filtering in
+recommender systems. In this setting, the rows in the data matrix
+correspond to users, the columns to items such as movies, and entries to
+feedback provided by users for items. The goal is to predict missing
+entries of the rating matrix. This implementation uses the alternating
+least-squares (ALS) technique for solving large-scale matrix completion
+problems.
+
+
+### Usage
+
+**ALS**:
+
+ hadoop jar SystemML.jar -f ALS.dml
+ -nvargs V=<file>
+ L=<file>
+ R=<file>
+ rank=[int]
+ reg=[L2|wL2]
+ lambda=[double]
+ maxi=[int]
+ check=[boolean]
+ thr=[double]
+ fmt=[format]
+
+**ALS Prediction**:
+
+ hadoop jar SystemML.jar -f ALS_predict.dml
+ -nvargs X=<file>
+ Y=<file>
+ L=<file>
+ R=<file>
+ Vrows=<int>
+ Vcols=<int>
+ fmt=[format]
+
+**ALS Top-K Prediction**:
+
+ hadoop jar SystemML.jar -f ALS_topk_predict.dml
+ -nvargs X=<file>
+ Y=<file>
+ L=<file>
+ R=<file>
+ V=<file>
+ K=[int]
+ fmt=[format]
+
+
+### Arguments - ALS
+
+**V**: Location (on HDFS) to read the input (user-item) matrix $V$ to be
+factorized
+
+**L**: Location (on HDFS) to write the left (user) factor matrix $L$
+
+**R**: Location (on HDFS) to write the right (item) factor matrix $R$
+
+**rank**: (default: `10`) Rank of the factorization
+
+**reg**: (default: `L2`) Regularization:
+
+ * `L2` = `L2` regularization
+ * `wL2` = weighted `L2` regularization
+
+**lambda**: (default: `0.000001`) Regularization parameter
+
+**maxi**: (default: `50`) Maximum number of iterations
+
+**check**: (default: `FALSE`) Check for convergence after every iteration, i.e., updating
+$L$ and $R$ once
+
+**thr**: (default: `0.0001`) Assuming `check=TRUE`, the algorithm stops and
+convergence is declared if the decrease in loss in any two consecutive
+iterations falls below threshold `thr`; if
+`check=FALSE`, parameter `thr` is ignored.
+
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
+
+
+### Arguments - ALS Prediction/Top-K Prediction
+
+**X**: Location (on HDFS) to read the input matrix $X$ with following format:
+
+ * for `ALS_predict.dml`: a 2-column matrix that contains
+ the user-ids (first column) and the item-ids (second column)
+ * for `ALS_topk_predict.dml`: a 1-column matrix that
+ contains the user-ids
+
+**Y**: Location (on HDFS) to write the output of prediction with the following
+format:
+
+ * for `ALS_predict.dml`: a 3-column matrix that contains
+ the user-ids (first column), the item-ids (second column) and the
+ predicted ratings (third column)
+ * for `ALS_topk_predict.dml`: a (`K+1`)-column matrix
+ that contains the user-ids in the first column and the top-K
+ item-ids in the remaining `K` columns will be stored at
+ `Y`. Additionally, a matrix with the same dimensions that
+ contains the corresponding actual top-K ratings will be stored at
+ `Y.ratings`; see below for details
+
+**L**: Location (on HDFS) to read the left (user) factor matrix $L$
+
+**R**: Location (on HDFS) to write the right (item) factor matrix $R$
+
+**V**: Location (on HDFS) to read the user-item matrix $V$
+
+**Vrows**: Number of rows of $V$ (i.e., number of users)
+
+**Vcols**: Number of columns of $V$ (i.e., number of items)
+
+**K**: (default: `5`) Number of top-K items for top-K prediction
+
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
+
+
+### Examples
+
+**ALS**:
+
+ hadoop jar SystemML.jar -f ALS.dml
+ -nvargs V=/user/ml/V
+ L=/user/ml/L
+ R=/user/ml/R
+ rank=10
+ reg="wL"
+ lambda=0.0001
+ maxi=50
+ check=TRUE
+ thr=0.001
+ fmt=csv
+
+
+**ALS Prediction**:
+
+To compute predicted ratings for a given list of users and items:
+
+ hadoop jar SystemML.jar -f ALS_predict.dml
+ -nvargs X=/user/ml/X
+ Y=/user/ml/Y
+ L=/user/ml/L
+ R=/user/ml/R
+ Vrows=100000
+ Vcols=10000
+ fmt=csv
+
+
+**ALS Top-K Prediction**:
+
+To compute top-K items with highest predicted ratings together with the
+predicted ratings for a given list of users:
+
+ hadoop jar SystemML.jar -f ALS_topk_predict.dml
+ -nvargs X=/user/ml/X
+ Y=/user/ml/Y
+ L=/user/ml/L
+ R=/user/ml/R
+ V=/user/ml/V
+ K=10
+ fmt=csv
+
+
+### Details
+
+Given an $m \times n$ input matrix $V$ and a rank parameter
+$r \ll \min{(m,n)}$, low-rank matrix factorization seeks to find an
+$m \times r$ matrix $L$ and an $r \times n$ matrix $R$ such that
+$V \approx LR$, i.e., we aim to approximate $V$ by the low-rank matrix
+$LR$. The quality of the approximation is determined by an
+application-dependent loss function $\mathcal{L}$. We aim at finding the
+loss-minimizing factor matrices, i.e.,
+
+$$
+\begin{equation}
+(L^*, R^*) = \textrm{argmin}_{L,R}{\mathcal{L}(V,L,R)}
+\end{equation}
+$$
+
+In the
+context of collaborative filtering in the recommender systems it is
+often the case that the input matrix $V$ contains several missing
+entries. Such entries are coded with the 0 value and the loss function
+is computed only based on the nonzero entries in $V$, i.e.,
+
+$$%\label{eq:loss}
+ \mathcal{L}=\sum_{(i,j)\in\Omega}l(V_{ij},L_{i*},R_{*j})$$
+
+where
+$$L_{i*}$$ and $$R_{*j}$$, respectively, denote the $i$th row of $L$ and the
+$j$th column of $R$, $$\Omega=\{\omega_1,\dots,\omega_N\}$$ denotes the
+training set containing the observed (nonzero) entries in $V$, and $l$
+is some local loss function.
+
+ALS is an optimization technique that can
+be used to solve quadratic problems. For matrix completion, the
+algorithm repeatedly keeps one of the unknown matrices ($L$ or $R$)
+fixed and optimizes the other one. In particular, ALS alternates between
+recomputing the rows of $L$ in one step and the columns of $R$ in the
+subsequent step. Our implementation of the ALS algorithm supports the
+loss functions summarized in [**Table 16**](algorithms-matrix-factorization.html#table16)
+commonly used for matrix completion [[ZhouWSP08]](algorithms-bibliography.html).
+
+* * *
+
+<a name="table16" />
+**Table 16**: Popular loss functions supported by our ALS implementation; $$N_{i*}$$
+ and $$N_{*j}$$, respectively, denote the number of nonzero entries in
+ row $i$ and column $j$ of $V$.
+
+| Loss | Definition |
+| --------------------- | ---------- |
+| $$\mathcal{L}_\text{Nzsl}$$ | $$\sum_{i,j:V_{ij}\neq 0} (V_{ij} - [LR]_{ij})^2$$
+| $$\mathcal{L}_\text{Nzsl+L2}$$ | $$\mathcal{L}_\text{Nzsl} + \lambda \Bigl( \sum_{ik} L_{ik}^2 + \sum_{kj} R_{kj}^2 \Bigr)$$
+| $$\mathcal{L}_\text{Nzsl+wL2}$$ | $$\mathcal{L}_\text{Nzsl} + \lambda \Bigl(\sum_{ik}N_{i*} L_{ik}^2 + \sum_{kj}N_{*j} R_{kj}^2 \Bigr)$$
+
+
+* * *
+
+
+Note that the matrix completion problem as defined in (1)
+is a non-convex problem for all loss functions from
+[**Table 16**](algorithms-matrix-factorization.html#table16). However, when fixing one of the matrices
+$L$ or $R$, we get a least-squares problem with a globally optimal
+solution. For example, for the case of $$\mathcal{L}_\text{Nzsl+wL2}$$ we
+have the following closed form solutions
+
+$$\begin{aligned}
+ L^\top_{n+1,i*} &\leftarrow (R^{(i)}_n {[R^{(i)}_n]}^\top + \lambda N_2 I)^{-1} R_n V^\top_{i*}, \\
+ R_{n+1,*j} &\leftarrow ({[L^{(j)}_{n+1}]}^\top L^{(j)}_{n+1} + \lambda N_1 I)^{-1} L^\top_{n+1} V_{*j},
+ \end{aligned}
+$$
+
+where $$L_{n+1,i*}$$ (resp. $$R_{n+1,*j}$$) denotes the
+$i$th row of $$L_{n+1}$$ (resp. $j$th column of $$R_{n+1}$$), $\lambda$
+denotes the regularization parameter, $I$ is the identity matrix of
+appropriate dimensionality, $$V_{i*}$$ (resp. $$V_{*j}$$) denotes the
+revealed entries in row $i$ (column $j$), $$R^{(i)}_n$$ (resp.
+$$L^{(j)}_{n+1}$$) refers to the corresponding columns of $R_n$ (rows of
+$$L_{n+1}$$), and $N_1$ (resp. $N_2$) denotes a diagonal matrix that
+contains the number of nonzero entries in row $i$ (column $j$) of $V$.
+
+
+**Prediction.** Based on the factor matrices computed by ALS we provide
+two prediction scripts:
+
+ 1. `ALS_predict.dml` computes the predicted ratings for a given
+list of users and items.
+ 2. `ALS_topk_predict.dml` computes top-K item (where $K$ is
+given as input) with highest predicted ratings together with their
+corresponding ratings for a given list of users.
+
+
+### Returns
+
+We output the factor matrices $L$ and $R$ after the algorithm has
+converged. The algorithm is declared as converged if one of the two
+criteria is meet: (1) the decrease in the value of loss function falls
+below `thr` given as an input parameter (if parameter
+`check=TRUE`), or (2) the maximum number of iterations
+(defined as parameter `maxi`) is reached. Note that for a
+given user $i$ prediction is possible only if user $i$ has rated at
+least one item, i.e., row $i$ in matrix $V$ has at least one nonzero
+entry. In case, some users have not rated any items the corresponding
+factor in $L$ will be all 0s. Similarly if some items have not been
+rated at all the corresponding factors in $R$ will contain only 0s. Our
+prediction scripts output the predicted ratings for a given list of
+users and items as well as the top-K items with highest predicted
+ratings together with the predicted ratings for a given list of users.
+Note that the predictions will only be provided for the users who have
+rated at least one item, i.e., the corresponding rows contain at least
+one nonzero entry.
+
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2a2520bd/algorithms-reference.md
----------------------------------------------------------------------
diff --git a/algorithms-reference.md b/algorithms-reference.md
index 58d4aa0..4685761 100644
--- a/algorithms-reference.md
+++ b/algorithms-reference.md
@@ -17,14 +17,26 @@ displayTitle: SystemML Algorithms Reference
* [Binary-Class Support Vector Machines](algorithms-classification.html#binary-class-support-vector-machines)
* [Multi-Class Support Vector Machines](algorithms-classification.html#multi-class-support-vector-machines)
* [Naive Bayes](algorithms-classification.html#naive-bayes)
+ * [Decision Trees](algorithms-classification.html#decision-trees)
+ * [Random Forests](algorithms-classification.html#random-forests)
* [Clustering](algorithms-clustering.html)
* [K-Means Clustering](algorithms-clustering.html#k-means-clustering)
* [Regression](algorithms-regression.html)
* [Linear Regression](algorithms-regression.html#linear-regression)
+ * [Stepwise Linear Regression](algorithms-regression.html#stepwise-linear-regression)
* [Generalized Linear Models](algorithms-regression.html#generalized-linear-models)
+ * [Stepwise Generalized Linear Regression](algorithms-regression.html#stepwise-generalized-linear-regression)
* [Regression Scoring and Prediction](algorithms-regression.html#regression-scoring-and-prediction)
* [Matrix Factorization](algorithms-matrix-factorization.html)
* [Principle Component Analysis](algorithms-matrix-factorization.html#principle-component-analysis)
+ * [Matrix Completion via Alternating Minimizations](algorithms-matrix-factorization.html#matrix-completion-via-alternating-minimizations)
+
+* [Survival Analysis](algorithms-survival-analysis.html)
+ * [Kaplan-Meier Survival Analysis](algorithms-survival-analysis.html#kaplan-meier-survival-analysis)
+ * [Cox Proportional Hazard Regression Model](algorithms-survival-analysis.html#cox-proportional-hazard-regression-model)
+
+* [Bibliography](algorithms-bibliography.html)
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2a2520bd/algorithms-regression.md
----------------------------------------------------------------------
diff --git a/algorithms-regression.md b/algorithms-regression.md
index 0302a18..f1da3c9 100644
--- a/algorithms-regression.md
+++ b/algorithms-regression.md
@@ -3,7 +3,6 @@ layout: global
title: SystemML Algorithms Reference - Regression
displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
---
-
# 4. Regression
@@ -60,31 +59,31 @@ efficient when the number of features $m$ is relatively small
### Usage
-Linear Regression - Direct Solve
+**Linear Regression - Direct Solve**:
hadoop jar SystemML.jar -f LinearRegDS.dml
- -nvargs X=file
- Y=file
- B=file
- O=file
- icpt=int
- reg=double
- fmt=format
+ -nvargs X=<file>
+ Y=<file>
+ B=<file>
+ O=[file]
+ icpt=[int]
+ reg=[double]
+ fmt=[format]
-Linear Regression - Conjugate Gradient
+**Linear Regression - Conjugate Gradient**:
hadoop jar SystemML.jar -f LinearRegCG.dml
- -nvargs X=file
- Y=file
- B=file
- O=file
- Log=file
- icpt=int
- reg=double
- tol=double
- maxi=int
- fmt=format
+ -nvargs X=<file>
+ Y=<file>
+ B=<file>
+ O=[file]
+ Log=[file]
+ icpt=[int]
+ reg=[double]
+ tol=[double]
+ maxi=[int]
+ fmt=[format]
### Arguments
@@ -107,26 +106,25 @@ iteration-specific variables for monitoring and debugging purposes, see
[**Table 8**](algorithms-regression.html#table8)
for details.
-**icpt**: (default: 0) Intercept presence and shifting/rescaling the features
+**icpt**: (default: `0`) Intercept presence and shifting/rescaling the features
in $X$:
-<ul>
-<li>0 = no intercept (hence no $\beta_0$), no shifting or
-rescaling of the features</li>
-<li>1 = add intercept, but do not shift/rescale the features
-in $X$</li>
-<li>2 = add intercept, shift/rescale the features in $X$ to
-mean 0, variance 1</li>
-</ul>
-
-**reg**: (default: 0.000001) L2-regularization parameter $\lambda\geq 0$; set to nonzero for highly
+
+ * 0 = no intercept (hence no $\beta_0$), no shifting or
+rescaling of the features
+ * 1 = add intercept, but do not shift/rescale the features
+in $X$
+ * 2 = add intercept, shift/rescale the features in $X$ to
+mean 0, variance 1
+
+**reg**: (default: `0.000001`) L2-regularization parameter $\lambda\geq 0$; set to nonzero for highly
dependent, sparse, or numerous ($m \gtrsim n/10$) features
-**tol**: (default: 0.000001, `LinearRegCG.dml` only) Tolerance $\varepsilon\geq 0$ used in the
+**tol**: (default: `0.000001`, `LinearRegCG.dml` only) Tolerance $\varepsilon\geq 0$ used in the
convergence criterion: we terminate conjugate gradient iterations when
the $\beta$-residual reduces in L2-norm by this factor
-**maxi**: (default: 0, `LinearRegCG.dml` only) Maximum number of conjugate
-gradient iterations, or 0 if no maximum limit provided
+**maxi**: (default: `0`, `LinearRegCG.dml` only) Maximum number of conjugate
+gradient iterations, or `0` if no maximum limit provided
**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
`mm`, or `csv`; see read/write functions in
@@ -135,7 +133,7 @@ SystemML Language Reference for details.
### Examples
-Linear Regression - Direct Solve
+**Linear Regression - Direct Solve**:
hadoop jar SystemML.jar -f LinearRegDS.dml
-nvargs X=/user/ml/X.mtx
@@ -147,7 +145,7 @@ Linear Regression - Direct Solve
reg=1.0
-Linear Regression - Conjugate Gradient
+**Linear Regression - Conjugate Gradient**:
hadoop jar SystemML.jar -f LinearRegCG.dml
-nvargs X=/user/ml/X.mtx
@@ -359,7 +357,124 @@ can also be made available, see [**Table 8**](algorithms-regression.html#table
* * *
-## 4.2. Generalized Linear Models
+## 4.2. Stepwise Linear Regression
+
+### Description
+
+Our stepwise linear regression script selects a linear model based on
+the Akaike information criterion (AIC): the model that gives rise to the
+lowest AIC is computed.
+
+
+### Usage
+
+ hadoop jar SystemML.jar -f StepLinearRegDS.dml
+ -nvargs X=<file>
+ Y=<file>
+ B=<file>
+ S=[file]
+ O=[file]
+ icpt=[int]
+ thr=[double]
+ fmt=[format]
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read the matrix of feature vectors, each row
+contains one feature vector.
+
+**Y**: Location (on HDFS) to read the 1-column matrix of response values
+
+**B**: Location (on HDFS) to store the estimated regression parameters (the
+$\beta_j$’s), with the intercept parameter $\beta_0$ at position
+B\[$m\,{+}\,1$, 1\] if available
+
+**S**: (default: `" "`) Location (on HDFS) to store the selected feature-ids in the
+order as computed by the algorithm; by default the selected feature-ids
+are forwarded to the standard output.
+
+**O**: (default: `" "`) Location (on HDFS) to store the CSV-file of summary
+statistics defined in [**Table 7**](algorithms-regression.html#table7); by default the
+summary statistics are forwarded to the standard output.
+
+**icpt**: (default: `0`) Intercept presence and shifting/rescaling the features
+in $X$:
+
+ * 0 = no intercept (hence no $\beta_0$), no shifting or
+rescaling of the features;
+ * 1 = add intercept, but do not shift/rescale the features
+in $X$;
+ * 2 = add intercept, shift/rescale the features in $X$ to
+mean 0, variance 1
+
+**thr**: (default: `0.01`) Threshold to stop the algorithm: if the decrease in the value
+of the AIC falls below `thr` no further features are being
+checked and the algorithm stops.
+
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
+
+
+### Examples
+
+ hadoop jar SystemML.jar -f StepLinearRegDS.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ S=/user/ml/selected.csv
+ O=/user/ml/stats.csv
+ icpt=2
+ thr=0.05
+ fmt=csv
+
+
+### Details
+
+Stepwise linear regression iteratively selects predictive variables in
+an automated procedure. Currently, our implementation supports forward
+selection: starting from an empty model (without any variable) the
+algorithm examines the addition of each variable based on the AIC as a
+model comparison criterion. The AIC is defined as
+
+$$
+\begin{equation}
+AIC = -2 \log{L} + 2 edf,\label{eq:AIC}
+\end{equation}
+$$
+
+where $L$ denotes the
+likelihood of the fitted model and $edf$ is the equivalent degrees of
+freedom, i.e., the number of estimated parameters. This procedure is
+repeated until including no additional variable improves the model by a
+certain threshold specified in the input parameter `thr`.
+
+For fitting a model in each iteration we use the `direct solve` method
+as in the script `LinearRegDS.dml` discussed in
+[Linear Regression](algorithms-regression.html#linear-regression).
+
+
+### Returns
+
+Similar to the outputs from `LinearRegDS.dml` the stepwise
+linear regression script computes the estimated regression coefficients
+and stores them in matrix $B$ on HDFS. The format of matrix $B$ is
+identical to the one produced by the scripts for linear regression (see
+[Linear Regression](algorithms-regression.html#linear-regression)). Additionally, `StepLinearRegDS.dml`
+outputs the variable indices (stored in the 1-column matrix $S$) in the
+order they have been selected by the algorithm, i.e., $i$th entry in
+matrix $S$ corresponds to the variable which improves the AIC the most
+in $i$th iteration. If the model with the lowest AIC includes no
+variables matrix $S$ will be empty (contains one 0). Moreover, the
+estimated summary statistics as defined in [**Table 7**](algorithms-regression.html#table7)
+are printed out or stored in a file (if requested). In the case where an
+empty model achieves the best AIC these statistics will not be produced.
+
+
+* * *
+
+## 4.3. Generalized Linear Models
### Description
@@ -384,7 +499,7 @@ $$x_i \,\,\,\,\mapsto\,\,\,\, \eta_i = \beta_0 + \sum\nolimits_{j=1}^m \beta_j x
In linear regression the response mean $\mu_i$ *equals* some linear
combination over $x_i$, denoted above by $\eta_i$. In logistic
-regression with $y\in\{0, 1\}$ (Bernoulli) the mean of $y$ is the same
+regression with $$y\in\{0, 1\}$$ (Bernoulli) the mean of $y$ is the same
as $Prob[y=1]$
and equals $1/(1+e^{-\eta_i})$, the logistic function of $\eta_i$. In
GLM, $\mu_i$ and $\eta_i$ can be related via any given smooth monotone
@@ -407,23 +522,23 @@ distributions and link functions, see below for details.
### Usage
hadoop jar SystemML.jar -f GLM.dml
- -nvargs X=file
- Y=file
- B=file
- fmt=format
- O=file
- Log=file
- dfam=int
- vpow=double
- link=int
- lpow=double
- yneg=double
- icpt=int
- reg=double
- tol=double
- disp=double
- moi=int
- mii=int
+ -nvargs X=<file>
+ Y=<file>
+ B=<file>
+ fmt=[format]
+ O=[file]
+ Log=[file]
+ dfam=[int]
+ vpow=[double]
+ link=[int]
+ lpow=[double]
+ yneg=[double]
+ icpt=[int]
+ reg=[double]
+ tol=[double]
+ disp=[double]
+ moi=[int]
+ mii=[int]
### Arguments
@@ -448,14 +563,14 @@ by default it is standard output.
**Log**: (default: `" "`) Location to store iteration-specific variables for monitoring
and debugging purposes, see [**Table 10**](algorithms-regression.html#table10) for details.
-**dfam**: (default: 1) Distribution family code to specify
+**dfam**: (default: `1`) Distribution family code to specify
$Prob[y\mid \mu]$,
see [**Table 11**](algorithms-regression.html#table11):
* 1 = power distributions with $Var(y) = \mu^{\alpha}$
* 2 = binomial or Bernoulli
-**vpow**: (default: 0.0) When `dfam=1`, this provides the $q$ in
+**vpow**: (default: `0.0`) When `dfam=1`, this provides the $q$ in
$Var(y) = a\mu^q$, the power
dependence of the variance of $y$ on its mean. In particular, use:
@@ -464,7 +579,7 @@ dependence of the variance of $y$ on its mean. In particular, use:
* 2.0 = Gamma
* 3.0 = inverse Gaussian
-**link**: (default: 0) Link function code to determine the link
+**link**: (default: `0`) Link function code to determine the link
function $\eta = g(\mu)$:
* 0 = canonical link (depends on the distribution family),
@@ -475,7 +590,7 @@ see [**Table 11**](algorithms-regression.html#table11)
* 4 = cloglog
* 5 = cauchit
-**lpow**: (default: 1.0) When link=1, this provides the $s$ in
+**lpow**: (default: `1.0`) When link=1, this provides the $s$ in
$\eta = \mu^s$, the power link function; `lpow=0.0` gives the
log link $\eta = \log\mu$. Common power links:
@@ -485,13 +600,13 @@ log link $\eta = \log\mu$. Common power links:
* 0.5 = sqrt
* 1.0 = identity
-**yneg**: (default: 0.0) When `dfam=2` and the response matrix $Y$ has
+**yneg**: (default: `0.0`) When `dfam=2` and the response matrix $Y$ has
1 column, this specifies the $y$-value used for Bernoulli “No” label
(“Yes” is $1$):
0.0 when $y\in\\{0, 1\\}$; -1.0 when
$y\in\\{-1, 1\\}$
-**icpt**: (default: 0) Intercept and shifting/rescaling of the features in $X$:
+**icpt**: (default: `0`) Intercept and shifting/rescaling of the features in $X$:
* 0 = no intercept (hence no $\beta_0$), no
shifting/rescaling of the features
@@ -500,18 +615,18 @@ in $X$
* 2 = add intercept, shift/rescale the features in $X$ to
mean 0, variance 1
-**reg**: (default: 0.0) L2-regularization parameter ($\lambda$)
+**reg**: (default: `0.0`) L2-regularization parameter ($\lambda$)
-**tol**: (default: 0.000001) Tolerance ($\varepsilon$) used in the convergence criterion: we
+**tol**: (default: `0.000001`) Tolerance ($\varepsilon$) used in the convergence criterion: we
terminate the outer iterations when the deviance changes by less than
this factor; see below for details
-**disp**: (default: 0.0) Dispersion parameter, or 0.0 to estimate it from
+**disp**: (default: `0.0`) Dispersion parameter, or 0.0 to estimate it from
data
-**moi**: (default: 200) Maximum number of outer (Fisher scoring) iterations
+**moi**: (default: `200`) Maximum number of outer (Fisher scoring) iterations
-**mii**: (default: 0) Maximum number of inner (conjugate gradient) iterations, or 0
+**mii**: (default: `0`) Maximum number of inner (conjugate gradient) iterations, or 0
if no maximum limit provided
* * *
@@ -702,7 +817,7 @@ $$f(\beta; X, Y) \,\,=\,\, -\sum\nolimits_{i=1}^n \big(y_i\theta_i - b(\theta_i)
\,+\,(\lambda/2) \sum\nolimits_{j=1}^m \beta_j^2\,\,\to\,\,\min$$
where
-$\theta_i$ and $b(\theta_i)$ are from (5); note that $a$ and
+$\theta_i$ and $b(\theta_i)$ are from (6); note that $a$ and
$c(y, a)$ are constant w.r.t. $\beta$ and can be ignored here. The
canonical parameter $\theta_i$ depends on both $\beta$ and $x_i$:
@@ -748,7 +863,7 @@ $0.5\sqrt{m}\,/ \max\nolimits_i \|x_i\|_2$ and updated as described
in [[Nocedal2006]](algorithms-bibliography.html).
The user can specify the maximum number of
the outer and the inner iterations with input parameters
-moi and mii, respectively. The Fisher scoring
+`moi` and `mii`, respectively. The Fisher scoring
algorithm terminates successfully if
$2|\varDelta f(z; \beta)| < (D_1(\beta) + 0.1)\hspace{0.5pt}{\varepsilon}$
where ${\varepsilon}> 0$ is a tolerance supplied by the user via
@@ -761,7 +876,7 @@ $$D_1(\beta) \,\,=\,\, 2 \cdot \big(Prob[Y \mid \!
The deviance estimate is also produced as part of the output. Once the
Fisher scoring algorithm terminates, if requested by the user, we
-estimate the dispersion $a$ from Eq. 5 using Pearson residuals
+estimate the dispersion $a$ from (6) using Pearson residuals
$$
\begin{equation}
@@ -772,7 +887,7 @@ $$
and use it to adjust our deviance estimate:
$D_{\hat{a}}(\beta) = D_1(\beta)/\hat{a}$. If input argument
disp is 0.0 we estimate $\hat{a}$, otherwise
-we use its value as $a$. Note that in (6) $m$ counts
+we use its value as $a$. Note that in (7) $m$ counts
the intercept ($m \leftarrow m+1$) if it is present.
@@ -817,7 +932,144 @@ to use the corresponding specialized scripts instead of GLM.
* * *
-## 4.3. Regression Scoring and Prediction
+## 4.4. Stepwise Generalized Linear Regression
+
+### Description
+
+Our stepwise generalized linear regression script selects a model based
+on the Akaike information criterion (AIC): the model that gives rise to
+the lowest AIC is provided. Note that currently only the Bernoulli
+distribution family is supported (see below for details).
+
+
+### Usage
+
+ hadoop jar SystemML.jar -f StepGLM.dml
+ -nvargs X=<file>
+ Y=<file>
+ B=<file>
+ S=[file]
+ O=[file]
+ link=[int]
+ yneg=[double]
+ icpt=[int]
+ tol=[double]
+ disp=[double]
+ moi=[int]
+ mii=[int]
+ thr=[double]
+ fmt=[format]
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read the matrix of feature vectors; each row is an
+example.
+
+**Y**: Location (on HDFS) to read the response matrix, which may have 1 or 2
+columns
+
+**B**: Location (on HDFS) to store the estimated regression parameters (the
+$\beta_j$’s), with the intercept parameter $\beta_0$ at position
+B\[$m\,{+}\,1$, 1\] if available
+
+**S**: (default: `" "`) Location (on HDFS) to store the selected feature-ids in the
+order as computed by the algorithm, by default it is standard output.
+
+**O**: (default: `" "`) Location (on HDFS) to write certain summary statistics
+described in [**Table 9**](algorithms-regression.html#table9), by default it is standard
+output.
+
+**link**: (default: `2`) Link function code to determine the link
+function $\eta = g(\mu)$, see [**Table 11**](algorithms-regression.html#table11); currently the
+following link functions are supported:
+
+ * 1 = log
+ * 2 = logit
+ * 3 = probit
+ * 4 = cloglog
+
+**yneg**: (default: `0.0`) Response value for Bernoulli “No” label, usually 0.0 or -1.0
+
+**icpt**: (default: `0`) Intercept and shifting/rescaling of the features in $X$:
+
+ * 0 = no intercept (hence no $\beta_0$), no
+shifting/rescaling of the features
+ * 1 = add intercept, but do not shift/rescale the features
+in $X$
+ * 2 = add intercept, shift/rescale the features in $X$ to
+mean 0, variance 1
+
+**tol**: (default: `0.000001`) Tolerance ($\epsilon$) used in the convergence criterion: we
+terminate the outer iterations when the deviance changes by less than
+this factor; see below for details.
+
+**disp**: (default: `0.0`) Dispersion parameter, or `0.0` to estimate it from
+data
+
+**moi**: (default: `200`) Maximum number of outer (Fisher scoring) iterations
+
+**mii**: (default: `0`) Maximum number of inner (conjugate gradient) iterations, or 0
+if no maximum limit provided
+
+**thr**: (default: `0.01`) Threshold to stop the algorithm: if the decrease in the value
+of the AIC falls below `thr` no further features are being
+checked and the algorithm stops.
+
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
+
+
+### Examples
+
+ hadoop jar SystemML.jar -f StepGLM.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ S=/user/ml/selected.csv
+ O=/user/ml/stats.csv
+ link=2
+ yneg=-1.0
+ icpt=2
+ tol=0.000001
+ moi=100
+ mii=10
+ thr=0.05
+ fmt=csv
+
+
+### Details
+
+Similar to `StepLinearRegDS.dml` our stepwise GLM script
+builds a model by iteratively selecting predictive variables using a
+forward selection strategy based on the AIC (5). Note that
+currently only the Bernoulli distribution family (`fam=2` in
+[**Table 11**](algorithms-regression.html#table11)) together with the following link functions
+are supported: `log`, `logit`, `probit`, and `cloglog` (link
+$$\in\{1,2,3,4\}$$ in [**Table 11**](algorithms-regression.html#table11)).
+
+
+### Returns
+
+Similar to the outputs from `GLM.dml` the stepwise GLM script
+computes the estimated regression coefficients and stores them in matrix
+$B$ on HDFS; matrix $B$ follows the same format as the one produced by
+`GLM.dml` (see [Generalized Linear Models](algorithms-regression.html#generalized-linear-models)). Additionally,
+`StepGLM.dml` outputs the variable indices (stored in the
+1-column matrix $S$) in the order they have been selected by the
+algorithm, i.e., $i$th entry in matrix $S$ stores the variable which
+improves the AIC the most in $i$th iteration. If the model with the
+lowest AIC includes no variables matrix $S$ will be empty. Moreover, the
+estimated summary statistics as defined in [**Table 9**](algorithms-regression.html#table9) are
+printed out or stored in a file on HDFS (if requested); these statistics
+will be provided only if the selected model is nonempty, i.e., contains
+at least one variable.
+
+
+* * *
+
+## 4.5. Regression Scoring and Prediction
### Description
@@ -874,7 +1126,7 @@ If $y_i$ is categorical, i.e. a vector of label counts for record $i$,
then $\mu_i$ is a vector of non-negative real numbers, one number
$$\mu_{i,l}$$ per each label $l$. In this case we divide the $$\mu_{i,l}$$
by their sum $\sum_l \mu_{i,l}$ to obtain predicted label
-probabilities . The output matrix $M$ is the
+probabilities $$p_{i,l}\in [0, 1]$$. The output matrix $M$ is the
$n \times (k\,{+}\,1)$-matrix of these probabilities, where $n$ is the
number of records and $k\,{+}\,1$ is the number of categories[^3]. Note
again that we do not predict the labels themselves, nor their actual
@@ -894,17 +1146,17 @@ this step outside the scope of `GLM-predict.dml` for now.
### Usage
hadoop jar SystemML.jar -f GLM-predict.dml
- -nvargs X=file
- Y=file
- B=file
- M=file
- O=file
- dfam=int
- vpow=double
- link=int
- lpow=double
- disp=double
- fmt=format
+ -nvargs X=<file>
+ Y=[file]
+ B=<file>
+ M=[file]
+ O=[file]
+ dfam=[int]
+ vpow=[double]
+ link=[int]
+ lpow=[double]
+ disp=[double]
+ fmt=[format]
### Arguments
@@ -946,7 +1198,7 @@ statistics defined in [**Table 13**](algorithms-regression.html#table13),
the default is to
print them to the standard output
-**dfam**: (default: 1) GLM distribution family code to specify the type of
+**dfam**: (default: `1`) GLM distribution family code to specify the type of
distribution
$Prob[y\,|\,\mu]$
that we assume:
@@ -957,7 +1209,7 @@ $Var(y) = \mu^{\alpha}$, see
* 2 = binomial
* 3 = multinomial logit
-**vpow**: (default: 0.0) Power for variance defined as (mean)$^{\textrm{power}}$
+**vpow**: (default: `0.0`) Power for variance defined as (mean)$^{\textrm{power}}$
(ignored if `dfam`$\,{\neq}\,1$): when `dfam=1`,
this provides the $q$ in
$Var(y) = a\mu^q$, the power
@@ -968,7 +1220,7 @@ dependence of the variance of $y$ on its mean. In particular, use:
* 2.0 = Gamma
* 3.0 = inverse Gaussian
-**link**: (default: 0) Link function code to determine the link
+**link**: (default: `0`) Link function code to determine the link
function $\eta = g(\mu)$, ignored for multinomial logit
(`dfam=3`):
@@ -980,7 +1232,7 @@ see [**Table 11**](algorithms-regression.html#table11)
* 4 = cloglog
* 5 = cauchit
-**lpow**: (default: 1.0) Power for link function defined as
+**lpow**: (default: `1.0`) Power for link function defined as
(mean)$^{\textrm{power}}$ (ignored if `link`$\,{\neq}\,1$):
when `link=1`, this provides the $s$ in $\eta = \mu^s$, the
power link function; `lpow=0.0` gives the log link
@@ -992,7 +1244,7 @@ $\eta = \log\mu$. Common power links:
* 0.5 = sqrt
* 1.0 = identity
-**disp**: (default: 1.0) Dispersion value, when available; must be positive
+**disp**: (default: `1.0`) Dispersion value, when available; must be positive
**fmt**: (default: `"text"`) Matrix M file output format, such as
`text`, `mm`, or `csv`; see read/write
@@ -1004,7 +1256,7 @@ functions in SystemML Language Reference for details.
Note that in the examples below the value for the `disp` input
argument is set arbitrarily. The correct dispersion value should be
computed from the training data during model estimation, or omitted if
-unknown (which sets it to 1.0).
+unknown (which sets it to `1.0`).
Linear regression example:
@@ -1263,7 +1515,7 @@ $k = \mathop{\texttt{ncol}}(Y) - 1$. Given the dispersion parameter
dispersion is accurate, $X^2 / \texttt{disp}$ should be close to \#d.f.
In fact, $X^2 / \textrm{\#d.f.}$ over the *training* data is the
dispersion estimator used in our `GLM.dml` script,
-see (6). Here we provide $X^2 / \textrm{\#d.f.}$ and
+see (7). Here we provide $X^2 / \textrm{\#d.f.}$ and
$X^2_{\texttt{disp}} / \textrm{\#d.f.}$ as
`PEARSON_X2_BY_DF` to enable dispersion comparison between
the training data and the test data.
@@ -1291,8 +1543,8 @@ $\mu_i^{\mathrm{sat}}$ to equal $y_i$ for every record (for categorical
data, $$p_{i,j}^{sat} = y_{i,j} / N_i$$), which represents the
“perfect fit.” For records with $y_{i,j} \in \{0, N_i\}$ or otherwise at
a boundary, by continuity we set $0 \log 0 = 0$. The GLM likelihood
-functions defined in (5) become simplified in
-ratio (7) due to canceling out the term $c(y, a)$
+functions defined in (6) become simplified in
+ratio (8) due to canceling out the term $c(y, a)$
since it is the same in both models.
The log of a likelihood ratio between two nested models, times two, is
@@ -1384,8 +1636,8 @@ $m$ with the intercept or $m+1$ without the intercept.
### Returns
The matrix of predicted means (if the response is numerical) or
-probabilities (if the response is categorical), see “Description”
-subsection above for more information. Given Y, we return
+probabilities (if the response is categorical), see Description
+subsection above for more information. Given `Y`, we return
some statistics in CSV format as described in
[**Table 13**](algorithms-regression.html#table13) and in the above text.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2a2520bd/algorithms-survival-analysis.md
----------------------------------------------------------------------
diff --git a/algorithms-survival-analysis.md b/algorithms-survival-analysis.md
new file mode 100644
index 0000000..b76ab91
--- /dev/null
+++ b/algorithms-survival-analysis.md
@@ -0,0 +1,709 @@
+---
+layout: global
+title: SystemML Algorithms Reference - Survival Analysis
+displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
+---
+
+
+# 6. Survival Analysis
+
+## 6.1. Kaplan-Meier Survival Analysis
+
+### Description
+
+Survival analysis examines the time needed for a particular event of
+interest to occur. In medical research, for example, the prototypical
+such event is the death of a patient but the methodology can be applied
+to other application areas, e.g., completing a task by an individual in
+a psychological experiment or the failure of electrical components in
+engineering. Kaplan-Meier or (product limit) method is a simple
+non-parametric approach for estimating survival probabilities from both
+censored and uncensored survival times.
+
+
+### Usage
+
+ hadoop jar SystemML.jar -f KM.dml
+ -nvargs X=<file>
+ TE=<file>
+ GI=<file>
+ SI=<file>
+ O=<file>
+ M=<file>
+ T=<file>
+ alpha=[double]
+ etype=[greenwood|peto]
+ ctype=[plain|log|log-log]
+ ttype=[none|log-rank|wilcoxon]
+ fmt=[format]
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read the input matrix of the survival data
+containing:
+
+ * timestamps
+ * whether event occurred (1) or data is censored (0)
+ * a number of factors (i.e., categorical features) for grouping and/or
+stratifying
+
+**TE**: Location (on HDFS) to read the 1-column matrix $TE$ that contains the
+column indices of the input matrix $X$ corresponding to timestamps
+(first entry) and event information (second entry)
+
+**GI**: Location (on HDFS) to read the 1-column matrix $GI$ that contains the
+column indices of the input matrix $X$ corresponding to the factors
+(i.e., categorical features) to be used for grouping
+
+**SI**: Location (on HDFS) to read the 1-column matrix $SI$ that contains the
+column indices of the input matrix $X$ corresponding to the factors
+(i.e., categorical features) to be used for grouping
+
+**O**: Location (on HDFS) to write the matrix containing the results of the
+Kaplan-Meier analysis $KM$
+
+**M**: Location (on HDFS) to write Matrix $M$ containing the following
+statistics: total number of events, median and its confidence intervals;
+if survival data for multiple groups and strata are provided each row of
+$M$ contains the above statistics per group and stratum.
+
+**T**: If survival data from multiple groups is available and
+`ttype=log-rank` or `ttype=wilcoxon`, location (on
+HDFS) to write the two matrices that contains the result of the
+(stratified) test for comparing these groups; see below for details.
+
+**alpha**: (default: `0.05`) Parameter to compute $100(1-\alpha)\%$ confidence intervals
+for the survivor function and its median
+
+**etype**: (default: `"greenwood"`) Parameter to specify the error type according to `greenwood`
+or `peto`
+
+**ctype**: (default: `"log"`) Parameter to modify the confidence interval; `plain` keeps
+the lower and upper bound of the confidence interval unmodified, `log`
+corresponds to logistic transformation and `log-log` corresponds to the
+complementary log-log transformation
+
+**ttype**: (default: `"none"`) If survival data for multiple groups is available specifies
+which test to perform for comparing survival data across multiple
+groups: `none`, `log-rank` or `wilcoxon` test
+
+**fmt**: (default:`"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
+
+
+### Examples
+
+ hadoop jar SystemML.jar -f KM.dml
+ -nvargs X=/user/ml/X.mtx
+ TE=/user/ml/TE
+ GI=/user/ml/GI
+ SI=/user/ml/SI
+ O=/user/ml/kaplan-meier.csv
+ M=/user/ml/model.csv
+ alpha=0.01
+ etype=greenwood
+ ctype=plain
+ fmt=csv
+
+ hadoop jar SystemML.jar -f KM.dml
+ -nvargs X=/user/ml/X.mtx
+ TE=/user/ml/TE
+ GI=/user/ml/GI
+ SI=/user/ml/SI
+ O=/user/ml/kaplan-meier.csv
+ M=/user/ml/model.csv
+ T=/user/ml/test.csv
+ alpha=0.01
+ etype=peto
+ ctype=log
+ ttype=log-rank
+ fmt=csv
+
+
+### Details
+
+The Kaplan-Meier estimate is a non-parametric maximum likelihood
+estimate (MLE) of the survival function $S(t)$, i.e., the probability of
+survival from the time origin to a given future time. As an illustration
+suppose that there are $n$ individuals with observed survival times
+$t_1,t_2,\ldots t_n$ out of which there are $r\leq n$ distinct death
+times $$t_{(1)}\leq t_{(2)}\leq t_{(r)}$$—since some of the observations
+may be censored, in the sense that the end-point of interest has not
+been observed for those individuals, and there may be more than one
+individual with the same survival time. Let $S(t_j)$ denote the
+probability of survival until time $t_j$, $d_j$ be the number of events
+at time $t_j$, and $n_j$ denote the number of individual at risk (i.e.,
+those who die at time $t_j$ or later). Assuming that the events occur
+independently, in Kaplan-Meier method the probability of surviving from
+$t_j$ to $$t_{j+1}$$ is estimated from $S(t_j)$ and given by
+
+$$\hat{S}(t) = \prod_{j=1}^{k} \left( \frac{n_j-d_j}{n_j} \right)$$
+
+for
+$$t_k\leq t<t_{k+1}$$, $$k=1,2,\ldots r$$, $$\hat{S}(t)=1$$ for $$t<t_{(1)}$$,
+and $$t_{(r+1)}=\infty$$. Note that the value of $\hat{S}(t)$ is constant
+between times of event and therefore the estimate is a step function
+with jumps at observed event times. If there are no censored data this
+estimator would simply reduce to the empirical survivor function defined
+as $\frac{n_j}{n}$. Thus, the Kaplan-Meier estimate can be seen as the
+generalization of the empirical survivor function that handles censored
+observations.
+
+The methodology used in our `KM.dml` script closely
+follows Section 2 of [[Collett2003]](algorithms-bibliography.html). For completeness we briefly
+discuss the equations used in our implementation.
+
+**Standard error of the survivor function.** The standard error of the
+estimated survivor function (controlled by parameter `etype`)
+can be calculated as
+
+$$\text{se} \{\hat{S}(t)\} \approx \hat{S}(t) {\bigg\{ \sum_{j=1}^{k} \frac{d_j}{n_j(n_j - d_j)}\biggr\}}^2$$
+
+for $$t_{(k)}\leq t<t_{(k+1)}$$. This equation is known as the
+*Greenwood’s* formula. An alternative approach is to apply
+the *Petos’s* expression
+
+$$\text{se}\{\hat{S}(t)\}=\frac{\hat{S}(t)\sqrt{1-\hat{S}(t)}}{\sqrt{n_k}}$$
+
+for $$t_{(k)}\leq t<t_{(k+1)}$$. Once the standard error of $\hat{S}$ has
+been found we compute the following types of confidence intervals
+(controlled by parameter `cctype`): The `plain`
+$100(1-\alpha)\%$ confidence interval for $S(t)$ is computed using
+
+$$\hat{S}(t)\pm z_{\alpha/2} \text{se}\{\hat{S}(t)\}$$
+
+where
+$z_{\alpha/2}$ is the upper $\alpha/2$-point of the standard normal
+distribution. Alternatively, we can apply the `log` transformation using
+
+$$\hat{S}(t)^{\exp[\pm z_{\alpha/2} \text{se}\{\hat{S}(t)\}/\hat{S}(t)]}$$
+
+or the `log-log` transformation using
+
+$$\hat{S}(t)^{\exp [\pm z_{\alpha/2} \text{se} \{\log [-\log \hat{S}(t)]\}]}$$
+
+**Median, its standard error and confidence interval.** Denote by
+$$\hat{t}(50)$$ the estimated median of $$\hat{S}$$, i.e.,
+$$\hat{t}(50)=\min \{ t_i \mid \hat{S}(t_i) < 0.5\}$$, where $$t_i$$ is the
+observed survival time for individual $$i$$. The standard error of
+$$\hat{t}(50)$$ is given by
+
+$$\text{se}\{ \hat{t}(50) \} = \frac{1}{\hat{f}\{\hat{t}(50)\}} \text{se}[\hat{S}\{ \hat{t}(50) \}]$$
+
+where $$\hat{f}\{ \hat{t}(50) \}$$ can be found from
+
+$$\hat{f}\{ \hat{t}(50) \} = \frac{\hat{S}\{ \hat{u}(50) \} -\hat{S}\{ \hat{l}(50) \} }{\hat{l}(50) - \hat{u}(50)}$$
+
+Above, $\hat{u}(50)$ is the largest survival time for which $\hat{S}$
+exceeds $0.5+\epsilon$, i.e.,
+$$\hat{u}(50)=\max \bigl\{ t_{(j)} \mid \hat{S}(t_{(j)}) \geq 0.5+\epsilon \bigr\}$$,
+and $\hat{l}(50)$ is the smallest survivor time for which $\hat{S}$ is
+less than $0.5-\epsilon$, i.e.,
+$$\hat{l}(50)=\min \bigl\{ t_{(j)} \mid \hat{S}(t_{(j)}) \leq 0.5+\epsilon \bigr\}$$,
+for small $\epsilon$.
+
+**Log-rank test and Wilcoxon test.** Our implementation supports
+comparison of survival data from several groups using two non-parametric
+procedures (controlled by parameter `ttype`): the
+*log-rank test* and the *Wilcoxon test* (also
+known as the *Breslow test*). Assume that the survival
+times in $g\geq 2$ groups of survival data are to be compared. Consider
+the *null hypothesis* that there is no difference in the
+survival times of the individuals in different groups. One way to
+examine the null hypothesis is to consider the difference between the
+observed number of deaths with the numbers expected under the null
+hypothesis. In both tests we define the $U$-statistics ($$U_{L}$$ for the
+log-rank test and $$U_{W}$$ for the Wilcoxon test) to compare the observed
+and the expected number of deaths in $1,2,\ldots,g-1$ groups as follows:
+
+$$\begin{aligned}
+U_{Lk} &= \sum_{j=1}^{r}\left( d_{kj} - \frac{n_{kj}d_j}{n_j} \right) \\
+U_{Wk} &= \sum_{j=1}^{r}n_j\left( d_{kj} - \frac{n_{kj}d_j}{n_j} \right)\end{aligned}$$
+
+where $$d_{kj}$$ is the of number deaths at time $$t_{(j)}$$ in group $k$,
+$$n_{kj}$$ is the number of individuals at risk at time $$t_{(j)}$$ in group
+$k$, and $k=1,2,\ldots,g-1$ to form the vectors $U_L$ and $U_W$ with
+$(g-1)$ components. The covariance (variance) between $$U_{Lk}$$ and
+$$U_{Lk'}$$ (when $k=k'$) is computed as
+
+$$V_{Lkk'}=\sum_{j=1}^{r} \frac{n_{kj}d_j(n_j-d_j)}{n_j(n_j-1)} \left( \delta_{kk'}-\frac{n_{k'j}}{n_j} \right)$$
+
+for $k,k'=1,2,\ldots,g-1$, with
+
+$$\delta_{kk'} =
+\begin{cases}
+1 & \text{if } k=k'\\
+0 & \text{otherwise}
+\end{cases}$$
+
+These terms are combined in a
+*variance-covariance* matrix $V_L$ (referred to as the
+$V$-statistic). Similarly, the variance-covariance matrix for the
+Wilcoxon test $V_W$ is a matrix where the entry at position $(k,k')$ is
+given by
+
+$$V_{Wkk'}=\sum_{j=1}^{r} n_j^2 \frac{n_{kj}d_j(n_j-d_j)}{n_j(n_j-1)} \left( \delta_{kk'}-\frac{n_{k'j}}{n_j} \right)$$
+
+Under the null hypothesis of no group differences, the test statistics
+$U_L^\top V_L^{-1} U_L$ for the log-rank test and
+$U_W^\top V_W^{-1} U_W$ for the Wilcoxon test have a Chi-squared
+distribution on $(g-1)$ degrees of freedom. Our `KM.dml`
+script also provides a stratified version of the log-rank or Wilcoxon
+test if requested. In this case, the values of the $U$- and $V$-
+statistics are computed for each stratum and then combined over all
+strata.
+
+### Returns
+
+
+Below we list the results of the survival analysis computed by
+`KM.dml`. The calculated statistics are stored in matrix $KM$
+with the following schema:
+
+ * Column 1: timestamps
+ * Column 2: number of individuals at risk
+ * Column 3: number of events
+ * Column 4: Kaplan-Meier estimate of the survivor function $\hat{S}$
+ * Column 5: standard error of $\hat{S}$
+ * Column 6: lower bound of $100(1-\alpha)\%$ confidence interval for
+ $\hat{S}$
+ * Column 7: upper bound of $100(1-\alpha)\%$ confidence interval for
+ $\hat{S}$
+
+Note that if survival data for multiple groups and/or strata is
+available, each collection of 7 columns in $KM$ stores the results per
+group and/or per stratum. In this case $KM$ has $7g+7s$ columns, where
+$g\geq 1$ and $s\geq 1$ denote the number of groups and strata,
+respectively.
+
+Additionally, `KM.dml` stores the following statistics in the
+1-row matrix $M$ whose number of columns depends on the number of groups
+($g$) and strata ($s$) in the data. Below $k$ denotes the number of
+factors used for grouping and $l$ denotes the number of factors used for
+stratifying.
+
+ * Columns 1 to $k$: unique combination of values in the $k$ factors
+ used for grouping
+ * Columns $k+1$ to $k+l$: unique combination of values in the $l$
+ factors used for stratifying
+ * Column $k+l+1$: total number of records
+ * Column $k+l+2$: total number of events
+ * Column $k+l+3$: median of $\hat{S}$
+ * Column $k+l+4$: lower bound of $100(1-\alpha)\%$ confidence interval
+ for the median of $\hat{S}$
+ * Column $k+l+5$: upper bound of $100(1-\alpha)\%$ confidence interval
+ for the median of $\hat{S}$.
+
+If there is only 1 group and 1 stratum available $M$ will be a 1-row
+matrix with 5 columns where
+
+ * Column 1: total number of records
+ * Column 2: total number of events
+ * Column 3: median of $\hat{S}$
+ * Column 4: lower bound of $100(1-\alpha)\%$ confidence interval for
+ the median of $\hat{S}$
+ * Column 5: upper bound of $100(1-\alpha)\%$ confidence interval for
+ the median of $\hat{S}$.
+
+If a comparison of the survival data across multiple groups needs to be
+performed, `KM.dml` computes two matrices $T$ and
+$$T\_GROUPS\_OE$$ that contain a summary of the test. The 1-row matrix $T$
+stores the following statistics:
+
+ * Column 1: number of groups in the survival data
+ * Column 2: degree of freedom for Chi-squared distributed test
+ statistic
+ * Column 3: value of test statistic
+ * Column 4: $P$-value.
+
+Matrix $$T\_GROUPS\_OE$$ contains the following statistics for each of $g$
+groups:
+
+ * Column 1: number of events
+ * Column 2: number of observed death times ($O$)
+ * Column 3: number of expected death times ($E$)
+ * Column 4: $(O-E)^2/E$
+ * Column 5: $(O-E)^2/V$.
+
+
+* * *
+
+## 6.2. Cox Proportional Hazard Regression Model
+
+### Description
+
+The Cox (proportional hazard or PH) is a semi-parametric statistical
+approach commonly used for analyzing survival data. Unlike
+non-parametric approaches, e.g., the [Kaplan-Meier estimates](algorithms-survival-analysis.html#kaplan-meier-survival-analysis),
+which can be used to analyze single sample of
+survival data or to compare between groups of survival times, the Cox PH
+models the dependency of the survival times on the values of
+*explanatory variables* (i.e., covariates) recorded for
+each individual at the time origin. Our focus is on covariates that do
+not change value over time, i.e., time-independent covariates, and that
+may be categorical (ordinal or nominal) as well as continuous-valued.
+
+
+### Usage
+
+**Cox**:
+
+ hadoop jar SystemML.jar -f Cox.dml
+ -nvargs X=<file>
+ TE=<file>
+ F=<file>
+ R=[file]
+ M=<file>
+ S=[file]
+ T=[file]
+ COV=<file>
+ RT=<file>
+ XO=<file>
+ MF=<file>
+ alpha=[double]
+ tol=[double]
+ moi=[int]
+ mii=[int]
+ fmt=[format]
+
+**Cox Prediction**:
+
+ hadoop jar SystemML.jar -f Cox-predict.dml
+ -nvargs X=<file>
+ RT=<file>
+ M=<file>
+ Y=<file>
+ COV=<file>
+ MF=<file>
+ P=<file>
+ fmt=[format]
+
+### Arguments - Cox Model Fitting/Prediction
+
+**X**: Location (on HDFS) to read the input matrix of the survival data
+containing:
+
+ * timestamps
+ * whether event occurred (1) or data is censored (0)
+ * feature vectors
+
+**Y**: Location (on HDFS) to the read matrix used for prediction
+
+**TE**: Location (on HDFS) to read the 1-column matrix $TE$ that contains the
+column indices of the input matrix $X$ corresponding to timestamps
+(first entry) and event information (second entry)
+
+**F**: Location (on HDFS) to read the 1-column matrix $F$ that contains the
+column indices of the input matrix $X$ corresponding to the features to
+be used for fitting the Cox model
+
+**R**: (default: `" "`) If factors (i.e., categorical features) are available in the
+input matrix $X$, location (on HDFS) to read matrix $R$ containing the
+start (first column) and end (second column) indices of each factor in
+$X$; alternatively, user can specify the indices of the baseline level
+of each factor which needs to be removed from $X$. If $R$ is not
+provided by default all variables are considered to be
+continuous-valued.
+
+**M**: Location (on HDFS) to store the results of Cox regression analysis
+including regression coefficients $\beta_j$s, their standard errors,
+confidence intervals, and $P$-values
+
+**S**: (default: `" "`) Location (on HDFS) to store a summary of some statistics of
+the fitted model including number of records, number of events,
+log-likelihood, AIC, Rsquare (Cox & Snell), and maximum possible Rsquare
+
+**T**: (default: `" "`) Location (on HDFS) to store the results of Likelihood ratio
+test, Wald test, and Score (log-rank) test of the fitted model
+
+**COV**: Location (on HDFS) to store the variance-covariance matrix of
+$\beta_j$s; note that parameter `COV` needs to be provided as
+input to prediction.
+
+**RT**: Location (on HDFS) to store matrix $RT$ containing the order-preserving
+recoded timestamps from $X$; note that parameter `RT` needs
+to be provided as input for prediction.
+
+**XO**: Location (on HDFS) to store the input matrix $X$ ordered by the
+timestamps; note that parameter `XO` needs to be provided as
+input for prediction.
+
+**MF**: Location (on HDFS) to store column indices of $X$ excluding the baseline
+factors if available; note that parameter `MF` needs to be
+provided as input for prediction.
+
+**P**: Location (on HDFS) to store matrix $P$ containing the results of
+prediction
+
+**alpha**: (default: `0.05`) Parameter to compute a $100(1-\alpha)\%$ confidence interval
+for $\beta_j$s
+
+**tol**: (default: `0.000001`) Tolerance ($\epsilon$) used in the convergence criterion
+
+**moi**: (default: `100`) Maximum number of outer (Fisher scoring) iterations
+
+**mii**: (default: `0`) Maximum number of inner (conjugate gradient) iterations, or 0
+if no maximum limit provided
+
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
+
+
+### Examples
+
+**Cox**:
+
+ hadoop jar SystemML.jar -f Cox.dml
+ -nvargs X=/user/ml/X.mtx
+ TE=/user/ml/TE
+ F=/user/ml/F
+ R=/user/ml/R
+ M=/user/ml/model.csv
+ T=/user/ml/test.csv
+ COV=/user/ml/var-covar.csv
+ XO=/user/ml/X-sorted.mtx
+ fmt=csv
+
+ hadoop jar SystemML.jar -f Cox.dml
+ -nvargs X=/user/ml/X.mtx
+ TE=/user/ml/TE
+ F=/user/ml/F
+ R=/user/ml/R
+ M=/user/ml/model.csv
+ T=/user/ml/test.csv
+ COV=/user/ml/var-covar.csv
+ RT=/user/ml/recoded-timestamps.csv
+ XO=/user/ml/X-sorted.csv
+ MF=/user/ml/baseline.csv
+ alpha=0.01
+ tol=0.000001
+ moi=100
+ mii=20
+ fmt=csv
+
+**Cox Prediction**:
+
+ hadoop jar SystemML.jar -f Cox-predict.dml
+ -nvargs X=/user/ml/X-sorted.mtx
+ RT=/user/ml/recoded-timestamps.csv
+ M=/user/ml/model.csv
+ Y=/user/ml/Y.mtx
+ COV=/user/ml/var-covar.csv
+ MF=/user/ml/baseline.csv
+ P=/user/ml/predictions.csv
+ fmt=csv
+
+
+### Details
+
+In the Cox PH regression model, the relationship between the hazard
+function — i.e., the probability of event occurrence at a given time — and
+the covariates is described as
+
+$$
+\begin{equation}
+h_i(t)=h_0(t)\exp\Bigl\{ \sum_{j=1}^{p} \beta_jx_{ij} \Bigr\}
+\end{equation}
+$$
+
+where the hazard function for the $i$th individual
+($$i\in\{1,2,\ldots,n\}$$) depends on a set of $p$ covariates
+$$x_i=(x_{i1},x_{i2},\ldots,x_{ip})$$, whose importance is measured by the
+magnitude of the corresponding coefficients
+$$\beta=(\beta_1,\beta_2,\ldots,\beta_p)$$. The term $$h_0(t)$$ is the
+baseline hazard and is related to a hazard value if all covariates equal
+0. In the Cox PH model the hazard function for the individuals may vary
+over time, however the baseline hazard is estimated non-parametrically
+and can take any form. Note that re-writing (1) we have
+
+$$\log\biggl\{ \frac{h_i(t)}{h_0(t)} \biggr\} = \sum_{j=1}^{p} \beta_jx_{ij}$$
+
+Thus, the Cox PH model is essentially a linear model for the logarithm
+of the hazard ratio and the hazard of event for any individual is a
+constant multiple of the hazard of any other. We follow similar notation
+and methodology as in Section 3 of
+[[Collett2003]](algorithms-bibliography.html). For
+completeness we briefly discuss the equations used in our
+implementation.
+
+**Factors in the model.** Note that if some of the feature variables are
+factors they need to *dummy code* as follows. Let $\alpha$
+be such a variable (i.e., a factor) with $a$ levels. We introduce $a-1$
+indicator (or dummy coded) variables $X_2,X_3\ldots,X_a$ with $X_j=1$ if
+$\alpha=j$ and 0 otherwise, for $$j\in\{ 2,3,\ldots,a\}$$. In particular,
+one of $a$ levels of $\alpha$ will be considered as the baseline and is
+not included in the model. In our implementation, user can specify a
+baseline level for each of the factor (as selecting the baseline level
+for each factor is arbitrary). On the other hand, if for a given factor
+$\alpha$ no baseline is specified by the user, the most frequent level
+of $\alpha$ will be considered as the baseline.
+
+**Fitting the model.** We estimate the coefficients of the Cox model via
+negative log-likelihood method. In particular the Cox PH model is fitted
+by using trust region Newton method with conjugate
+gradient [[Nocedal2006]](algorithms-bibliography.html). Define the risk set $R(t_j)$ at time
+$t_j$ to be the set of individuals who die at time $t_i$ or later. The
+PH model assumes that survival times are distinct. In order to handle
+tied observations we use the *Breslow* approximation of the likelihood
+function
+
+$$\mathcal{L}=\prod_{j=1}^{r} \frac{\exp(\beta^\top s_j)}{\biggl\{ \sum_{l\in R(t_j)} \exp(\beta^\top x_l) \biggr\}^{d_j} }$$
+
+where $d_j$ is number individuals who die at time $t_j$ and $s_j$
+denotes the element-wise sum of the covariates for those individuals who
+die at time $t_j$, $j=1,2,\ldots,r$, i.e., the $h$th element of $s_j$ is
+given by $$s_{hj}=\sum_{k=1}^{d_j}x_{hjk}$$, where $x_{hjk}$ is the value
+of $h$th variable ($$h\in \{1,2,\ldots,p\}$$) for the $k$th of the $d_j$
+individuals ($$k\in\{ 1,2,\ldots,d_j \}$$) who die at the $j$th death time
+($$j\in\{ 1,2,\ldots,r \}$$).
+
+**Standard error and confidence interval for coefficients.** Note that
+the variance-covariance matrix of the estimated coefficients
+$\hat{\beta}$ can be approximated by the inverse of the Hessian
+evaluated at $\hat{\beta}$. The square root of the diagonal elements of
+this matrix are the standard errors of estimated coefficients. Once the
+standard errors of the coefficients $se(\hat{\beta})$ is obtained we can
+compute a $100(1-\alpha)\%$ confidence interval using
+$$\hat{\beta}\pm z_{\alpha/2}se(\hat{\beta})$$, where $z_{\alpha/2}$ is
+the upper $\alpha/2$-point of the standard normal distribution. In
+`Cox.dml`, we utilize the built-in function
+`inv()` to compute the inverse of the Hessian. Note that this
+build-in function can be used only if the Hessian fits in the main
+memory of a single machine.
+
+**Wald test, likelihood ratio test, and log-rank test.** In order to
+test the *null hypothesis* that all of the coefficients
+$\beta_j$s are 0, our implementation provides three statistical test:
+*Wald test*, *likelihood ratio test*, the
+*log-rank test* (also known as the *score
+test*). Let $p$ be the number of coefficients. The Wald test is
+based on the test statistic ${\hat{\beta}}^2/{se(\hat{\beta})}^2$, which
+is compared to percentage points of the Chi-squared distribution to
+obtain the $P$-value. The likelihood ratio test relies on the test
+statistic $$-2\log\{ {L}(\textbf{0})/{L}(\hat{\beta}) \}$$ ($\textbf{0}$
+denotes a zero vector of size $p$ ) which has an approximate Chi-squared
+distribution with $p$ degrees of freedom under the null hypothesis that
+all $\beta_j$s are 0. The Log-rank test is based on the test statistic
+$l=\nabla^\top L(\textbf{0}) {\mathcal{H}}^{-1}(\textbf{0}) \nabla L(\textbf{0})$,
+where $\nabla L(\textbf{0})$ is the gradient of $L$ and
+$\mathcal{H}(\textbf{0})$ is the Hessian of $L$ evaluated at **0**.
+Under the null hypothesis that $\beta=\textbf{0}$, $l$ has a Chi-squared
+distribution on $p$ degrees of freedom.
+
+**Prediction.** Once the parameters of the model are fitted, we compute
+the following predictions together with their standard errors
+
+ * linear predictors
+ * risk
+ * estimated cumulative hazard
+
+Given feature vector $$X_i$$ for individual $$i$$, we obtain the above
+predictions at time $$t$$ as follows. The linear predictors (denoted as
+$$\mathcal{LP}$$) as well as the risk (denoted as $\mathcal{R}$) are
+computed relative to a baseline whose feature values are the mean of the
+values in the corresponding features. Let $$X_i^\text{rel} = X_i - \mu$$,
+where $$\mu$$ is a row vector that contains the mean values for each
+feature. We have $$\mathcal{LP}=X_i^\text{rel} \hat{\beta}$$ and
+$$\mathcal{R}=\exp\{ X_i^\text{rel}\hat{\beta} \}$$. The standard errors
+of the linear predictors $$se\{\mathcal{LP} \}$$ are computed as the
+square root of $${(X_i^\text{rel})}^\top V(\hat{\beta}) X_i^\text{rel}$$
+and the standard error of the risk $$se\{ \mathcal{R} \}$$ are given by
+the square root of
+$${(X_i^\text{rel} \odot \mathcal{R})}^\top V(\hat{\beta}) (X_i^\text{rel} \odot \mathcal{R})$$,
+where $$V(\hat{\beta})$$ is the variance-covariance matrix of the
+coefficients and $$\odot$$ is the element-wise multiplication.
+
+We estimate the cumulative hazard function for individual $i$ by
+
+$$\hat{H}_i(t) = \exp(\hat{\beta}^\top X_i) \hat{H}_0(t)$$
+
+where
+$$\hat{H}_0(t)$$ is the *Breslow estimate* of the cumulative baseline
+hazard given by
+
+$$\hat{H}_0(t) = \sum_{j=1}^{k} \frac{d_j}{\sum_{l\in R(t_{(j)})} \exp(\hat{\beta}^\top X_l)}$$
+
+In the equation above, as before, $d_j$ is the number of deaths, and
+$$R(t_{(j)})$$ is the risk set at time $$t_{(j)}$$, for
+$$t_{(k)} \leq t \leq t_{(k+1)}$$, $$k=1,2,\ldots,r-1$$. The standard error
+of $$\hat{H}_i(t)$$ is obtained using the estimation
+
+$$se\{ \hat{H}_i(t) \} = \sum_{j=1}^{k} \frac{d_j}{ {\left[ \sum_{l\in R(t_{(j)})} \exp(X_l\hat{\beta}) \right]}^2 } + J_i^\top(t) V(\hat{\beta}) J_i(t)\$$
+
+where
+
+$$J_i(t) = \sum_{j-1}^{k} d_j \frac{\sum_{l\in R(t_{(j)})} (X_l-X_i)\exp \{ (X_l-X_i)\hat{\beta} \}}{ {\left[ \sum_{l\in R(t_{(j)})} \exp\{(X_l-X_i)\hat{\beta}\} \right]}^2 }$$
+
+for $$t_{(k)} \leq t \leq t_{(k+1)}$, $k=1,2,\ldots,r-1$$.
+
+
+### Returns
+
+Below we list the results of fitting a Cox regression model stored in
+matrix $M$ with the following schema:
+
+ * Column 1: estimated regression coefficients $\hat{\beta}$
+ * Column 2: $\exp(\hat{\beta})$
+ * Column 3: standard error of the estimated coefficients
+ $se\{\hat{\beta}\}$
+ * Column 4: ratio of $\hat{\beta}$ to $se\{\hat{\beta}\}$ denoted by
+ $Z$
+ * Column 5: $P$-value of $Z$
+ * Column 6: lower bound of $100(1-\alpha)\%$ confidence interval for
+ $\hat{\beta}$
+ * Column 7: upper bound of $100(1-\alpha)\%$ confidence interval for
+ $\hat{\beta}$.
+
+Note that above $Z$ is the Wald test statistic which is asymptotically
+standard normal under the hypothesis that $\beta=\textbf{0}$.
+
+Moreover, `Cox.dml` outputs two log files `S` and
+`T` containing a summary statistics of the fitted model as
+follows. File `S` stores the following information
+
+ * Line 1: total number of observations
+ * Line 2: total number of events
+ * Line 3: log-likelihood (of the fitted model)
+ * Line 4: AIC
+ * Line 5: Cox & Snell Rsquare
+ * Line 6: maximum possible Rsquare.
+
+Above, the AIC is computed as in [[Stepwise Linear Regression]](algorithms-regression.html#stepwise-linear-regression), the Cox & Snell Rsquare
+is equal to $$1-\exp\{ -l/n \}$$, where $l$ is the log-rank test statistic
+as discussed above and $n$ is total number of observations, and the
+maximum possible Rsquare computed as $$1-\exp\{ -2 L(\textbf{0})/n \}$$,
+where $L(\textbf{0})$ denotes the initial likelihood.
+
+File `T` contains the following information
+
+ * Line 1: Likelihood ratio test statistic, degree of freedom of the
+ corresponding Chi-squared distribution, $P$-value
+ * Line 2: Wald test statistic, degree of freedom of the corresponding
+ Chi-squared distribution, $P$-value
+ * Line 3: Score (log-rank) test statistic, degree of freedom of the
+ corresponding Chi-squared distribution, $P$-value.
+
+Additionally, the following matrices will be stored. Note that these
+matrices are required for prediction.
+
+ * Order-preserving recoded timestamps $RT$, i.e., contiguously
+ numbered from 1 $\ldots$ \#timestamps
+ * Feature matrix ordered by the timestamps $XO$
+ * Variance-covariance matrix of the coefficients $COV$
+ * Column indices of the feature matrix with baseline factors removed
+ (if available) $MF$.
+
+**Prediction.** Finally, the results of prediction is stored in Matrix
+$P$ with the following schema
+
+ * Column 1: linear predictors
+ * Column 2: standard error of the linear predictors
+ * Column 3: risk
+ * Column 4: standard error of the risk
+ * Column 5: estimated cumulative hazard
+ * Column 6: standard error of the estimated cumulative hazard.
+
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2a2520bd/img/algorithms-reference-figure-1-computation-of-quartiles-median-and-interquartile-mean.png
----------------------------------------------------------------------
diff --git a/img/algorithms-reference-figure-1-computation-of-quartiles-median-and-interquartile-mean.png b/img/algorithms-reference-figure-1-computation-of-quartiles-median-and-interquartile-mean.png
deleted file mode 100644
index e53fbb7..0000000
Binary files a/img/algorithms-reference-figure-1-computation-of-quartiles-median-and-interquartile-mean.png and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2a2520bd/img/algorithms-reference/computation-of-quartiles-median-and-interquartile-mean.png
----------------------------------------------------------------------
diff --git a/img/algorithms-reference/computation-of-quartiles-median-and-interquartile-mean.png b/img/algorithms-reference/computation-of-quartiles-median-and-interquartile-mean.png
new file mode 100644
index 0000000..e53fbb7
Binary files /dev/null and b/img/algorithms-reference/computation-of-quartiles-median-and-interquartile-mean.png differ
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2a2520bd/img/algorithms-reference/example-tree.png
----------------------------------------------------------------------
diff --git a/img/algorithms-reference/example-tree.png b/img/algorithms-reference/example-tree.png
new file mode 100644
index 0000000..17e4d93
Binary files /dev/null and b/img/algorithms-reference/example-tree.png differ
[30/47] incubator-systemml git commit: Generalization wdivmm
rewrite/compiler/runtime (* patterns), for als_cg
Posted by du...@apache.org.
Generalization wdivmm rewrite/compiler/runtime (* patterns), for als_cg
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/ec7b2626
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/ec7b2626
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/ec7b2626
Branch: refs/heads/gh-pages
Commit: ec7b2626e9afe7e220292425871a8abe9e5b52e1
Parents: ebb052d
Author: Matthias Boehm <mb...@us.ibm.com>
Authored: Thu Sep 24 19:28:26 2015 -0700
Committer: Matthias Boehm <mb...@us.ibm.com>
Committed: Thu Sep 24 19:28:26 2015 -0700
----------------------------------------------------------------------
devdocs/MatrixMultiplicationOperators.txt | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/ec7b2626/devdocs/MatrixMultiplicationOperators.txt
----------------------------------------------------------------------
diff --git a/devdocs/MatrixMultiplicationOperators.txt b/devdocs/MatrixMultiplicationOperators.txt
index 839a168..2fc831f 100644
--- a/devdocs/MatrixMultiplicationOperators.txt
+++ b/devdocs/MatrixMultiplicationOperators.txt
@@ -1,6 +1,6 @@
#####################################################################
# TITLE: An Overview of Matrix Multiplication Operators in SystemML #
-# DATE MODIFIED: 09/24/2015 #
+# DATE MODIFIED: 09/25/2015 #
#####################################################################
In the following, we give an overview of backend-specific physical matrix multiplication operators in SystemML as well as their internally used matrix multiplication block operations.
@@ -104,9 +104,10 @@ C) CORE MATRIX MULT PRIMITIVES LibMatrixMult (incl related script patterns)
- sequential / multi-threaded (same block ops, par over rows in W)
- all dense, sparse-dense factors, sparse/dense-* x 4 patterns
-* 7) wdivmm ((a) t(t(U)%*%(X/(U%*%t(V)))), (b) (X/(U%*%t(V)))%*%V)
+* 7) wdivmm ((a) t(t(U)%*%(X/(U%*%t(V)))), (b) (X/(U%*%t(V)))%*%V),
+ (c) t(t(U)%*%(X*(U%*%t(V)))), (d) (X*(U%*%t(V)))%*%V)
- sequential / multi-threaded (same block ops, par over rows in X)
- - all dense, sparse-dense factors, sparse/dense-* x 2 patterns
+ - all dense, sparse-dense factors, sparse/dense-* x 4 patterns
* 8) wcemm (sum(X*log(U%*%t(V))))
- sequential / multi-threaded (same block ops, par over rows in X)
[11/47] incubator-systemml git commit: initial port of documentation
to md, algorithms reference converted to md, template based on spark docs
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/css/main.css
----------------------------------------------------------------------
diff --git a/css/main.css b/css/main.css
new file mode 100644
index 0000000..485f08b
--- /dev/null
+++ b/css/main.css
@@ -0,0 +1,172 @@
+/* ==========================================================================
+ Author's custom styles
+ ========================================================================== */
+
+.navbar .brand {
+ height: 50px;
+ width: 180px;
+ margin-left: 1px;
+ padding: 0;
+}
+
+.version {
+ line-height: 30px;
+ vertical-align: bottom;
+ font-size: 12px;
+ padding: 0;
+ margin: 0;
+ font-weight: bold;
+ color: #777;
+}
+
+.navbar-inner {
+ padding-top: 2px;
+ height: 50px;
+}
+
+.navbar-inner .nav {
+ margin-top: 5px;
+ font-size: 15px;
+}
+
+.navbar .divider-vertical {
+ border-right-color: lightgray;
+}
+
+.navbar-text .version-text {
+ color: #555555;
+ padding: 5px;
+ margin-left: 10px;
+}
+
+body #content {
+ line-height: 1.6; /* Inspired by Github's wiki style */
+}
+
+.title {
+ font-size: 32px;
+}
+
+h1 {
+ font-size: 28px;
+ margin-top: 12px;
+}
+
+h2 {
+ font-size: 24px;
+ margin-top: 12px;
+}
+
+h3 {
+ font-size: 21px;
+ margin-top: 10px;
+}
+
+pre {
+ font-family: "Menlo", "Lucida Console", monospace;
+}
+
+code {
+ font-family: "Menlo", "Lucida Console", monospace;
+ background: white;
+ border: none;
+ padding: 0;
+ color: #444444;
+}
+
+a code {
+ color: #0088cc;
+}
+
+a:hover code {
+ color: #005580;
+ text-decoration: underline;
+}
+
+.container {
+ max-width: 914px;
+}
+
+.dropdown-menu {
+ /* Remove the default 2px top margin which causes a small
+ gap between the hover trigger area and the popup menu */
+ margin-top: 0;
+ /* Avoid too much whitespace at the right for shorter menu items */
+ min-width: 50px;
+}
+
+/**
+ * Make dropdown menus in nav bars show on hover instead of click
+ * using solution at http://stackoverflow.com/questions/8878033/how-
+ * to-make-twitter-bootstrap-menu-dropdown-on-hover-rather-than-click
+ **/
+ul.nav li.dropdown:hover ul.dropdown-menu{
+ display: block;
+}
+
+a.menu:after, .dropdown-toggle:after {
+ content: none;
+}
+
+/** Make the submenus open on hover on the parent menu item */
+ul.nav li.dropdown ul.dropdown-menu li.dropdown-submenu:hover ul.dropdown-menu {
+ display: block;
+}
+
+/** Make the submenus be invisible until the parent menu item is hovered upon */
+ul.nav li.dropdown ul.dropdown-menu li.dropdown-submenu ul.dropdown-menu {
+ display: none;
+}
+
+/**
+ * Made the navigation bar buttons not grey out when clicked.
+ * Essentially making nav bar buttons not react to clicks, only hover events.
+ */
+.navbar .nav li.dropdown.open > .dropdown-toggle {
+ background-color: transparent;
+}
+
+/**
+ * Made the active tab caption blue. Otherwise the active tab is black, and inactive tab is blue.
+ * That looks weird. Changed the colors to active - blue, inactive - black, and
+ * no color change on hover.
+ */
+.nav-tabs > .active > a, .nav-tabs > .active > a:hover {
+ color: #08c;
+}
+
+.nav-tabs > li > a, .nav-tabs > li > a:hover {
+ color: #333;
+}
+
+/**
+ * MathJax (embedded latex formulas)
+ */
+.MathJax .mo { color: inherit }
+.MathJax .mi { color: inherit }
+.MathJax .mf { color: inherit }
+.MathJax .mh { color: inherit }
+
+/**
+ * AnchorJS (anchor links when hovering over headers)
+ */
+a.anchorjs-link:hover { text-decoration: none; }
+
+/**
+SystemML additions
+*/
+
+table td, table th {
+ border: 1px solid #333;
+ padding: 0 .5em;
+}
+
+table th {
+ background: #CCC;
+ padding: 0 .5em;
+}
+
+body {
+ padding-top: 60px;
+ padding-bottom: 40px;
+}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/css/pygments-default.css
----------------------------------------------------------------------
diff --git a/css/pygments-default.css b/css/pygments-default.css
new file mode 100644
index 0000000..6247cd8
--- /dev/null
+++ b/css/pygments-default.css
@@ -0,0 +1,76 @@
+/*
+Documentation for pygments (and Jekyll for that matter) is super sparse.
+To generate this, I had to run
+ `pygmentize -S default -f html > pygments-default.css`
+But first I had to install pygments via easy_install pygments
+
+I had to override the conflicting bootstrap style rules by linking to
+this stylesheet lower in the html than the bootstap css.
+
+Also, I was thrown off for a while at first when I was using markdown
+code block inside my {% highlight scala %} ... {% endhighlight %} tags
+(I was using 4 spaces for this), when it turns out that pygments will
+insert the code (or pre?) tags for you.
+*/
+
+.hll { background-color: #ffffcc }
+.c { color: #60a0b0; font-style: italic } /* Comment */
+.err { } /* Error */
+.k { color: #007020; font-weight: bold } /* Keyword */
+.o { color: #666666 } /* Operator */
+.cm { color: #60a0b0; font-style: italic } /* Comment.Multiline */
+.cp { color: #007020 } /* Comment.Preproc */
+.c1 { color: #60a0b0; font-style: italic } /* Comment.Single */
+.cs { color: #60a0b0; background-color: #fff0f0 } /* Comment.Special */
+.gd { color: #A00000 } /* Generic.Deleted */
+.ge { font-style: italic } /* Generic.Emph */
+.gr { color: #FF0000 } /* Generic.Error */
+.gh { color: #000080; font-weight: bold } /* Generic.Heading */
+.gi { color: #00A000 } /* Generic.Inserted */
+.go { color: #808080 } /* Generic.Output */
+.gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */
+.gs { font-weight: bold } /* Generic.Strong */
+.gu { color: #800080; font-weight: bold } /* Generic.Subheading */
+.gt { color: #0040D0 } /* Generic.Traceback */
+.kc { color: #007020; font-weight: bold } /* Keyword.Constant */
+.kd { color: #007020; font-weight: bold } /* Keyword.Declaration */
+.kn { color: #007020; font-weight: bold } /* Keyword.Namespace */
+.kp { color: #007020 } /* Keyword.Pseudo */
+.kr { color: #007020; font-weight: bold } /* Keyword.Reserved */
+.kt { color: #902000 } /* Keyword.Type */
+.m { color: #40a070 } /* Literal.Number */
+.s { color: #4070a0 } /* Literal.String */
+.na { color: #4070a0 } /* Name.Attribute */
+.nb { color: #007020 } /* Name.Builtin */
+.nc { color: #0e84b5; font-weight: bold } /* Name.Class */
+.no { color: #60add5 } /* Name.Constant */
+.nd { color: #555555; font-weight: bold } /* Name.Decorator */
+.ni { color: #d55537; font-weight: bold } /* Name.Entity */
+.ne { color: #007020 } /* Name.Exception */
+.nf { color: #06287e } /* Name.Function */
+.nl { color: #002070; font-weight: bold } /* Name.Label */
+.nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */
+.nt { color: #062873; font-weight: bold } /* Name.Tag */
+.nv { color: #bb60d5 } /* Name.Variable */
+.ow { color: #007020; font-weight: bold } /* Operator.Word */
+.w { color: #bbbbbb } /* Text.Whitespace */
+.mf { color: #40a070 } /* Literal.Number.Float */
+.mh { color: #40a070 } /* Literal.Number.Hex */
+.mi { color: #40a070 } /* Literal.Number.Integer */
+.mo { color: #40a070 } /* Literal.Number.Oct */
+.sb { color: #4070a0 } /* Literal.String.Backtick */
+.sc { color: #4070a0 } /* Literal.String.Char */
+.sd { color: #4070a0; font-style: italic } /* Literal.String.Doc */
+.s2 { color: #4070a0 } /* Literal.String.Double */
+.se { color: #4070a0; font-weight: bold } /* Literal.String.Escape */
+.sh { color: #4070a0 } /* Literal.String.Heredoc */
+.si { color: #70a0d0; font-style: italic } /* Literal.String.Interpol */
+.sx { color: #c65d09 } /* Literal.String.Other */
+.sr { color: #235388 } /* Literal.String.Regex */
+.s1 { color: #4070a0 } /* Literal.String.Single */
+.ss { color: #517918 } /* Literal.String.Symbol */
+.bp { color: #007020 } /* Name.Builtin.Pseudo */
+.vc { color: #bb60d5 } /* Name.Variable.Class */
+.vg { color: #bb60d5 } /* Name.Variable.Global */
+.vi { color: #bb60d5 } /* Name.Variable.Instance */
+.il { color: #40a070 } /* Literal.Number.Integer.Long */
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/img/algorithms-reference-figure-1-computation-of-quartiles-median-and-interquartile-mean.png
----------------------------------------------------------------------
diff --git a/img/algorithms-reference-figure-1-computation-of-quartiles-median-and-interquartile-mean.png b/img/algorithms-reference-figure-1-computation-of-quartiles-median-and-interquartile-mean.png
new file mode 100644
index 0000000..e53fbb7
Binary files /dev/null and b/img/algorithms-reference-figure-1-computation-of-quartiles-median-and-interquartile-mean.png differ
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/index.md
----------------------------------------------------------------------
diff --git a/index.md b/index.md
new file mode 100644
index 0000000..1a8690c
--- /dev/null
+++ b/index.md
@@ -0,0 +1,22 @@
+---
+layout: global
+displayTitle: SystemML Overview
+title: SystemML Overview
+description: SystemML documentation homepage
+---
+
+SystemML is a flexible, scalable machine learning (ML) library written in Java.
+SystemML's distinguishing characteristics are: (1) algorithm customizability,
+(2) multiple execution modes, including Standalone, Hadoop Batch, and Spark Batch,
+and (3) automatic optimization.
+
+
+## SystemML Documentation
+
+For more information about SystemML, please consult the following references:
+
+* [SystemML GitHub README](http://www.github.com/SparkTC/systemml)
+* Quick Start - **Coming Soon**
+* [Algorithms Reference](algorithms-reference.html)
+* R-Like Declarative Machine Learning (DML) Language Reference - **Coming Soon**
+* Python-Like Declarative Machine Learning (DML) Language Reference - **Coming Soon**
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/js/main.js
----------------------------------------------------------------------
diff --git a/js/main.js b/js/main.js
new file mode 100644
index 0000000..f5d66b1
--- /dev/null
+++ b/js/main.js
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/* Custom JavaScript code in the MarkDown docs */
+
+// Enable language-specific code tabs
+function codeTabs() {
+ var counter = 0;
+ var langImages = {
+ "scala": "img/scala-sm.png",
+ "python": "img/python-sm.png",
+ "java": "img/java-sm.png"
+ };
+ $("div.codetabs").each(function() {
+ $(this).addClass("tab-content");
+
+ // Insert the tab bar
+ var tabBar = $('<ul class="nav nav-tabs" data-tabs="tabs"></ul>');
+ $(this).before(tabBar);
+
+ // Add each code sample to the tab bar:
+ var codeSamples = $(this).children("div");
+ codeSamples.each(function() {
+ $(this).addClass("tab-pane");
+ var lang = $(this).data("lang");
+ var image = $(this).data("image");
+ var notabs = $(this).data("notabs");
+ var capitalizedLang = lang.substr(0, 1).toUpperCase() + lang.substr(1);
+ var id = "tab_" + lang + "_" + counter;
+ $(this).attr("id", id);
+ if (image != null && langImages[lang]) {
+ var buttonLabel = "<img src='" +langImages[lang] + "' alt='" + capitalizedLang + "' />";
+ } else if (notabs == null) {
+ var buttonLabel = "<b>" + capitalizedLang + "</b>";
+ } else {
+ var buttonLabel = ""
+ }
+ tabBar.append(
+ '<li><a class="tab_' + lang + '" href="#' + id + '">' + buttonLabel + '</a></li>'
+ );
+ });
+
+ codeSamples.first().addClass("active");
+ tabBar.children("li").first().addClass("active");
+ counter++;
+ });
+ $("ul.nav-tabs a").click(function (e) {
+ // Toggling a tab should switch all tabs corresponding to the same language
+ // while retaining the scroll position
+ e.preventDefault();
+ var scrollOffset = $(this).offset().top - $(document).scrollTop();
+ $("." + $(this).attr('class')).tab('show');
+ $(document).scrollTop($(this).offset().top - scrollOffset);
+ });
+}
+
+
+// A script to fix internal hash links because we have an overlapping top bar.
+// Based on https://github.com/twitter/bootstrap/issues/193#issuecomment-2281510
+function maybeScrollToHash() {
+ if (window.location.hash && $(window.location.hash).length) {
+ var newTop = $(window.location.hash).offset().top - 57;
+ $(window).scrollTop(newTop);
+ }
+}
+
+$(function() {
+ codeTabs();
+ // Display anchor links when hovering over headers. For documentation of the
+ // configuration options, see the AnchorJS documentation.
+ anchors.options = {
+ placement: 'left'
+ };
+ anchors.add();
+
+ $(window).bind('hashchange', function() {
+ maybeScrollToHash();
+ });
+
+ // Scroll now too in case we had opened the page on a hash, but wait a bit because some browsers
+ // will try to do *their* initial scroll after running the onReady handler.
+ $(window).load(function() { setTimeout(function() { maybeScrollToHash(); }, 25); });
+});
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/js/vendor/anchor.min.js
----------------------------------------------------------------------
diff --git a/js/vendor/anchor.min.js b/js/vendor/anchor.min.js
new file mode 100644
index 0000000..68c3cb7
--- /dev/null
+++ b/js/vendor/anchor.min.js
@@ -0,0 +1,6 @@
+/*!
+ * AnchorJS - v1.1.1 - 2015-05-23
+ * https://github.com/bryanbraun/anchorjs
+ * Copyright (c) 2015 Bryan Braun; Licensed MIT
+ */
+function AnchorJS(A){"use strict";this.options=A||{},this._applyRemainingDefaultOptions=function(A){this.options.icon=this.options.hasOwnProperty("icon")?A.icon:"",this.options.visible=this.options.hasOwnProperty("visible")?A.visible:"hover",this.options.placement=this.options.hasOwnProperty("placement")?A.placement:"right",this.options.class=this.options.hasOwnProperty("class")?A.class:""},this._applyRemainingDefaultOptions(A),this.add=function(A){var e,t,o,n,i,s,a,l,c,r,h,g,B,Q;if(this._applyRemainingDefaultOptions(this.options),A){if("string"!=typeof A)throw new Error("The selector provided to AnchorJS was invalid.")}else A="h1, h2, h3, h4, h5, h6";if(e=document.querySelectorAll(A),0===e.length)return!1;for(this._addBaselineStyles(),t=document.querySelectorAll("[id]"),o=[].map.call(t,function(A){return A.id}),i=0;i<e.length;i++){if(e[i].hasAttribute("id"))n=e[i].getAttribute("id");else{s=e[i].textContent,a=s.replace(/[^\w\s-]/gi,"").replace(/\s+/g,"-").replace(/-{2,}/g,"-"
).substring(0,64).replace(/^-+|-+$/gm,"").toLowerCase(),r=a,c=0;do void 0!==l&&(r=a+"-"+c),l=o.indexOf(r),c+=1;while(-1!==l);l=void 0,o.push(r),e[i].setAttribute("id",r),n=r}h=n.replace(/-/g," "),g='<a class="anchorjs-link '+this.options.class+'" href="#'+n+'" aria-label="Anchor link for: '+h+'" data-anchorjs-icon="'+this.options.icon+'"></a>',B=document.createElement("div"),B.innerHTML=g,Q=B.childNodes,"always"===this.options.visible&&(Q[0].style.opacity="1"),""===this.options.icon&&(Q[0].style.fontFamily="anchorjs-icons",Q[0].style.fontStyle="normal",Q[0].style.fontVariant="normal",Q[0].style.fontWeight="normal"),"left"===this.options.placement?(Q[0].style.position="absolute",Q[0].style.marginLeft="-1em",Q[0].style.paddingRight="0.5em",e[i].insertBefore(Q[0],e[i].firstChild)):(Q[0].style.paddingLeft="0.375em",e[i].appendChild(Q[0]))}return this},this.remove=function(A){for(var e,t=document.querySelectorAll(A),o=0;o<t.length;o++)e=t[o].querySelector(".anchorjs-link"),e&&t[o]
.removeChild(e);return this},this._addBaselineStyles=function(){if(null===document.head.querySelector("style.anchorjs")){var A,e=document.createElement("style"),t=" .anchorjs-link { opacity: 0; text-decoration: none; -webkit-font-smoothing: antialiased; -moz-osx-font-smoothing: grayscale; }",o=" *:hover > .anchorjs-link, .anchorjs-link:focus { opacity: 1; }",n=' @font-face { font-family: "anchorjs-icons"; font-style: normal; font-weight: normal; src: url(data:application/x-font-ttf;charset=utf-8;base64,AAEAAAALAIAAAwAwT1MvMg8SBTUAAAC8AAAAYGNtYXAWi9QdAAABHAAAAFRnYXNwAAAAEAAAAXAAAAAIZ2x5Zgq29TcAAAF4AAABNGhlYWQEZM3pAAACrAAAADZoaGVhBhUDxgAAAuQAAAAkaG10eASAADEAAAMIAAAAFGxvY2EAKACuAAADHAAAAAxtYXhwAAgAVwAAAygAAAAgbmFtZQ5yJ3cAAANIAAAB2nBvc3QAAwAAAAAFJAAAACAAAwJAAZAABQAAApkCzAAAAI8CmQLMAAAB6wAzAQkAAAAAAAAAAAAAAAAAAAABEAAAAAAAAAAAAAAAAAAAAABAAADpywPA/8AAQAPAAEAAAAABAAAAAAAAAAAAAAAgAAAAAAADAAAAAwAAABwAAQADAAAAHAADAAEAAAAcAAQAOAAAAAoACAACAAIAAQAg6cv//f//AAAAAAAg6cv//f//AAH/4x
Y5AAMAAQAAAAAAAAAAAAAAAQAB//8ADwABAAAAAAAAAAAAAgAANzkBAAAAAAEAAAAAAAAAAAACAAA3OQEAAAAAAQAAAAAAAAAAAAIAADc5AQAAAAACADEARAJTAsAAKwBUAAABIiYnJjQ/AT4BMzIWFxYUDwEGIicmND8BNjQnLgEjIgYPAQYUFxYUBw4BIwciJicmND8BNjIXFhQPAQYUFx4BMzI2PwE2NCcmNDc2MhcWFA8BDgEjARQGDAUtLXoWOR8fORYtLTgKGwoKCjgaGg0gEhIgDXoaGgkJBQwHdR85Fi0tOAobCgoKOBoaDSASEiANehoaCQkKGwotLXoWOR8BMwUFLYEuehYXFxYugC44CQkKGwo4GkoaDQ0NDXoaShoKGwoFBe8XFi6ALjgJCQobCjgaShoNDQ0NehpKGgobCgoKLYEuehYXAAEAAAABAACiToc1Xw889QALBAAAAAAA0XnFFgAAAADRecUWAAAAAAJTAsAAAAAIAAIAAAAAAAAAAQAAA8D/wAAABAAAAAAAAlMAAQAAAAAAAAAAAAAAAAAAAAUAAAAAAAAAAAAAAAACAAAAAoAAMQAAAAAACgAUAB4AmgABAAAABQBVAAIAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAADgCuAAEAAAAAAAEADgAAAAEAAAAAAAIABwCfAAEAAAAAAAMADgBLAAEAAAAAAAQADgC0AAEAAAAAAAUACwAqAAEAAAAAAAYADgB1AAEAAAAAAAoAGgDeAAMAAQQJAAEAHAAOAAMAAQQJAAIADgCmAAMAAQQJAAMAHABZAAMAAQQJAAQAHADCAAMAAQQJAAUAFgA1AAMAAQQJAAYAHACDAAMAAQQJAAoANAD4YW5jaG9yanMtaWNvbnMAYQBuAGMAaABvAHIAagBzAC0AaQBjAG8AbgBzVmVyc2lvbiAxLjAAVgBlAHIAcwBpAG8AbgAgADEALgAwYW5jaG9yanMtaWN
vbnMAYQBuAGMAaABvAHIAagBzAC0AaQBjAG8AbgBzYW5jaG9yanMtaWNvbnMAYQBuAGMAaABvAHIAagBzAC0AaQBjAG8AbgBzUmVndWxhcgBSAGUAZwB1AGwAYQByYW5jaG9yanMtaWNvbnMAYQBuAGMAaABvAHIAagBzAC0AaQBjAG8AbgBzRm9udCBnZW5lcmF0ZWQgYnkgSWNvTW9vbi4ARgBvAG4AdAAgAGcAZQBuAGUAcgBhAHQAZQBkACAAYgB5ACAASQBjAG8ATQBvAG8AbgAuAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==) format("truetype"); }',i=" [data-anchorjs-icon]::after { content: attr(data-anchorjs-icon); }";e.className="anchorjs",e.appendChild(document.createTextNode("")),A=document.head.querySelector('[rel="stylesheet"], style'),void 0===A?document.head.appendChild(e):document.head.insertBefore(e,A),e.sheet.insertRule(t,e.sheet.cssRules.length),e.sheet.insertRule(o,e.sheet.cssRules.length),e.sheet.insertRule(i,e.sheet.cssRules.length),e.sheet.insertRule(n,e.sheet.cssRules.length)}}}var anchors=new AnchorJS;
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/js/vendor/bootstrap.js
----------------------------------------------------------------------
diff --git a/js/vendor/bootstrap.js b/js/vendor/bootstrap.js
new file mode 100644
index 0000000..7f303eb
--- /dev/null
+++ b/js/vendor/bootstrap.js
@@ -0,0 +1,2027 @@
+/* ===================================================
+ * bootstrap-transition.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#transitions
+ * ===================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ========================================================== */
+
+
+!function ($) {
+
+ $(function () {
+
+ "use strict"; // jshint ;_;
+
+
+ /* CSS TRANSITION SUPPORT (http://www.modernizr.com/)
+ * ======================================================= */
+
+ $.support.transition = (function () {
+
+ var transitionEnd = (function () {
+
+ var el = document.createElement('bootstrap')
+ , transEndEventNames = {
+ 'WebkitTransition' : 'webkitTransitionEnd'
+ , 'MozTransition' : 'transitionend'
+ , 'OTransition' : 'oTransitionEnd otransitionend'
+ , 'transition' : 'transitionend'
+ }
+ , name
+
+ for (name in transEndEventNames){
+ if (el.style[name] !== undefined) {
+ return transEndEventNames[name]
+ }
+ }
+
+ }())
+
+ return transitionEnd && {
+ end: transitionEnd
+ }
+
+ })()
+
+ })
+
+}(window.jQuery);/* ==========================================================
+ * bootstrap-alert.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#alerts
+ * ==========================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ========================================================== */
+
+
+!function ($) {
+
+ "use strict"; // jshint ;_;
+
+
+ /* ALERT CLASS DEFINITION
+ * ====================== */
+
+ var dismiss = '[data-dismiss="alert"]'
+ , Alert = function (el) {
+ $(el).on('click', dismiss, this.close)
+ }
+
+ Alert.prototype.close = function (e) {
+ var $this = $(this)
+ , selector = $this.attr('data-target')
+ , $parent
+
+ if (!selector) {
+ selector = $this.attr('href')
+ selector = selector && selector.replace(/.*(?=#[^\s]*$)/, '') //strip for ie7
+ }
+
+ $parent = $(selector)
+
+ e && e.preventDefault()
+
+ $parent.length || ($parent = $this.hasClass('alert') ? $this : $this.parent())
+
+ $parent.trigger(e = $.Event('close'))
+
+ if (e.isDefaultPrevented()) return
+
+ $parent.removeClass('in')
+
+ function removeElement() {
+ $parent
+ .trigger('closed')
+ .remove()
+ }
+
+ $.support.transition && $parent.hasClass('fade') ?
+ $parent.on($.support.transition.end, removeElement) :
+ removeElement()
+ }
+
+
+ /* ALERT PLUGIN DEFINITION
+ * ======================= */
+
+ $.fn.alert = function (option) {
+ return this.each(function () {
+ var $this = $(this)
+ , data = $this.data('alert')
+ if (!data) $this.data('alert', (data = new Alert(this)))
+ if (typeof option == 'string') data[option].call($this)
+ })
+ }
+
+ $.fn.alert.Constructor = Alert
+
+
+ /* ALERT DATA-API
+ * ============== */
+
+ $(function () {
+ $('body').on('click.alert.data-api', dismiss, Alert.prototype.close)
+ })
+
+}(window.jQuery);/* ============================================================
+ * bootstrap-button.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#buttons
+ * ============================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ============================================================ */
+
+
+!function ($) {
+
+ "use strict"; // jshint ;_;
+
+
+ /* BUTTON PUBLIC CLASS DEFINITION
+ * ============================== */
+
+ var Button = function (element, options) {
+ this.$element = $(element)
+ this.options = $.extend({}, $.fn.button.defaults, options)
+ }
+
+ Button.prototype.setState = function (state) {
+ var d = 'disabled'
+ , $el = this.$element
+ , data = $el.data()
+ , val = $el.is('input') ? 'val' : 'html'
+
+ state = state + 'Text'
+ data.resetText || $el.data('resetText', $el[val]())
+
+ $el[val](data[state] || this.options[state])
+
+ // push to event loop to allow forms to submit
+ setTimeout(function () {
+ state == 'loadingText' ?
+ $el.addClass(d).attr(d, d) :
+ $el.removeClass(d).removeAttr(d)
+ }, 0)
+ }
+
+ Button.prototype.toggle = function () {
+ var $parent = this.$element.parent('[data-toggle="buttons-radio"]')
+
+ $parent && $parent
+ .find('.active')
+ .removeClass('active')
+
+ this.$element.toggleClass('active')
+ }
+
+
+ /* BUTTON PLUGIN DEFINITION
+ * ======================== */
+
+ $.fn.button = function (option) {
+ return this.each(function () {
+ var $this = $(this)
+ , data = $this.data('button')
+ , options = typeof option == 'object' && option
+ if (!data) $this.data('button', (data = new Button(this, options)))
+ if (option == 'toggle') data.toggle()
+ else if (option) data.setState(option)
+ })
+ }
+
+ $.fn.button.defaults = {
+ loadingText: 'loading...'
+ }
+
+ $.fn.button.Constructor = Button
+
+
+ /* BUTTON DATA-API
+ * =============== */
+
+ $(function () {
+ $('body').on('click.button.data-api', '[data-toggle^=button]', function ( e ) {
+ var $btn = $(e.target)
+ if (!$btn.hasClass('btn')) $btn = $btn.closest('.btn')
+ $btn.button('toggle')
+ })
+ })
+
+}(window.jQuery);/* ==========================================================
+ * bootstrap-carousel.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#carousel
+ * ==========================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ========================================================== */
+
+
+!function ($) {
+
+ "use strict"; // jshint ;_;
+
+
+ /* CAROUSEL CLASS DEFINITION
+ * ========================= */
+
+ var Carousel = function (element, options) {
+ this.$element = $(element)
+ this.options = options
+ this.options.slide && this.slide(this.options.slide)
+ this.options.pause == 'hover' && this.$element
+ .on('mouseenter', $.proxy(this.pause, this))
+ .on('mouseleave', $.proxy(this.cycle, this))
+ }
+
+ Carousel.prototype = {
+
+ cycle: function (e) {
+ if (!e) this.paused = false
+ this.options.interval
+ && !this.paused
+ && (this.interval = setInterval($.proxy(this.next, this), this.options.interval))
+ return this
+ }
+
+ , to: function (pos) {
+ var $active = this.$element.find('.item.active')
+ , children = $active.parent().children()
+ , activePos = children.index($active)
+ , that = this
+
+ if (pos > (children.length - 1) || pos < 0) return
+
+ if (this.sliding) {
+ return this.$element.one('slid', function () {
+ that.to(pos)
+ })
+ }
+
+ if (activePos == pos) {
+ return this.pause().cycle()
+ }
+
+ return this.slide(pos > activePos ? 'next' : 'prev', $(children[pos]))
+ }
+
+ , pause: function (e) {
+ if (!e) this.paused = true
+ if (this.$element.find('.next, .prev').length && $.support.transition.end) {
+ this.$element.trigger($.support.transition.end)
+ this.cycle()
+ }
+ clearInterval(this.interval)
+ this.interval = null
+ return this
+ }
+
+ , next: function () {
+ if (this.sliding) return
+ return this.slide('next')
+ }
+
+ , prev: function () {
+ if (this.sliding) return
+ return this.slide('prev')
+ }
+
+ , slide: function (type, next) {
+ var $active = this.$element.find('.item.active')
+ , $next = next || $active[type]()
+ , isCycling = this.interval
+ , direction = type == 'next' ? 'left' : 'right'
+ , fallback = type == 'next' ? 'first' : 'last'
+ , that = this
+ , e = $.Event('slide', {
+ relatedTarget: $next[0]
+ })
+
+ this.sliding = true
+
+ isCycling && this.pause()
+
+ $next = $next.length ? $next : this.$element.find('.item')[fallback]()
+
+ if ($next.hasClass('active')) return
+
+ if ($.support.transition && this.$element.hasClass('slide')) {
+ this.$element.trigger(e)
+ if (e.isDefaultPrevented()) return
+ $next.addClass(type)
+ $next[0].offsetWidth // force reflow
+ $active.addClass(direction)
+ $next.addClass(direction)
+ this.$element.one($.support.transition.end, function () {
+ $next.removeClass([type, direction].join(' ')).addClass('active')
+ $active.removeClass(['active', direction].join(' '))
+ that.sliding = false
+ setTimeout(function () { that.$element.trigger('slid') }, 0)
+ })
+ } else {
+ this.$element.trigger(e)
+ if (e.isDefaultPrevented()) return
+ $active.removeClass('active')
+ $next.addClass('active')
+ this.sliding = false
+ this.$element.trigger('slid')
+ }
+
+ isCycling && this.cycle()
+
+ return this
+ }
+
+ }
+
+
+ /* CAROUSEL PLUGIN DEFINITION
+ * ========================== */
+
+ $.fn.carousel = function (option) {
+ return this.each(function () {
+ var $this = $(this)
+ , data = $this.data('carousel')
+ , options = $.extend({}, $.fn.carousel.defaults, typeof option == 'object' && option)
+ , action = typeof option == 'string' ? option : options.slide
+ if (!data) $this.data('carousel', (data = new Carousel(this, options)))
+ if (typeof option == 'number') data.to(option)
+ else if (action) data[action]()
+ else if (options.interval) data.cycle()
+ })
+ }
+
+ $.fn.carousel.defaults = {
+ interval: 5000
+ , pause: 'hover'
+ }
+
+ $.fn.carousel.Constructor = Carousel
+
+
+ /* CAROUSEL DATA-API
+ * ================= */
+
+ $(function () {
+ $('body').on('click.carousel.data-api', '[data-slide]', function ( e ) {
+ var $this = $(this), href
+ , $target = $($this.attr('data-target') || (href = $this.attr('href')) && href.replace(/.*(?=#[^\s]+$)/, '')) //strip for ie7
+ , options = !$target.data('modal') && $.extend({}, $target.data(), $this.data())
+ $target.carousel(options)
+ e.preventDefault()
+ })
+ })
+
+}(window.jQuery);/* =============================================================
+ * bootstrap-collapse.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#collapse
+ * =============================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ============================================================ */
+
+
+!function ($) {
+
+ "use strict"; // jshint ;_;
+
+
+ /* COLLAPSE PUBLIC CLASS DEFINITION
+ * ================================ */
+
+ var Collapse = function (element, options) {
+ this.$element = $(element)
+ this.options = $.extend({}, $.fn.collapse.defaults, options)
+
+ if (this.options.parent) {
+ this.$parent = $(this.options.parent)
+ }
+
+ this.options.toggle && this.toggle()
+ }
+
+ Collapse.prototype = {
+
+ constructor: Collapse
+
+ , dimension: function () {
+ var hasWidth = this.$element.hasClass('width')
+ return hasWidth ? 'width' : 'height'
+ }
+
+ , show: function () {
+ var dimension
+ , scroll
+ , actives
+ , hasData
+
+ if (this.transitioning) return
+
+ dimension = this.dimension()
+ scroll = $.camelCase(['scroll', dimension].join('-'))
+ actives = this.$parent && this.$parent.find('> .accordion-group > .in')
+
+ if (actives && actives.length) {
+ hasData = actives.data('collapse')
+ if (hasData && hasData.transitioning) return
+ actives.collapse('hide')
+ hasData || actives.data('collapse', null)
+ }
+
+ this.$element[dimension](0)
+ this.transition('addClass', $.Event('show'), 'shown')
+ $.support.transition && this.$element[dimension](this.$element[0][scroll])
+ }
+
+ , hide: function () {
+ var dimension
+ if (this.transitioning) return
+ dimension = this.dimension()
+ this.reset(this.$element[dimension]())
+ this.transition('removeClass', $.Event('hide'), 'hidden')
+ this.$element[dimension](0)
+ }
+
+ , reset: function (size) {
+ var dimension = this.dimension()
+
+ this.$element
+ .removeClass('collapse')
+ [dimension](size || 'auto')
+ [0].offsetWidth
+
+ this.$element[size !== null ? 'addClass' : 'removeClass']('collapse')
+
+ return this
+ }
+
+ , transition: function (method, startEvent, completeEvent) {
+ var that = this
+ , complete = function () {
+ if (startEvent.type == 'show') that.reset()
+ that.transitioning = 0
+ that.$element.trigger(completeEvent)
+ }
+
+ this.$element.trigger(startEvent)
+
+ if (startEvent.isDefaultPrevented()) return
+
+ this.transitioning = 1
+
+ this.$element[method]('in')
+
+ $.support.transition && this.$element.hasClass('collapse') ?
+ this.$element.one($.support.transition.end, complete) :
+ complete()
+ }
+
+ , toggle: function () {
+ this[this.$element.hasClass('in') ? 'hide' : 'show']()
+ }
+
+ }
+
+
+ /* COLLAPSIBLE PLUGIN DEFINITION
+ * ============================== */
+
+ $.fn.collapse = function (option) {
+ return this.each(function () {
+ var $this = $(this)
+ , data = $this.data('collapse')
+ , options = typeof option == 'object' && option
+ if (!data) $this.data('collapse', (data = new Collapse(this, options)))
+ if (typeof option == 'string') data[option]()
+ })
+ }
+
+ $.fn.collapse.defaults = {
+ toggle: true
+ }
+
+ $.fn.collapse.Constructor = Collapse
+
+
+ /* COLLAPSIBLE DATA-API
+ * ==================== */
+
+ $(function () {
+ $('body').on('click.collapse.data-api', '[data-toggle=collapse]', function (e) {
+ var $this = $(this), href
+ , target = $this.attr('data-target')
+ || e.preventDefault()
+ || (href = $this.attr('href')) && href.replace(/.*(?=#[^\s]+$)/, '') //strip for ie7
+ , option = $(target).data('collapse') ? 'toggle' : $this.data()
+ $this[$(target).hasClass('in') ? 'addClass' : 'removeClass']('collapsed')
+ $(target).collapse(option)
+ })
+ })
+
+}(window.jQuery);/* ============================================================
+ * bootstrap-dropdown.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#dropdowns
+ * ============================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ============================================================ */
+
+
+!function ($) {
+
+ "use strict"; // jshint ;_;
+
+
+ /* DROPDOWN CLASS DEFINITION
+ * ========================= */
+
+ var toggle = '[data-toggle=dropdown]'
+ , Dropdown = function (element) {
+ var $el = $(element).on('click.dropdown.data-api', this.toggle)
+ $('html').on('click.dropdown.data-api', function () {
+ $el.parent().removeClass('open')
+ })
+ }
+
+ Dropdown.prototype = {
+
+ constructor: Dropdown
+
+ , toggle: function (e) {
+ var $this = $(this)
+ , $parent
+ , isActive
+
+ if ($this.is('.disabled, :disabled')) return
+
+ $parent = getParent($this)
+
+ isActive = $parent.hasClass('open')
+
+ clearMenus()
+
+ if (!isActive) {
+ $parent.toggleClass('open')
+ $this.focus()
+ }
+
+ return false
+ }
+
+ , keydown: function (e) {
+ var $this
+ , $items
+ , $active
+ , $parent
+ , isActive
+ , index
+
+ if (!/(38|40|27)/.test(e.keyCode)) return
+
+ $this = $(this)
+
+ e.preventDefault()
+ e.stopPropagation()
+
+ if ($this.is('.disabled, :disabled')) return
+
+ $parent = getParent($this)
+
+ isActive = $parent.hasClass('open')
+
+ if (!isActive || (isActive && e.keyCode == 27)) return $this.click()
+
+ $items = $('[role=menu] li:not(.divider) a', $parent)
+
+ if (!$items.length) return
+
+ index = $items.index($items.filter(':focus'))
+
+ if (e.keyCode == 38 && index > 0) index-- // up
+ if (e.keyCode == 40 && index < $items.length - 1) index++ // down
+ if (!~index) index = 0
+
+ $items
+ .eq(index)
+ .focus()
+ }
+
+ }
+
+ function clearMenus() {
+ getParent($(toggle))
+ .removeClass('open')
+ }
+
+ function getParent($this) {
+ var selector = $this.attr('data-target')
+ , $parent
+
+ if (!selector) {
+ selector = $this.attr('href')
+ selector = selector && selector.replace(/.*(?=#[^\s]*$)/, '') //strip for ie7
+ }
+
+ $parent = $(selector)
+ $parent.length || ($parent = $this.parent())
+
+ return $parent
+ }
+
+
+ /* DROPDOWN PLUGIN DEFINITION
+ * ========================== */
+
+ $.fn.dropdown = function (option) {
+ return this.each(function () {
+ var $this = $(this)
+ , data = $this.data('dropdown')
+ if (!data) $this.data('dropdown', (data = new Dropdown(this)))
+ if (typeof option == 'string') data[option].call($this)
+ })
+ }
+
+ $.fn.dropdown.Constructor = Dropdown
+
+
+ /* APPLY TO STANDARD DROPDOWN ELEMENTS
+ * =================================== */
+
+ $(function () {
+ $('html')
+ .on('click.dropdown.data-api touchstart.dropdown.data-api', clearMenus)
+ $('body')
+ .on('click.dropdown touchstart.dropdown.data-api', '.dropdown', function (e) { e.stopPropagation() })
+ .on('click.dropdown.data-api touchstart.dropdown.data-api' , toggle, Dropdown.prototype.toggle)
+ .on('keydown.dropdown.data-api touchstart.dropdown.data-api', toggle + ', [role=menu]' , Dropdown.prototype.keydown)
+ })
+
+}(window.jQuery);/* =========================================================
+ * bootstrap-modal.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#modals
+ * =========================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ========================================================= */
+
+
+!function ($) {
+
+ "use strict"; // jshint ;_;
+
+
+ /* MODAL CLASS DEFINITION
+ * ====================== */
+
+ var Modal = function (element, options) {
+ this.options = options
+ this.$element = $(element)
+ .delegate('[data-dismiss="modal"]', 'click.dismiss.modal', $.proxy(this.hide, this))
+ this.options.remote && this.$element.find('.modal-body').load(this.options.remote)
+ }
+
+ Modal.prototype = {
+
+ constructor: Modal
+
+ , toggle: function () {
+ return this[!this.isShown ? 'show' : 'hide']()
+ }
+
+ , show: function () {
+ var that = this
+ , e = $.Event('show')
+
+ this.$element.trigger(e)
+
+ if (this.isShown || e.isDefaultPrevented()) return
+
+ $('body').addClass('modal-open')
+
+ this.isShown = true
+
+ this.escape()
+
+ this.backdrop(function () {
+ var transition = $.support.transition && that.$element.hasClass('fade')
+
+ if (!that.$element.parent().length) {
+ that.$element.appendTo(document.body) //don't move modals dom position
+ }
+
+ that.$element
+ .show()
+
+ if (transition) {
+ that.$element[0].offsetWidth // force reflow
+ }
+
+ that.$element
+ .addClass('in')
+ .attr('aria-hidden', false)
+ .focus()
+
+ that.enforceFocus()
+
+ transition ?
+ that.$element.one($.support.transition.end, function () { that.$element.trigger('shown') }) :
+ that.$element.trigger('shown')
+
+ })
+ }
+
+ , hide: function (e) {
+ e && e.preventDefault()
+
+ var that = this
+
+ e = $.Event('hide')
+
+ this.$element.trigger(e)
+
+ if (!this.isShown || e.isDefaultPrevented()) return
+
+ this.isShown = false
+
+ $('body').removeClass('modal-open')
+
+ this.escape()
+
+ $(document).off('focusin.modal')
+
+ this.$element
+ .removeClass('in')
+ .attr('aria-hidden', true)
+
+ $.support.transition && this.$element.hasClass('fade') ?
+ this.hideWithTransition() :
+ this.hideModal()
+ }
+
+ , enforceFocus: function () {
+ var that = this
+ $(document).on('focusin.modal', function (e) {
+ if (that.$element[0] !== e.target && !that.$element.has(e.target).length) {
+ that.$element.focus()
+ }
+ })
+ }
+
+ , escape: function () {
+ var that = this
+ if (this.isShown && this.options.keyboard) {
+ this.$element.on('keyup.dismiss.modal', function ( e ) {
+ e.which == 27 && that.hide()
+ })
+ } else if (!this.isShown) {
+ this.$element.off('keyup.dismiss.modal')
+ }
+ }
+
+ , hideWithTransition: function () {
+ var that = this
+ , timeout = setTimeout(function () {
+ that.$element.off($.support.transition.end)
+ that.hideModal()
+ }, 500)
+
+ this.$element.one($.support.transition.end, function () {
+ clearTimeout(timeout)
+ that.hideModal()
+ })
+ }
+
+ , hideModal: function (that) {
+ this.$element
+ .hide()
+ .trigger('hidden')
+
+ this.backdrop()
+ }
+
+ , removeBackdrop: function () {
+ this.$backdrop.remove()
+ this.$backdrop = null
+ }
+
+ , backdrop: function (callback) {
+ var that = this
+ , animate = this.$element.hasClass('fade') ? 'fade' : ''
+
+ if (this.isShown && this.options.backdrop) {
+ var doAnimate = $.support.transition && animate
+
+ this.$backdrop = $('<div class="modal-backdrop ' + animate + '" />')
+ .appendTo(document.body)
+
+ if (this.options.backdrop != 'static') {
+ this.$backdrop.click($.proxy(this.hide, this))
+ }
+
+ if (doAnimate) this.$backdrop[0].offsetWidth // force reflow
+
+ this.$backdrop.addClass('in')
+
+ doAnimate ?
+ this.$backdrop.one($.support.transition.end, callback) :
+ callback()
+
+ } else if (!this.isShown && this.$backdrop) {
+ this.$backdrop.removeClass('in')
+
+ $.support.transition && this.$element.hasClass('fade')?
+ this.$backdrop.one($.support.transition.end, $.proxy(this.removeBackdrop, this)) :
+ this.removeBackdrop()
+
+ } else if (callback) {
+ callback()
+ }
+ }
+ }
+
+
+ /* MODAL PLUGIN DEFINITION
+ * ======================= */
+
+ $.fn.modal = function (option) {
+ return this.each(function () {
+ var $this = $(this)
+ , data = $this.data('modal')
+ , options = $.extend({}, $.fn.modal.defaults, $this.data(), typeof option == 'object' && option)
+ if (!data) $this.data('modal', (data = new Modal(this, options)))
+ if (typeof option == 'string') data[option]()
+ else if (options.show) data.show()
+ })
+ }
+
+ $.fn.modal.defaults = {
+ backdrop: true
+ , keyboard: true
+ , show: true
+ }
+
+ $.fn.modal.Constructor = Modal
+
+
+ /* MODAL DATA-API
+ * ============== */
+
+ $(function () {
+ $('body').on('click.modal.data-api', '[data-toggle="modal"]', function ( e ) {
+ var $this = $(this)
+ , href = $this.attr('href')
+ , $target = $($this.attr('data-target') || (href && href.replace(/.*(?=#[^\s]+$)/, ''))) //strip for ie7
+ , option = $target.data('modal') ? 'toggle' : $.extend({ remote: !/#/.test(href) && href }, $target.data(), $this.data())
+
+ e.preventDefault()
+
+ $target
+ .modal(option)
+ .one('hide', function () {
+ $this.focus()
+ })
+ })
+ })
+
+}(window.jQuery);/* ===========================================================
+ * bootstrap-tooltip.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#tooltips
+ * Inspired by the original jQuery.tipsy by Jason Frame
+ * ===========================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ========================================================== */
+
+
+!function ($) {
+
+ "use strict"; // jshint ;_;
+
+
+ /* TOOLTIP PUBLIC CLASS DEFINITION
+ * =============================== */
+
+ var Tooltip = function (element, options) {
+ this.init('tooltip', element, options)
+ }
+
+ Tooltip.prototype = {
+
+ constructor: Tooltip
+
+ , init: function (type, element, options) {
+ var eventIn
+ , eventOut
+
+ this.type = type
+ this.$element = $(element)
+ this.options = this.getOptions(options)
+ this.enabled = true
+
+ if (this.options.trigger == 'click') {
+ this.$element.on('click.' + this.type, this.options.selector, $.proxy(this.toggle, this))
+ } else if (this.options.trigger != 'manual') {
+ eventIn = this.options.trigger == 'hover' ? 'mouseenter' : 'focus'
+ eventOut = this.options.trigger == 'hover' ? 'mouseleave' : 'blur'
+ this.$element.on(eventIn + '.' + this.type, this.options.selector, $.proxy(this.enter, this))
+ this.$element.on(eventOut + '.' + this.type, this.options.selector, $.proxy(this.leave, this))
+ }
+
+ this.options.selector ?
+ (this._options = $.extend({}, this.options, { trigger: 'manual', selector: '' })) :
+ this.fixTitle()
+ }
+
+ , getOptions: function (options) {
+ options = $.extend({}, $.fn[this.type].defaults, options, this.$element.data())
+
+ if (options.delay && typeof options.delay == 'number') {
+ options.delay = {
+ show: options.delay
+ , hide: options.delay
+ }
+ }
+
+ return options
+ }
+
+ , enter: function (e) {
+ var self = $(e.currentTarget)[this.type](this._options).data(this.type)
+
+ if (!self.options.delay || !self.options.delay.show) return self.show()
+
+ clearTimeout(this.timeout)
+ self.hoverState = 'in'
+ this.timeout = setTimeout(function() {
+ if (self.hoverState == 'in') self.show()
+ }, self.options.delay.show)
+ }
+
+ , leave: function (e) {
+ var self = $(e.currentTarget)[this.type](this._options).data(this.type)
+
+ if (this.timeout) clearTimeout(this.timeout)
+ if (!self.options.delay || !self.options.delay.hide) return self.hide()
+
+ self.hoverState = 'out'
+ this.timeout = setTimeout(function() {
+ if (self.hoverState == 'out') self.hide()
+ }, self.options.delay.hide)
+ }
+
+ , show: function () {
+ var $tip
+ , inside
+ , pos
+ , actualWidth
+ , actualHeight
+ , placement
+ , tp
+
+ if (this.hasContent() && this.enabled) {
+ $tip = this.tip()
+ this.setContent()
+
+ if (this.options.animation) {
+ $tip.addClass('fade')
+ }
+
+ placement = typeof this.options.placement == 'function' ?
+ this.options.placement.call(this, $tip[0], this.$element[0]) :
+ this.options.placement
+
+ inside = /in/.test(placement)
+
+ $tip
+ .remove()
+ .css({ top: 0, left: 0, display: 'block' })
+ .appendTo(inside ? this.$element : document.body)
+
+ pos = this.getPosition(inside)
+
+ actualWidth = $tip[0].offsetWidth
+ actualHeight = $tip[0].offsetHeight
+
+ switch (inside ? placement.split(' ')[1] : placement) {
+ case 'bottom':
+ tp = {top: pos.top + pos.height, left: pos.left + pos.width / 2 - actualWidth / 2}
+ break
+ case 'top':
+ tp = {top: pos.top - actualHeight, left: pos.left + pos.width / 2 - actualWidth / 2}
+ break
+ case 'left':
+ tp = {top: pos.top + pos.height / 2 - actualHeight / 2, left: pos.left - actualWidth}
+ break
+ case 'right':
+ tp = {top: pos.top + pos.height / 2 - actualHeight / 2, left: pos.left + pos.width}
+ break
+ }
+
+ $tip
+ .css(tp)
+ .addClass(placement)
+ .addClass('in')
+ }
+ }
+
+ , setContent: function () {
+ var $tip = this.tip()
+ , title = this.getTitle()
+
+ $tip.find('.tooltip-inner')[this.options.html ? 'html' : 'text'](title)
+ $tip.removeClass('fade in top bottom left right')
+ }
+
+ , hide: function () {
+ var that = this
+ , $tip = this.tip()
+
+ $tip.removeClass('in')
+
+ function removeWithAnimation() {
+ var timeout = setTimeout(function () {
+ $tip.off($.support.transition.end).remove()
+ }, 500)
+
+ $tip.one($.support.transition.end, function () {
+ clearTimeout(timeout)
+ $tip.remove()
+ })
+ }
+
+ $.support.transition && this.$tip.hasClass('fade') ?
+ removeWithAnimation() :
+ $tip.remove()
+
+ return this
+ }
+
+ , fixTitle: function () {
+ var $e = this.$element
+ if ($e.attr('title') || typeof($e.attr('data-original-title')) != 'string') {
+ $e.attr('data-original-title', $e.attr('title') || '').removeAttr('title')
+ }
+ }
+
+ , hasContent: function () {
+ return this.getTitle()
+ }
+
+ , getPosition: function (inside) {
+ return $.extend({}, (inside ? {top: 0, left: 0} : this.$element.offset()), {
+ width: this.$element[0].offsetWidth
+ , height: this.$element[0].offsetHeight
+ })
+ }
+
+ , getTitle: function () {
+ var title
+ , $e = this.$element
+ , o = this.options
+
+ title = $e.attr('data-original-title')
+ || (typeof o.title == 'function' ? o.title.call($e[0]) : o.title)
+
+ return title
+ }
+
+ , tip: function () {
+ return this.$tip = this.$tip || $(this.options.template)
+ }
+
+ , validate: function () {
+ if (!this.$element[0].parentNode) {
+ this.hide()
+ this.$element = null
+ this.options = null
+ }
+ }
+
+ , enable: function () {
+ this.enabled = true
+ }
+
+ , disable: function () {
+ this.enabled = false
+ }
+
+ , toggleEnabled: function () {
+ this.enabled = !this.enabled
+ }
+
+ , toggle: function () {
+ this[this.tip().hasClass('in') ? 'hide' : 'show']()
+ }
+
+ , destroy: function () {
+ this.hide().$element.off('.' + this.type).removeData(this.type)
+ }
+
+ }
+
+
+ /* TOOLTIP PLUGIN DEFINITION
+ * ========================= */
+
+ $.fn.tooltip = function ( option ) {
+ return this.each(function () {
+ var $this = $(this)
+ , data = $this.data('tooltip')
+ , options = typeof option == 'object' && option
+ if (!data) $this.data('tooltip', (data = new Tooltip(this, options)))
+ if (typeof option == 'string') data[option]()
+ })
+ }
+
+ $.fn.tooltip.Constructor = Tooltip
+
+ $.fn.tooltip.defaults = {
+ animation: true
+ , placement: 'top'
+ , selector: false
+ , template: '<div class="tooltip"><div class="tooltip-arrow"></div><div class="tooltip-inner"></div></div>'
+ , trigger: 'hover'
+ , title: ''
+ , delay: 0
+ , html: true
+ }
+
+}(window.jQuery);
+/* ===========================================================
+ * bootstrap-popover.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#popovers
+ * ===========================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * =========================================================== */
+
+
+!function ($) {
+
+ "use strict"; // jshint ;_;
+
+
+ /* POPOVER PUBLIC CLASS DEFINITION
+ * =============================== */
+
+ var Popover = function (element, options) {
+ this.init('popover', element, options)
+ }
+
+
+ /* NOTE: POPOVER EXTENDS BOOTSTRAP-TOOLTIP.js
+ ========================================== */
+
+ Popover.prototype = $.extend({}, $.fn.tooltip.Constructor.prototype, {
+
+ constructor: Popover
+
+ , setContent: function () {
+ var $tip = this.tip()
+ , title = this.getTitle()
+ , content = this.getContent()
+
+ $tip.find('.popover-title')[this.options.html ? 'html' : 'text'](title)
+ $tip.find('.popover-content > *')[this.options.html ? 'html' : 'text'](content)
+
+ $tip.removeClass('fade top bottom left right in')
+ }
+
+ , hasContent: function () {
+ return this.getTitle() || this.getContent()
+ }
+
+ , getContent: function () {
+ var content
+ , $e = this.$element
+ , o = this.options
+
+ content = $e.attr('data-content')
+ || (typeof o.content == 'function' ? o.content.call($e[0]) : o.content)
+
+ return content
+ }
+
+ , tip: function () {
+ if (!this.$tip) {
+ this.$tip = $(this.options.template)
+ }
+ return this.$tip
+ }
+
+ , destroy: function () {
+ this.hide().$element.off('.' + this.type).removeData(this.type)
+ }
+
+ })
+
+
+ /* POPOVER PLUGIN DEFINITION
+ * ======================= */
+
+ $.fn.popover = function (option) {
+ return this.each(function () {
+ var $this = $(this)
+ , data = $this.data('popover')
+ , options = typeof option == 'object' && option
+ if (!data) $this.data('popover', (data = new Popover(this, options)))
+ if (typeof option == 'string') data[option]()
+ })
+ }
+
+ $.fn.popover.Constructor = Popover
+
+ $.fn.popover.defaults = $.extend({} , $.fn.tooltip.defaults, {
+ placement: 'right'
+ , trigger: 'click'
+ , content: ''
+ , template: '<div class="popover"><div class="arrow"></div><div class="popover-inner"><h3 class="popover-title"></h3><div class="popover-content"><p></p></div></div></div>'
+ })
+
+}(window.jQuery);/* =============================================================
+ * bootstrap-scrollspy.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#scrollspy
+ * =============================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ============================================================== */
+
+
+!function ($) {
+
+ "use strict"; // jshint ;_;
+
+
+ /* SCROLLSPY CLASS DEFINITION
+ * ========================== */
+
+ function ScrollSpy(element, options) {
+ var process = $.proxy(this.process, this)
+ , $element = $(element).is('body') ? $(window) : $(element)
+ , href
+ this.options = $.extend({}, $.fn.scrollspy.defaults, options)
+ this.$scrollElement = $element.on('scroll.scroll-spy.data-api', process)
+ this.selector = (this.options.target
+ || ((href = $(element).attr('href')) && href.replace(/.*(?=#[^\s]+$)/, '')) //strip for ie7
+ || '') + ' .nav li > a'
+ this.$body = $('body')
+ this.refresh()
+ this.process()
+ }
+
+ ScrollSpy.prototype = {
+
+ constructor: ScrollSpy
+
+ , refresh: function () {
+ var self = this
+ , $targets
+
+ this.offsets = $([])
+ this.targets = $([])
+
+ $targets = this.$body
+ .find(this.selector)
+ .map(function () {
+ var $el = $(this)
+ , href = $el.data('target') || $el.attr('href')
+ , $href = /^#\w/.test(href) && $(href)
+ return ( $href
+ && $href.length
+ && [[ $href.position().top, href ]] ) || null
+ })
+ .sort(function (a, b) { return a[0] - b[0] })
+ .each(function () {
+ self.offsets.push(this[0])
+ self.targets.push(this[1])
+ })
+ }
+
+ , process: function () {
+ var scrollTop = this.$scrollElement.scrollTop() + this.options.offset
+ , scrollHeight = this.$scrollElement[0].scrollHeight || this.$body[0].scrollHeight
+ , maxScroll = scrollHeight - this.$scrollElement.height()
+ , offsets = this.offsets
+ , targets = this.targets
+ , activeTarget = this.activeTarget
+ , i
+
+ if (scrollTop >= maxScroll) {
+ return activeTarget != (i = targets.last()[0])
+ && this.activate ( i )
+ }
+
+ for (i = offsets.length; i--;) {
+ activeTarget != targets[i]
+ && scrollTop >= offsets[i]
+ && (!offsets[i + 1] || scrollTop <= offsets[i + 1])
+ && this.activate( targets[i] )
+ }
+ }
+
+ , activate: function (target) {
+ var active
+ , selector
+
+ this.activeTarget = target
+
+ $(this.selector)
+ .parent('.active')
+ .removeClass('active')
+
+ selector = this.selector
+ + '[data-target="' + target + '"],'
+ + this.selector + '[href="' + target + '"]'
+
+ active = $(selector)
+ .parent('li')
+ .addClass('active')
+
+ if (active.parent('.dropdown-menu').length) {
+ active = active.closest('li.dropdown').addClass('active')
+ }
+
+ active.trigger('activate')
+ }
+
+ }
+
+
+ /* SCROLLSPY PLUGIN DEFINITION
+ * =========================== */
+
+ $.fn.scrollspy = function (option) {
+ return this.each(function () {
+ var $this = $(this)
+ , data = $this.data('scrollspy')
+ , options = typeof option == 'object' && option
+ if (!data) $this.data('scrollspy', (data = new ScrollSpy(this, options)))
+ if (typeof option == 'string') data[option]()
+ })
+ }
+
+ $.fn.scrollspy.Constructor = ScrollSpy
+
+ $.fn.scrollspy.defaults = {
+ offset: 10
+ }
+
+
+ /* SCROLLSPY DATA-API
+ * ================== */
+
+ $(window).on('load', function () {
+ $('[data-spy="scroll"]').each(function () {
+ var $spy = $(this)
+ $spy.scrollspy($spy.data())
+ })
+ })
+
+}(window.jQuery);/* ========================================================
+ * bootstrap-tab.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#tabs
+ * ========================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ======================================================== */
+
+
+!function ($) {
+
+ "use strict"; // jshint ;_;
+
+
+ /* TAB CLASS DEFINITION
+ * ==================== */
+
+ var Tab = function (element) {
+ this.element = $(element)
+ }
+
+ Tab.prototype = {
+
+ constructor: Tab
+
+ , show: function () {
+ var $this = this.element
+ , $ul = $this.closest('ul:not(.dropdown-menu)')
+ , selector = $this.attr('data-target')
+ , previous
+ , $target
+ , e
+
+ if (!selector) {
+ selector = $this.attr('href')
+ selector = selector && selector.replace(/.*(?=#[^\s]*$)/, '') //strip for ie7
+ }
+
+ if ( $this.parent('li').hasClass('active') ) return
+
+ previous = $ul.find('.active a').last()[0]
+
+ e = $.Event('show', {
+ relatedTarget: previous
+ })
+
+ $this.trigger(e)
+
+ if (e.isDefaultPrevented()) return
+
+ $target = $(selector)
+
+ this.activate($this.parent('li'), $ul)
+ this.activate($target, $target.parent(), function () {
+ $this.trigger({
+ type: 'shown'
+ , relatedTarget: previous
+ })
+ })
+ }
+
+ , activate: function ( element, container, callback) {
+ var $active = container.find('> .active')
+ , transition = callback
+ && $.support.transition
+ && $active.hasClass('fade')
+
+ function next() {
+ $active
+ .removeClass('active')
+ .find('> .dropdown-menu > .active')
+ .removeClass('active')
+
+ element.addClass('active')
+
+ if (transition) {
+ element[0].offsetWidth // reflow for transition
+ element.addClass('in')
+ } else {
+ element.removeClass('fade')
+ }
+
+ if ( element.parent('.dropdown-menu') ) {
+ element.closest('li.dropdown').addClass('active')
+ }
+
+ callback && callback()
+ }
+
+ transition ?
+ $active.one($.support.transition.end, next) :
+ next()
+
+ $active.removeClass('in')
+ }
+ }
+
+
+ /* TAB PLUGIN DEFINITION
+ * ===================== */
+
+ $.fn.tab = function ( option ) {
+ return this.each(function () {
+ var $this = $(this)
+ , data = $this.data('tab')
+ if (!data) $this.data('tab', (data = new Tab(this)))
+ if (typeof option == 'string') data[option]()
+ })
+ }
+
+ $.fn.tab.Constructor = Tab
+
+
+ /* TAB DATA-API
+ * ============ */
+
+ $(function () {
+ $('body').on('click.tab.data-api', '[data-toggle="tab"], [data-toggle="pill"]', function (e) {
+ e.preventDefault()
+ $(this).tab('show')
+ })
+ })
+
+}(window.jQuery);/* =============================================================
+ * bootstrap-typeahead.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#typeahead
+ * =============================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ============================================================ */
+
+
+!function($){
+
+ "use strict"; // jshint ;_;
+
+
+ /* TYPEAHEAD PUBLIC CLASS DEFINITION
+ * ================================= */
+
+ var Typeahead = function (element, options) {
+ this.$element = $(element)
+ this.options = $.extend({}, $.fn.typeahead.defaults, options)
+ this.matcher = this.options.matcher || this.matcher
+ this.sorter = this.options.sorter || this.sorter
+ this.highlighter = this.options.highlighter || this.highlighter
+ this.updater = this.options.updater || this.updater
+ this.$menu = $(this.options.menu).appendTo('body')
+ this.source = this.options.source
+ this.shown = false
+ this.listen()
+ }
+
+ Typeahead.prototype = {
+
+ constructor: Typeahead
+
+ , select: function () {
+ var val = this.$menu.find('.active').attr('data-value')
+ this.$element
+ .val(this.updater(val))
+ .change()
+ return this.hide()
+ }
+
+ , updater: function (item) {
+ return item
+ }
+
+ , show: function () {
+ var pos = $.extend({}, this.$element.offset(), {
+ height: this.$element[0].offsetHeight
+ })
+
+ this.$menu.css({
+ top: pos.top + pos.height
+ , left: pos.left
+ })
+
+ this.$menu.show()
+ this.shown = true
+ return this
+ }
+
+ , hide: function () {
+ this.$menu.hide()
+ this.shown = false
+ return this
+ }
+
+ , lookup: function (event) {
+ var items
+
+ this.query = this.$element.val()
+
+ if (!this.query || this.query.length < this.options.minLength) {
+ return this.shown ? this.hide() : this
+ }
+
+ items = $.isFunction(this.source) ? this.source(this.query, $.proxy(this.process, this)) : this.source
+
+ return items ? this.process(items) : this
+ }
+
+ , process: function (items) {
+ var that = this
+
+ items = $.grep(items, function (item) {
+ return that.matcher(item)
+ })
+
+ items = this.sorter(items)
+
+ if (!items.length) {
+ return this.shown ? this.hide() : this
+ }
+
+ return this.render(items.slice(0, this.options.items)).show()
+ }
+
+ , matcher: function (item) {
+ return ~item.toLowerCase().indexOf(this.query.toLowerCase())
+ }
+
+ , sorter: function (items) {
+ var beginswith = []
+ , caseSensitive = []
+ , caseInsensitive = []
+ , item
+
+ while (item = items.shift()) {
+ if (!item.toLowerCase().indexOf(this.query.toLowerCase())) beginswith.push(item)
+ else if (~item.indexOf(this.query)) caseSensitive.push(item)
+ else caseInsensitive.push(item)
+ }
+
+ return beginswith.concat(caseSensitive, caseInsensitive)
+ }
+
+ , highlighter: function (item) {
+ var query = this.query.replace(/[\-\[\]{}()*+?.,\\\^$|#\s]/g, '\\$&')
+ return item.replace(new RegExp('(' + query + ')', 'ig'), function ($1, match) {
+ return '<strong>' + match + '</strong>'
+ })
+ }
+
+ , render: function (items) {
+ var that = this
+
+ items = $(items).map(function (i, item) {
+ i = $(that.options.item).attr('data-value', item)
+ i.find('a').html(that.highlighter(item))
+ return i[0]
+ })
+
+ items.first().addClass('active')
+ this.$menu.html(items)
+ return this
+ }
+
+ , next: function (event) {
+ var active = this.$menu.find('.active').removeClass('active')
+ , next = active.next()
+
+ if (!next.length) {
+ next = $(this.$menu.find('li')[0])
+ }
+
+ next.addClass('active')
+ }
+
+ , prev: function (event) {
+ var active = this.$menu.find('.active').removeClass('active')
+ , prev = active.prev()
+
+ if (!prev.length) {
+ prev = this.$menu.find('li').last()
+ }
+
+ prev.addClass('active')
+ }
+
+ , listen: function () {
+ this.$element
+ .on('blur', $.proxy(this.blur, this))
+ .on('keypress', $.proxy(this.keypress, this))
+ .on('keyup', $.proxy(this.keyup, this))
+
+ if ($.browser.webkit || $.browser.msie) {
+ this.$element.on('keydown', $.proxy(this.keydown, this))
+ }
+
+ this.$menu
+ .on('click', $.proxy(this.click, this))
+ .on('mouseenter', 'li', $.proxy(this.mouseenter, this))
+ }
+
+ , move: function (e) {
+ if (!this.shown) return
+
+ switch(e.keyCode) {
+ case 9: // tab
+ case 13: // enter
+ case 27: // escape
+ e.preventDefault()
+ break
+
+ case 38: // up arrow
+ e.preventDefault()
+ this.prev()
+ break
+
+ case 40: // down arrow
+ e.preventDefault()
+ this.next()
+ break
+ }
+
+ e.stopPropagation()
+ }
+
+ , keydown: function (e) {
+ this.suppressKeyPressRepeat = !~$.inArray(e.keyCode, [40,38,9,13,27])
+ this.move(e)
+ }
+
+ , keypress: function (e) {
+ if (this.suppressKeyPressRepeat) return
+ this.move(e)
+ }
+
+ , keyup: function (e) {
+ switch(e.keyCode) {
+ case 40: // down arrow
+ case 38: // up arrow
+ break
+
+ case 9: // tab
+ case 13: // enter
+ if (!this.shown) return
+ this.select()
+ break
+
+ case 27: // escape
+ if (!this.shown) return
+ this.hide()
+ break
+
+ default:
+ this.lookup()
+ }
+
+ e.stopPropagation()
+ e.preventDefault()
+ }
+
+ , blur: function (e) {
+ var that = this
+ setTimeout(function () { that.hide() }, 150)
+ }
+
+ , click: function (e) {
+ e.stopPropagation()
+ e.preventDefault()
+ this.select()
+ }
+
+ , mouseenter: function (e) {
+ this.$menu.find('.active').removeClass('active')
+ $(e.currentTarget).addClass('active')
+ }
+
+ }
+
+
+ /* TYPEAHEAD PLUGIN DEFINITION
+ * =========================== */
+
+ $.fn.typeahead = function (option) {
+ return this.each(function () {
+ var $this = $(this)
+ , data = $this.data('typeahead')
+ , options = typeof option == 'object' && option
+ if (!data) $this.data('typeahead', (data = new Typeahead(this, options)))
+ if (typeof option == 'string') data[option]()
+ })
+ }
+
+ $.fn.typeahead.defaults = {
+ source: []
+ , items: 8
+ , menu: '<ul class="typeahead dropdown-menu"></ul>'
+ , item: '<li><a href="#"></a></li>'
+ , minLength: 1
+ }
+
+ $.fn.typeahead.Constructor = Typeahead
+
+
+ /* TYPEAHEAD DATA-API
+ * ================== */
+
+ $(function () {
+ $('body').on('focus.typeahead.data-api', '[data-provide="typeahead"]', function (e) {
+ var $this = $(this)
+ if ($this.data('typeahead')) return
+ e.preventDefault()
+ $this.typeahead($this.data())
+ })
+ })
+
+}(window.jQuery);
+/* ==========================================================
+ * bootstrap-affix.js v2.1.0
+ * http://twitter.github.com/bootstrap/javascript.html#affix
+ * ==========================================================
+ * Copyright 2012 Twitter, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ * ========================================================== */
+
+
+!function ($) {
+
+ "use strict"; // jshint ;_;
+
+
+ /* AFFIX CLASS DEFINITION
+ * ====================== */
+
+ var Affix = function (element, options) {
+ this.options = $.extend({}, $.fn.affix.defaults, options)
+ this.$window = $(window).on('scroll.affix.data-api', $.proxy(this.checkPosition, this))
+ this.$element = $(element)
+ this.checkPosition()
+ }
+
+ Affix.prototype.checkPosition = function () {
+ if (!this.$element.is(':visible')) return
+
+ var scrollHeight = $(document).height()
+ , scrollTop = this.$window.scrollTop()
+ , position = this.$element.offset()
+ , offset = this.options.offset
+ , offsetBottom = offset.bottom
+ , offsetTop = offset.top
+ , reset = 'affix affix-top affix-bottom'
+ , affix
+
+ if (typeof offset != 'object') offsetBottom = offsetTop = offset
+ if (typeof offsetTop == 'function') offsetTop = offset.top()
+ if (typeof offsetBottom == 'function') offsetBottom = offset.bottom()
+
+ affix = this.unpin != null && (scrollTop + this.unpin <= position.top) ?
+ false : offsetBottom != null && (position.top + this.$element.height() >= scrollHeight - offsetBottom) ?
+ 'bottom' : offsetTop != null && scrollTop <= offsetTop ?
+ 'top' : false
+
+ if (this.affixed === affix) return
+
+ this.affixed = affix
+ this.unpin = affix == 'bottom' ? position.top - scrollTop : null
+
+ this.$element.removeClass(reset).addClass('affix' + (affix ? '-' + affix : ''))
+ }
+
+
+ /* AFFIX PLUGIN DEFINITION
+ * ======================= */
+
+ $.fn.affix = function (option) {
+ return this.each(function () {
+ var $this = $(this)
+ , data = $this.data('affix')
+ , options = typeof option == 'object' && option
+ if (!data) $this.data('affix', (data = new Affix(this, options)))
+ if (typeof option == 'string') data[option]()
+ })
+ }
+
+ $.fn.affix.Constructor = Affix
+
+ $.fn.affix.defaults = {
+ offset: 0
+ }
+
+
+ /* AFFIX DATA-API
+ * ============== */
+
+ $(window).on('load', function () {
+ $('[data-spy="affix"]').each(function () {
+ var $spy = $(this)
+ , data = $spy.data()
+
+ data.offset = data.offset || {}
+
+ data.offsetBottom && (data.offset.bottom = data.offsetBottom)
+ data.offsetTop && (data.offset.top = data.offsetTop)
+
+ $spy.affix(data)
+ })
+ })
+
+
+}(window.jQuery);
\ No newline at end of file
[12/47] incubator-systemml git commit: initial port of documentation
to md, algorithms reference converted to md, template based on spark docs
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/css/bootstrap.min.css
----------------------------------------------------------------------
diff --git a/css/bootstrap.min.css b/css/bootstrap.min.css
new file mode 100644
index 0000000..3fa12ac
--- /dev/null
+++ b/css/bootstrap.min.css
@@ -0,0 +1,9 @@
+/*!
+ * Bootstrap v2.1.0
+ *
+ * Copyright 2012 Twitter, Inc
+ * Licensed under the Apache License v2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Designed and built with all the love in the world @twitter by @mdo and @fat.
+ */article,aside,details,figcaption,figure,footer,header,hgroup,nav,section{display:block}audio,canvas,video{display:inline-block;*display:inline;*zoom:1}audio:not([controls]){display:none}html{font-size:100%;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%}a:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}a:hover,a:active{outline:0}sub,sup{position:relative;font-size:75%;line-height:0;vertical-align:baseline}sup{top:-0.5em}sub{bottom:-0.25em}img{height:auto;max-width:100%;vertical-align:middle;border:0;-ms-interpolation-mode:bicubic}#map_canvas img{max-width:none}button,input,select,textarea{margin:0;font-size:100%;vertical-align:middle}button,input{*overflow:visible;line-height:normal}button::-moz-focus-inner,input::-moz-focus-inner{padding:0;border:0}button,input[type="button"],input[type="reset"],input[type="submit"]{cursor:pointer;-webkit-appearance:button}input[type="search"]{-webkit-box-sizing:content-box;-moz-box-sizing:con
tent-box;box-sizing:content-box;-webkit-appearance:textfield}input[type="search"]::-webkit-search-decoration,input[type="search"]::-webkit-search-cancel-button{-webkit-appearance:none}textarea{overflow:auto;vertical-align:top}.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;line-height:0;content:""}.clearfix:after{clear:both}.hide-text{font:0/0 a;color:transparent;text-shadow:none;background-color:transparent;border:0}.input-block-level{display:block;width:100%;min-height:30px;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}body{margin:0;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:14px;line-height:20px;color:#333;background-color:#fff}a{color:#08c;text-decoration:none}a:hover{color:#005580;text-decoration:underline}.img-rounded{-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.img-polaroid{padding:4px;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.2);-webkit-box-shadow:0
1px 3px rgba(0,0,0,0.1);-moz-box-shadow:0 1px 3px rgba(0,0,0,0.1);box-shadow:0 1px 3px rgba(0,0,0,0.1)}.img-circle{-webkit-border-radius:500px;-moz-border-radius:500px;border-radius:500px}.row{margin-left:-20px;*zoom:1}.row:before,.row:after{display:table;line-height:0;content:""}.row:after{clear:both}[class*="span"]{float:left;margin-left:20px}.container,.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:940px}.span12{width:940px}.span11{width:860px}.span10{width:780px}.span9{width:700px}.span8{width:620px}.span7{width:540px}.span6{width:460px}.span5{width:380px}.span4{width:300px}.span3{width:220px}.span2{width:140px}.span1{width:60px}.offset12{margin-left:980px}.offset11{margin-left:900px}.offset10{margin-left:820px}.offset9{margin-left:740px}.offset8{margin-left:660px}.offset7{margin-left:580px}.offset6{margin-left:500px}.offset5{margin-left:420px}.offset4{margin-left:340px}.offset3{margin-left:260px}.offset2{margin-left:180px}.offs
et1{margin-left:100px}.row-fluid{width:100%;*zoom:1}.row-fluid:before,.row-fluid:after{display:table;line-height:0;content:""}.row-fluid:after{clear:both}.row-fluid [class*="span"]{display:block;float:left;width:100%;min-height:30px;margin-left:2.127659574468085%;*margin-left:2.074468085106383%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid [class*="span"]:first-child{margin-left:0}.row-fluid .span12{width:100%;*width:99.94680851063829%}.row-fluid .span11{width:91.48936170212765%;*width:91.43617021276594%}.row-fluid .span10{width:82.97872340425532%;*width:82.92553191489361%}.row-fluid .span9{width:74.46808510638297%;*width:74.41489361702126%}.row-fluid .span8{width:65.95744680851064%;*width:65.90425531914893%}.row-fluid .span7{width:57.44680851063829%;*width:57.39361702127659%}.row-fluid .span6{width:48.93617021276595%;*width:48.88297872340425%}.row-fluid .span5{width:40.42553191489362%;*width:40.37234042553192%}.row-fluid .span4{width:31.9
14893617021278%;*width:31.861702127659576%}.row-fluid .span3{width:23.404255319148934%;*width:23.351063829787233%}.row-fluid .span2{width:14.893617021276595%;*width:14.840425531914894%}.row-fluid .span1{width:6.382978723404255%;*width:6.329787234042553%}.row-fluid .offset12{margin-left:104.25531914893617%;*margin-left:104.14893617021275%}.row-fluid .offset12:first-child{margin-left:102.12765957446808%;*margin-left:102.02127659574467%}.row-fluid .offset11{margin-left:95.74468085106382%;*margin-left:95.6382978723404%}.row-fluid .offset11:first-child{margin-left:93.61702127659574%;*margin-left:93.51063829787232%}.row-fluid .offset10{margin-left:87.23404255319149%;*margin-left:87.12765957446807%}.row-fluid .offset10:first-child{margin-left:85.1063829787234%;*margin-left:84.99999999999999%}.row-fluid .offset9{margin-left:78.72340425531914%;*margin-left:78.61702127659572%}.row-fluid .offset9:first-child{margin-left:76.59574468085106%;*margin-left:76.48936170212764%}.row-fluid .offset8{mar
gin-left:70.2127659574468%;*margin-left:70.10638297872339%}.row-fluid .offset8:first-child{margin-left:68.08510638297872%;*margin-left:67.9787234042553%}.row-fluid .offset7{margin-left:61.70212765957446%;*margin-left:61.59574468085106%}.row-fluid .offset7:first-child{margin-left:59.574468085106375%;*margin-left:59.46808510638297%}.row-fluid .offset6{margin-left:53.191489361702125%;*margin-left:53.085106382978715%}.row-fluid .offset6:first-child{margin-left:51.063829787234035%;*margin-left:50.95744680851063%}.row-fluid .offset5{margin-left:44.68085106382979%;*margin-left:44.57446808510638%}.row-fluid .offset5:first-child{margin-left:42.5531914893617%;*margin-left:42.4468085106383%}.row-fluid .offset4{margin-left:36.170212765957444%;*margin-left:36.06382978723405%}.row-fluid .offset4:first-child{margin-left:34.04255319148936%;*margin-left:33.93617021276596%}.row-fluid .offset3{margin-left:27.659574468085104%;*margin-left:27.5531914893617%}.row-fluid .offset3:first-child{margin-left:25
.53191489361702%;*margin-left:25.425531914893618%}.row-fluid .offset2{margin-left:19.148936170212764%;*margin-left:19.04255319148936%}.row-fluid .offset2:first-child{margin-left:17.02127659574468%;*margin-left:16.914893617021278%}.row-fluid .offset1{margin-left:10.638297872340425%;*margin-left:10.53191489361702%}.row-fluid .offset1:first-child{margin-left:8.51063829787234%;*margin-left:8.404255319148938%}[class*="span"].hide,.row-fluid [class*="span"].hide{display:none}[class*="span"].pull-right,.row-fluid [class*="span"].pull-right{float:right}.container{margin-right:auto;margin-left:auto;*zoom:1}.container:before,.container:after{display:table;line-height:0;content:""}.container:after{clear:both}.container-fluid{padding-right:20px;padding-left:20px;*zoom:1}.container-fluid:before,.container-fluid:after{display:table;line-height:0;content:""}.container-fluid:after{clear:both}p{margin:0 0 10px}.lead{margin-bottom:20px;font-size:20px;font-weight:200;line-height:30px}small{font-size:8
5%}strong{font-weight:bold}em{font-style:italic}cite{font-style:normal}.muted{color:#999}h1,h2,h3,h4,h5,h6{margin:10px 0;font-family:inherit;font-weight:bold;line-height:1;color:inherit;text-rendering:optimizelegibility}h1 small,h2 small,h3 small,h4 small,h5 small,h6 small{font-weight:normal;line-height:1;color:#999}h1{font-size:36px;line-height:40px}h2{font-size:30px;line-height:40px}h3{font-size:24px;line-height:40px}h4{font-size:18px;line-height:20px}h5{font-size:14px;line-height:20px}h6{font-size:12px;line-height:20px}h1 small{font-size:24px}h2 small{font-size:18px}h3 small{font-size:14px}h4 small{font-size:14px}.page-header{padding-bottom:9px;margin:20px 0 30px;border-bottom:1px solid #eee}ul,ol{padding:0;margin:0 0 10px 25px}ul ul,ul ol,ol ol,ol ul{margin-bottom:0}li{line-height:20px}ul.unstyled,ol.unstyled{margin-left:0;list-style:none}dl{margin-bottom:20px}dt,dd{line-height:20px}dt{font-weight:bold}dd{margin-left:10px}.dl-horizontal dt{float:left;width:120px;overflow:hidden;
clear:left;text-align:right;text-overflow:ellipsis;white-space:nowrap}.dl-horizontal dd{margin-left:130px}hr{margin:20px 0;border:0;border-top:1px solid #eee;border-bottom:1px solid #fff}abbr[title]{cursor:help;border-bottom:1px dotted #999}abbr.initialism{font-size:90%;text-transform:uppercase}blockquote{padding:0 0 0 15px;margin:0 0 20px;border-left:5px solid #eee}blockquote p{margin-bottom:0;font-size:16px;font-weight:300;line-height:25px}blockquote small{display:block;line-height:20px;color:#999}blockquote small:before{content:'\2014 \00A0'}blockquote.pull-right{float:right;padding-right:15px;padding-left:0;border-right:5px solid #eee;border-left:0}blockquote.pull-right p,blockquote.pull-right small{text-align:right}blockquote.pull-right small:before{content:''}blockquote.pull-right small:after{content:'\00A0 \2014'}q:before,q:after,blockquote:before,blockquote:after{content:""}address{display:block;margin-bottom:20px;font-style:normal;line-height:20px}code,pre{padding:0 3px 2px
;font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:12px;color:#333;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}code{padding:2px 4px;color:#d14;background-color:#f7f7f9;border:1px solid #e1e1e8}pre{display:block;padding:9.5px;margin:0 0 10px;font-size:13px;line-height:20px;word-break:break-all;word-wrap:break-word;white-space:pre;white-space:pre-wrap;background-color:#f5f5f5;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.15);-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}pre.prettyprint{margin-bottom:20px}pre code{padding:0;color:inherit;background-color:transparent;border:0}.pre-scrollable{max-height:340px;overflow-y:scroll}form{margin:0 0 20px}fieldset{padding:0;margin:0;border:0}legend{display:block;width:100%;padding:0;margin-bottom:20px;font-size:21px;line-height:40px;color:#333;border:0;border-bottom:1px solid #e5e5e5}legend small{font-size:15px;color:#999}label,input,button,select,textarea{font-size:14px;font-we
ight:normal;line-height:20px}input,button,select,textarea{font-family:"Helvetica Neue",Helvetica,Arial,sans-serif}label{display:block;margin-bottom:5px}select,textarea,input[type="text"],input[type="password"],input[type="datetime"],input[type="datetime-local"],input[type="date"],input[type="month"],input[type="time"],input[type="week"],input[type="number"],input[type="email"],input[type="url"],input[type="search"],input[type="tel"],input[type="color"],.uneditable-input{display:inline-block;height:20px;padding:4px 6px;margin-bottom:9px;font-size:14px;line-height:20px;color:#555;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}input,textarea{width:210px}textarea{height:auto}textarea,input[type="text"],input[type="password"],input[type="datetime"],input[type="datetime-local"],input[type="date"],input[type="month"],input[type="time"],input[type="week"],input[type="number"],input[type="email"],input[type="url"],input[type="search"],input[type="tel"],input[type="color"]
,.uneditable-input{background-color:#fff;border:1px solid #ccc;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-webkit-transition:border linear .2s,box-shadow linear .2s;-moz-transition:border linear .2s,box-shadow linear .2s;-o-transition:border linear .2s,box-shadow linear .2s;transition:border linear .2s,box-shadow linear .2s}textarea:focus,input[type="text"]:focus,input[type="password"]:focus,input[type="datetime"]:focus,input[type="datetime-local"]:focus,input[type="date"]:focus,input[type="month"]:focus,input[type="time"]:focus,input[type="week"]:focus,input[type="number"]:focus,input[type="email"]:focus,input[type="url"]:focus,input[type="search"]:focus,input[type="tel"]:focus,input[type="color"]:focus,.uneditable-input:focus{border-color:rgba(82,168,236,0.8);outline:0;outline:thin dotted \9;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 8px rgba(82,168,236,0.6);-moz-
box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 8px rgba(82,168,236,0.6);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 8px rgba(82,168,236,0.6)}input[type="radio"],input[type="checkbox"]{margin:4px 0 0;margin-top:1px \9;*margin-top:0;line-height:normal;cursor:pointer}input[type="file"],input[type="image"],input[type="submit"],input[type="reset"],input[type="button"],input[type="radio"],input[type="checkbox"]{width:auto}select,input[type="file"]{height:30px;*margin-top:4px;line-height:30px}select{width:220px;background-color:#fff;border:1px solid #bbb}select[multiple],select[size]{height:auto}select:focus,input[type="file"]:focus,input[type="radio"]:focus,input[type="checkbox"]:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}.uneditable-input,.uneditable-textarea{color:#999;cursor:not-allowed;background-color:#fcfcfc;border-color:#ccc;-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,0.025);-moz-box-shadow:inset 0 1px 2px rgba(0,0,0,0.025)
;box-shadow:inset 0 1px 2px rgba(0,0,0,0.025)}.uneditable-input{overflow:hidden;white-space:nowrap}.uneditable-textarea{width:auto;height:auto}input:-moz-placeholder,textarea:-moz-placeholder{color:#999}input:-ms-input-placeholder,textarea:-ms-input-placeholder{color:#999}input::-webkit-input-placeholder,textarea::-webkit-input-placeholder{color:#999}.radio,.checkbox{min-height:18px;padding-left:18px}.radio input[type="radio"],.checkbox input[type="checkbox"]{float:left;margin-left:-18px}.controls>.radio:first-child,.controls>.checkbox:first-child{padding-top:5px}.radio.inline,.checkbox.inline{display:inline-block;padding-top:5px;margin-bottom:0;vertical-align:middle}.radio.inline+.radio.inline,.checkbox.inline+.checkbox.inline{margin-left:10px}.input-mini{width:60px}.input-small{width:90px}.input-medium{width:150px}.input-large{width:210px}.input-xlarge{width:270px}.input-xxlarge{width:530px}input[class*="span"],select[class*="span"],textarea[class*="span"],.uneditable-input[class*
="span"],.row-fluid input[class*="span"],.row-fluid select[class*="span"],.row-fluid textarea[class*="span"],.row-fluid .uneditable-input[class*="span"]{float:none;margin-left:0}.input-append input[class*="span"],.input-append .uneditable-input[class*="span"],.input-prepend input[class*="span"],.input-prepend .uneditable-input[class*="span"],.row-fluid input[class*="span"],.row-fluid select[class*="span"],.row-fluid textarea[class*="span"],.row-fluid .uneditable-input[class*="span"],.row-fluid .input-prepend [class*="span"],.row-fluid .input-append [class*="span"]{display:inline-block}input,textarea,.uneditable-input{margin-left:0}.controls-row [class*="span"]+[class*="span"]{margin-left:20px}input.span12,textarea.span12,.uneditable-input.span12{width:926px}input.span11,textarea.span11,.uneditable-input.span11{width:846px}input.span10,textarea.span10,.uneditable-input.span10{width:766px}input.span9,textarea.span9,.uneditable-input.span9{width:686px}input.span8,textarea.span8,.unedit
able-input.span8{width:606px}input.span7,textarea.span7,.uneditable-input.span7{width:526px}input.span6,textarea.span6,.uneditable-input.span6{width:446px}input.span5,textarea.span5,.uneditable-input.span5{width:366px}input.span4,textarea.span4,.uneditable-input.span4{width:286px}input.span3,textarea.span3,.uneditable-input.span3{width:206px}input.span2,textarea.span2,.uneditable-input.span2{width:126px}input.span1,textarea.span1,.uneditable-input.span1{width:46px}.controls-row{*zoom:1}.controls-row:before,.controls-row:after{display:table;line-height:0;content:""}.controls-row:after{clear:both}.controls-row [class*="span"]{float:left}input[disabled],select[disabled],textarea[disabled],input[readonly],select[readonly],textarea[readonly]{cursor:not-allowed;background-color:#eee}input[type="radio"][disabled],input[type="checkbox"][disabled],input[type="radio"][readonly],input[type="checkbox"][readonly]{background-color:transparent}.control-group.warning>label,.control-group.warning .h
elp-block,.control-group.warning .help-inline{color:#c09853}.control-group.warning .checkbox,.control-group.warning .radio,.control-group.warning input,.control-group.warning select,.control-group.warning textarea{color:#c09853;border-color:#c09853;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.warning .checkbox:focus,.control-group.warning .radio:focus,.control-group.warning input:focus,.control-group.warning select:focus,.control-group.warning textarea:focus{border-color:#a47e3c;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #dbc59e;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #dbc59e;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #dbc59e}.control-group.warning .input-prepend .add-on,.control-group.warning .input-append .add-on{color:#c09853;background-color:#fcf8e3;border-color:#c09853}.control-group.error>label,.control-group.error
.help-block,.control-group.error .help-inline{color:#b94a48}.control-group.error .checkbox,.control-group.error .radio,.control-group.error input,.control-group.error select,.control-group.error textarea{color:#b94a48;border-color:#b94a48;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.error .checkbox:focus,.control-group.error .radio:focus,.control-group.error input:focus,.control-group.error select:focus,.control-group.error textarea:focus{border-color:#953b39;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #d59392;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #d59392;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #d59392}.control-group.error .input-prepend .add-on,.control-group.error .input-append .add-on{color:#b94a48;background-color:#f2dede;border-color:#b94a48}.control-group.success>label,.control-group.success .help-block,.control
-group.success .help-inline{color:#468847}.control-group.success .checkbox,.control-group.success .radio,.control-group.success input,.control-group.success select,.control-group.success textarea{color:#468847;border-color:#468847;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075);box-shadow:inset 0 1px 1px rgba(0,0,0,0.075)}.control-group.success .checkbox:focus,.control-group.success .radio:focus,.control-group.success input:focus,.control-group.success select:focus,.control-group.success textarea:focus{border-color:#356635;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7aba7b;-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7aba7b;box-shadow:inset 0 1px 1px rgba(0,0,0,0.075),0 0 6px #7aba7b}.control-group.success .input-prepend .add-on,.control-group.success .input-append .add-on{color:#468847;background-color:#dff0d8;border-color:#468847}input:focus:required:invalid,textarea:focus:required:invalid,selec
t:focus:required:invalid{color:#b94a48;border-color:#ee5f5b}input:focus:required:invalid:focus,textarea:focus:required:invalid:focus,select:focus:required:invalid:focus{border-color:#e9322d;-webkit-box-shadow:0 0 6px #f8b9b7;-moz-box-shadow:0 0 6px #f8b9b7;box-shadow:0 0 6px #f8b9b7}.form-actions{padding:19px 20px 20px;margin-top:20px;margin-bottom:20px;background-color:#f5f5f5;border-top:1px solid #e5e5e5;*zoom:1}.form-actions:before,.form-actions:after{display:table;line-height:0;content:""}.form-actions:after{clear:both}.help-block,.help-inline{color:#595959}.help-block{display:block;margin-bottom:10px}.help-inline{display:inline-block;*display:inline;padding-left:5px;vertical-align:middle;*zoom:1}.input-append,.input-prepend{margin-bottom:5px;font-size:0;white-space:nowrap}.input-append input,.input-prepend input,.input-append select,.input-prepend select,.input-append .uneditable-input,.input-prepend .uneditable-input{position:relative;margin-bottom:0;*margin-left:0;font-size:1
4px;vertical-align:top;-webkit-border-radius:0 3px 3px 0;-moz-border-radius:0 3px 3px 0;border-radius:0 3px 3px 0}.input-append input:focus,.input-prepend input:focus,.input-append select:focus,.input-prepend select:focus,.input-append .uneditable-input:focus,.input-prepend .uneditable-input:focus{z-index:2}.input-append .add-on,.input-prepend .add-on{display:inline-block;width:auto;height:20px;min-width:16px;padding:4px 5px;font-size:14px;font-weight:normal;line-height:20px;text-align:center;text-shadow:0 1px 0 #fff;background-color:#eee;border:1px solid #ccc}.input-append .add-on,.input-prepend .add-on,.input-append .btn,.input-prepend .btn{margin-left:-1px;vertical-align:top;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.input-append .active,.input-prepend .active{background-color:#a9dba9;border-color:#46a546}.input-prepend .add-on,.input-prepend .btn{margin-right:-1px}.input-prepend .add-on:first-child,.input-prepend .btn:first-child{-webkit-border-radius:3px 0 0
3px;-moz-border-radius:3px 0 0 3px;border-radius:3px 0 0 3px}.input-append input,.input-append select,.input-append .uneditable-input{-webkit-border-radius:3px 0 0 3px;-moz-border-radius:3px 0 0 3px;border-radius:3px 0 0 3px}.input-append .add-on:last-child,.input-append .btn:last-child{-webkit-border-radius:0 3px 3px 0;-moz-border-radius:0 3px 3px 0;border-radius:0 3px 3px 0}.input-prepend.input-append input,.input-prepend.input-append select,.input-prepend.input-append .uneditable-input{-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.input-prepend.input-append .add-on:first-child,.input-prepend.input-append .btn:first-child{margin-right:-1px;-webkit-border-radius:3px 0 0 3px;-moz-border-radius:3px 0 0 3px;border-radius:3px 0 0 3px}.input-prepend.input-append .add-on:last-child,.input-prepend.input-append .btn:last-child{margin-left:-1px;-webkit-border-radius:0 3px 3px 0;-moz-border-radius:0 3px 3px 0;border-radius:0 3px 3px 0}input.search-query{padding-right:14px;pad
ding-right:4px \9;padding-left:14px;padding-left:4px \9;margin-bottom:0;-webkit-border-radius:15px;-moz-border-radius:15px;border-radius:15px}.form-search .input-append .search-query,.form-search .input-prepend .search-query{-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.form-search .input-append .search-query{-webkit-border-radius:14px 0 0 14px;-moz-border-radius:14px 0 0 14px;border-radius:14px 0 0 14px}.form-search .input-append .btn{-webkit-border-radius:0 14px 14px 0;-moz-border-radius:0 14px 14px 0;border-radius:0 14px 14px 0}.form-search .input-prepend .search-query{-webkit-border-radius:0 14px 14px 0;-moz-border-radius:0 14px 14px 0;border-radius:0 14px 14px 0}.form-search .input-prepend .btn{-webkit-border-radius:14px 0 0 14px;-moz-border-radius:14px 0 0 14px;border-radius:14px 0 0 14px}.form-search input,.form-inline input,.form-horizontal input,.form-search textarea,.form-inline textarea,.form-horizontal textarea,.form-search select,.form-inline select,.for
m-horizontal select,.form-search .help-inline,.form-inline .help-inline,.form-horizontal .help-inline,.form-search .uneditable-input,.form-inline .uneditable-input,.form-horizontal .uneditable-input,.form-search .input-prepend,.form-inline .input-prepend,.form-horizontal .input-prepend,.form-search .input-append,.form-inline .input-append,.form-horizontal .input-append{display:inline-block;*display:inline;margin-bottom:0;vertical-align:middle;*zoom:1}.form-search .hide,.form-inline .hide,.form-horizontal .hide{display:none}.form-search label,.form-inline label,.form-search .btn-group,.form-inline .btn-group{display:inline-block}.form-search .input-append,.form-inline .input-append,.form-search .input-prepend,.form-inline .input-prepend{margin-bottom:0}.form-search .radio,.form-search .checkbox,.form-inline .radio,.form-inline .checkbox{padding-left:0;margin-bottom:0;vertical-align:middle}.form-search .radio input[type="radio"],.form-search .checkbox input[type="checkbox"],.form-inli
ne .radio input[type="radio"],.form-inline .checkbox input[type="checkbox"]{float:left;margin-right:3px;margin-left:0}.control-group{margin-bottom:10px}legend+.control-group{margin-top:20px;-webkit-margin-top-collapse:separate}.form-horizontal .control-group{margin-bottom:20px;*zoom:1}.form-horizontal .control-group:before,.form-horizontal .control-group:after{display:table;line-height:0;content:""}.form-horizontal .control-group:after{clear:both}.form-horizontal .control-label{float:left;width:140px;padding-top:5px;text-align:right}.form-horizontal .controls{*display:inline-block;*padding-left:20px;margin-left:160px;*margin-left:0}.form-horizontal .controls:first-child{*padding-left:160px}.form-horizontal .help-block{margin-top:10px;margin-bottom:0}.form-horizontal .form-actions{padding-left:160px}table{max-width:100%;background-color:transparent;border-collapse:collapse;border-spacing:0}.table{width:100%;margin-bottom:20px}.table th,.table td{padding:8px;line-height:20px;text-alig
n:left;vertical-align:top;border-top:1px solid #ddd}.table th{font-weight:bold}.table thead th{vertical-align:bottom}.table caption+thead tr:first-child th,.table caption+thead tr:first-child td,.table colgroup+thead tr:first-child th,.table colgroup+thead tr:first-child td,.table thead:first-child tr:first-child th,.table thead:first-child tr:first-child td{border-top:0}.table tbody+tbody{border-top:2px solid #ddd}.table-condensed th,.table-condensed td{padding:4px 5px}.table-bordered{border:1px solid #ddd;border-collapse:separate;*border-collapse:collapse;border-left:0;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.table-bordered th,.table-bordered td{border-left:1px solid #ddd}.table-bordered caption+thead tr:first-child th,.table-bordered caption+tbody tr:first-child th,.table-bordered caption+tbody tr:first-child td,.table-bordered colgroup+thead tr:first-child th,.table-bordered colgroup+tbody tr:first-child th,.table-bordered colgroup+tbody tr:first-child
td,.table-bordered thead:first-child tr:first-child th,.table-bordered tbody:first-child tr:first-child th,.table-bordered tbody:first-child tr:first-child td{border-top:0}.table-bordered thead:first-child tr:first-child th:first-child,.table-bordered tbody:first-child tr:first-child td:first-child{-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-topleft:4px}.table-bordered thead:first-child tr:first-child th:last-child,.table-bordered tbody:first-child tr:first-child td:last-child{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-moz-border-radius-topright:4px}.table-bordered thead:last-child tr:last-child th:first-child,.table-bordered tbody:last-child tr:last-child td:first-child,.table-bordered tfoot:last-child tr:last-child td:first-child{-webkit-border-radius:0 0 0 4px;-moz-border-radius:0 0 0 4px;border-radius:0 0 0 4px;-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-moz-border-radius-bottomleft:4px}.table-b
ordered thead:last-child tr:last-child th:last-child,.table-bordered tbody:last-child tr:last-child td:last-child,.table-bordered tfoot:last-child tr:last-child td:last-child{-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-moz-border-radius-bottomright:4px}.table-bordered caption+thead tr:first-child th:first-child,.table-bordered caption+tbody tr:first-child td:first-child,.table-bordered colgroup+thead tr:first-child th:first-child,.table-bordered colgroup+tbody tr:first-child td:first-child{-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-topleft:4px}.table-bordered caption+thead tr:first-child th:last-child,.table-bordered caption+tbody tr:first-child td:last-child,.table-bordered colgroup+thead tr:first-child th:last-child,.table-bordered colgroup+tbody tr:first-child td:last-child{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-moz-border-right-topleft:4px}.table-striped tbody tr:nth-child(odd) td,.table-s
triped tbody tr:nth-child(odd) th{background-color:#f9f9f9}.table-hover tbody tr:hover td,.table-hover tbody tr:hover th{background-color:#f5f5f5}table [class*=span],.row-fluid table [class*=span]{display:table-cell;float:none;margin-left:0}table .span1{float:none;width:44px;margin-left:0}table .span2{float:none;width:124px;margin-left:0}table .span3{float:none;width:204px;margin-left:0}table .span4{float:none;width:284px;margin-left:0}table .span5{float:none;width:364px;margin-left:0}table .span6{float:none;width:444px;margin-left:0}table .span7{float:none;width:524px;margin-left:0}table .span8{float:none;width:604px;margin-left:0}table .span9{float:none;width:684px;margin-left:0}table .span10{float:none;width:764px;margin-left:0}table .span11{float:none;width:844px;margin-left:0}table .span12{float:none;width:924px;margin-left:0}table .span13{float:none;width:1004px;margin-left:0}table .span14{float:none;width:1084px;margin-left:0}table .span15{float:none;width:1164px;margin-left:
0}table .span16{float:none;width:1244px;margin-left:0}table .span17{float:none;width:1324px;margin-left:0}table .span18{float:none;width:1404px;margin-left:0}table .span19{float:none;width:1484px;margin-left:0}table .span20{float:none;width:1564px;margin-left:0}table .span21{float:none;width:1644px;margin-left:0}table .span22{float:none;width:1724px;margin-left:0}table .span23{float:none;width:1804px;margin-left:0}table .span24{float:none;width:1884px;margin-left:0}.table tbody tr.success td{background-color:#dff0d8}.table tbody tr.error td{background-color:#f2dede}.table tbody tr.info td{background-color:#d9edf7}[class^="icon-"],[class*=" icon-"]{display:inline-block;width:14px;height:14px;margin-top:1px;*margin-right:.3em;line-height:14px;vertical-align:text-top;background-image:url("../img/glyphicons-halflings.png");background-position:14px 14px;background-repeat:no-repeat}.icon-white,.nav>.active>a>[class^="icon-"],.nav>.active>a>[class*=" icon-"],.dropdown-menu>li>a:hover>[clas
s^="icon-"],.dropdown-menu>li>a:hover>[class*=" icon-"],.dropdown-menu>.active>a>[class^="icon-"],.dropdown-menu>.active>a>[class*=" icon-"]{background-image:url("../img/glyphicons-halflings-white.png")}.icon-glass{background-position:0 0}.icon-music{background-position:-24px 0}.icon-search{background-position:-48px 0}.icon-envelope{background-position:-72px 0}.icon-heart{background-position:-96px 0}.icon-star{background-position:-120px 0}.icon-star-empty{background-position:-144px 0}.icon-user{background-position:-168px 0}.icon-film{background-position:-192px 0}.icon-th-large{background-position:-216px 0}.icon-th{background-position:-240px 0}.icon-th-list{background-position:-264px 0}.icon-ok{background-position:-288px 0}.icon-remove{background-position:-312px 0}.icon-zoom-in{background-position:-336px 0}.icon-zoom-out{background-position:-360px 0}.icon-off{background-position:-384px 0}.icon-signal{background-position:-408px 0}.icon-cog{background-position:-432px 0}.icon-trash{back
ground-position:-456px 0}.icon-home{background-position:0 -24px}.icon-file{background-position:-24px -24px}.icon-time{background-position:-48px -24px}.icon-road{background-position:-72px -24px}.icon-download-alt{background-position:-96px -24px}.icon-download{background-position:-120px -24px}.icon-upload{background-position:-144px -24px}.icon-inbox{background-position:-168px -24px}.icon-play-circle{background-position:-192px -24px}.icon-repeat{background-position:-216px -24px}.icon-refresh{background-position:-240px -24px}.icon-list-alt{background-position:-264px -24px}.icon-lock{background-position:-287px -24px}.icon-flag{background-position:-312px -24px}.icon-headphones{background-position:-336px -24px}.icon-volume-off{background-position:-360px -24px}.icon-volume-down{background-position:-384px -24px}.icon-volume-up{background-position:-408px -24px}.icon-qrcode{background-position:-432px -24px}.icon-barcode{background-position:-456px -24px}.icon-tag{background-position:0 -48px}.ic
on-tags{background-position:-25px -48px}.icon-book{background-position:-48px -48px}.icon-bookmark{background-position:-72px -48px}.icon-print{background-position:-96px -48px}.icon-camera{background-position:-120px -48px}.icon-font{background-position:-144px -48px}.icon-bold{background-position:-167px -48px}.icon-italic{background-position:-192px -48px}.icon-text-height{background-position:-216px -48px}.icon-text-width{background-position:-240px -48px}.icon-align-left{background-position:-264px -48px}.icon-align-center{background-position:-288px -48px}.icon-align-right{background-position:-312px -48px}.icon-align-justify{background-position:-336px -48px}.icon-list{background-position:-360px -48px}.icon-indent-left{background-position:-384px -48px}.icon-indent-right{background-position:-408px -48px}.icon-facetime-video{background-position:-432px -48px}.icon-picture{background-position:-456px -48px}.icon-pencil{background-position:0 -72px}.icon-map-marker{background-position:-24px -72p
x}.icon-adjust{background-position:-48px -72px}.icon-tint{background-position:-72px -72px}.icon-edit{background-position:-96px -72px}.icon-share{background-position:-120px -72px}.icon-check{background-position:-144px -72px}.icon-move{background-position:-168px -72px}.icon-step-backward{background-position:-192px -72px}.icon-fast-backward{background-position:-216px -72px}.icon-backward{background-position:-240px -72px}.icon-play{background-position:-264px -72px}.icon-pause{background-position:-288px -72px}.icon-stop{background-position:-312px -72px}.icon-forward{background-position:-336px -72px}.icon-fast-forward{background-position:-360px -72px}.icon-step-forward{background-position:-384px -72px}.icon-eject{background-position:-408px -72px}.icon-chevron-left{background-position:-432px -72px}.icon-chevron-right{background-position:-456px -72px}.icon-plus-sign{background-position:0 -96px}.icon-minus-sign{background-position:-24px -96px}.icon-remove-sign{background-position:-48px -96px
}.icon-ok-sign{background-position:-72px -96px}.icon-question-sign{background-position:-96px -96px}.icon-info-sign{background-position:-120px -96px}.icon-screenshot{background-position:-144px -96px}.icon-remove-circle{background-position:-168px -96px}.icon-ok-circle{background-position:-192px -96px}.icon-ban-circle{background-position:-216px -96px}.icon-arrow-left{background-position:-240px -96px}.icon-arrow-right{background-position:-264px -96px}.icon-arrow-up{background-position:-289px -96px}.icon-arrow-down{background-position:-312px -96px}.icon-share-alt{background-position:-336px -96px}.icon-resize-full{background-position:-360px -96px}.icon-resize-small{background-position:-384px -96px}.icon-plus{background-position:-408px -96px}.icon-minus{background-position:-433px -96px}.icon-asterisk{background-position:-456px -96px}.icon-exclamation-sign{background-position:0 -120px}.icon-gift{background-position:-24px -120px}.icon-leaf{background-position:-48px -120px}.icon-fire{backgrou
nd-position:-72px -120px}.icon-eye-open{background-position:-96px -120px}.icon-eye-close{background-position:-120px -120px}.icon-warning-sign{background-position:-144px -120px}.icon-plane{background-position:-168px -120px}.icon-calendar{background-position:-192px -120px}.icon-random{width:16px;background-position:-216px -120px}.icon-comment{background-position:-240px -120px}.icon-magnet{background-position:-264px -120px}.icon-chevron-up{background-position:-288px -120px}.icon-chevron-down{background-position:-313px -119px}.icon-retweet{background-position:-336px -120px}.icon-shopping-cart{background-position:-360px -120px}.icon-folder-close{background-position:-384px -120px}.icon-folder-open{width:16px;background-position:-408px -120px}.icon-resize-vertical{background-position:-432px -119px}.icon-resize-horizontal{background-position:-456px -118px}.icon-hdd{background-position:0 -144px}.icon-bullhorn{background-position:-24px -144px}.icon-bell{background-position:-48px -144px}.icon-
certificate{background-position:-72px -144px}.icon-thumbs-up{background-position:-96px -144px}.icon-thumbs-down{background-position:-120px -144px}.icon-hand-right{background-position:-144px -144px}.icon-hand-left{background-position:-168px -144px}.icon-hand-up{background-position:-192px -144px}.icon-hand-down{background-position:-216px -144px}.icon-circle-arrow-right{background-position:-240px -144px}.icon-circle-arrow-left{background-position:-264px -144px}.icon-circle-arrow-up{background-position:-288px -144px}.icon-circle-arrow-down{background-position:-312px -144px}.icon-globe{background-position:-336px -144px}.icon-wrench{background-position:-360px -144px}.icon-tasks{background-position:-384px -144px}.icon-filter{background-position:-408px -144px}.icon-briefcase{background-position:-432px -144px}.icon-fullscreen{background-position:-456px -144px}.dropup,.dropdown{position:relative}.dropdown-toggle{*margin-bottom:-3px}.dropdown-toggle:active,.open .dropdown-toggle{outline:0}.car
et{display:inline-block;width:0;height:0;vertical-align:top;border-top:4px solid #000;border-right:4px solid transparent;border-left:4px solid transparent;content:""}.dropdown .caret{margin-top:8px;margin-left:2px}.dropdown-menu{position:absolute;top:100%;left:0;z-index:1000;display:none;float:left;min-width:160px;padding:5px 0;margin:2px 0 0;list-style:none;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.2);*border-right-width:2px;*border-bottom-width:2px;-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px;-webkit-box-shadow:0 5px 10px rgba(0,0,0,0.2);-moz-box-shadow:0 5px 10px rgba(0,0,0,0.2);box-shadow:0 5px 10px rgba(0,0,0,0.2);-webkit-background-clip:padding-box;-moz-background-clip:padding;background-clip:padding-box}.dropdown-menu.pull-right{right:0;left:auto}.dropdown-menu .divider{*width:100%;height:1px;margin:9px 1px;*margin:-5px 0 5px;overflow:hidden;background-color:#e5e5e5;border-bottom:1px solid #fff}.dropdown-menu a{display:blo
ck;padding:3px 20px;clear:both;font-weight:normal;line-height:20px;color:#333;white-space:nowrap}.dropdown-menu li>a:hover,.dropdown-menu li>a:focus,.dropdown-submenu:hover>a{color:#fff;text-decoration:none;background-color:#0088cc;background-color:#0088cc;background-image:-moz-linear-gradient(top,#0088cc,#0087b3);background-image:-webkit-gradient(linear,0 0,0 100%,from(#0088cc),to(#0087b3));background-image:-webkit-linear-gradient(top,#0088cc,#0087b3);background-image:-o-linear-gradient(top,#0088cc,#0087b3);background-image:linear-gradient(to bottom,#0088cc,#0087b3);background-repeat:repeat-x;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff0088cc',endColorstr='#ff0087b3',GradientType=0)}.dropdown-menu .active>a,.dropdown-menu .active>a:hover{color:#fff;text-decoration:none;background-color:#0088cc;background-color:#0081c2;background-image:linear-gradient(to bottom,#0088cc,#0087b3);background-image:-moz-linear-gradient(top,#0088cc,#0087b3);background-image:-webki
t-gradient(linear,0 0,0 100%,from(#0088cc),to(#0087b3));background-image:-webkit-linear-gradient(top,#0088cc,#0087b3);background-image:-o-linear-gradient(top,#0088cc,#0087b3);background-repeat:repeat-x;outline:0;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff0088cc',endColorstr='#ff0087b3',GradientType=0)}.dropdown-menu .disabled>a,.dropdown-menu .disabled>a:hover{color:#999}.dropdown-menu .disabled>a:hover{text-decoration:none;cursor:default;background-color:transparent}.open{*z-index:1000}.open>.dropdown-menu{display:block}.pull-right>.dropdown-menu{right:0;left:auto}.dropup .caret,.navbar-fixed-bottom .dropdown .caret{border-top:0;border-bottom:4px solid #000;content:"\2191"}.dropup .dropdown-menu,.navbar-fixed-bottom .dropdown .dropdown-menu{top:auto;bottom:100%;margin-bottom:1px}.dropdown-submenu{position:relative}.dropdown-submenu>.dropdown-menu{top:0;left:100%;margin-top:-6px;margin-left:-1px;-webkit-border-radius:0 6px 6px 6px;-moz-border-radius:0 6px 6p
x 6px;border-radius:0 6px 6px 6px}.dropdown-submenu:hover .dropdown-menu{display:block}.dropdown-submenu>a:after{display:block;float:right;width:0;height:0;margin-top:5px;margin-right:-10px;border-color:transparent;border-left-color:#ccc;border-style:solid;border-width:5px 0 5px 5px;content:" "}.dropdown-submenu:hover>a:after{border-left-color:#fff}.dropdown .dropdown-menu .nav-header{padding-right:20px;padding-left:20px}.typeahead{margin-top:2px;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.well{min-height:20px;padding:19px;margin-bottom:20px;background-color:#f5f5f5;border:1px solid #e3e3e3;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 1px 1px rgba(0,0,0,0.05);box-shadow:inset 0 1px 1px rgba(0,0,0,0.05)}.well blockquote{border-color:#ddd;border-color:rgba(0,0,0,0.15)}.well-large{padding:24px;-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.well-
small{padding:9px;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}.fade{opacity:0;-webkit-transition:opacity .15s linear;-moz-transition:opacity .15s linear;-o-transition:opacity .15s linear;transition:opacity .15s linear}.fade.in{opacity:1}.collapse{position:relative;height:0;overflow:hidden;overflow:visible \9;-webkit-transition:height .35s ease;-moz-transition:height .35s ease;-o-transition:height .35s ease;transition:height .35s ease}.collapse.in{height:auto}.close{float:right;font-size:20px;font-weight:bold;line-height:20px;color:#000;text-shadow:0 1px 0 #fff;opacity:.2;filter:alpha(opacity=20)}.close:hover{color:#000;text-decoration:none;cursor:pointer;opacity:.4;filter:alpha(opacity=40)}button.close{padding:0;cursor:pointer;background:transparent;border:0;-webkit-appearance:none}.btn{display:inline-block;*display:inline;padding:4px 14px;margin-bottom:0;*margin-left:.3em;font-size:14px;line-height:20px;*line-height:20px;color:#333;text-align:center;text-shad
ow:0 1px 1px rgba(255,255,255,0.75);vertical-align:middle;cursor:pointer;background-color:#f5f5f5;*background-color:#e6e6e6;background-image:-webkit-gradient(linear,0 0,0 100%,from(#fff),to(#e6e6e6));background-image:-webkit-linear-gradient(top,#fff,#e6e6e6);background-image:-o-linear-gradient(top,#fff,#e6e6e6);background-image:linear-gradient(to bottom,#fff,#e6e6e6);background-image:-moz-linear-gradient(top,#fff,#e6e6e6);background-repeat:repeat-x;border:1px solid #bbb;*border:0;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);border-color:#e6e6e6 #e6e6e6 #bfbfbf;border-bottom-color:#a2a2a2;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ffffffff',endColorstr='#ffe6e6e6',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false);*zoom:1;-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 1px 0 rgba(255,255,255,
0.2),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05)}.btn:hover,.btn:active,.btn.active,.btn.disabled,.btn[disabled]{color:#333;background-color:#e6e6e6;*background-color:#d9d9d9}.btn:active,.btn.active{background-color:#ccc \9}.btn:first-child{*margin-left:0}.btn:hover{color:#333;text-decoration:none;background-color:#e6e6e6;*background-color:#d9d9d9;background-position:0 -15px;-webkit-transition:background-position .1s linear;-moz-transition:background-position .1s linear;-o-transition:background-position .1s linear;transition:background-position .1s linear}.btn:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}.btn.active,.btn:active{background-color:#e6e6e6;background-color:#d9d9d9 \9;background-image:none;outline:0;-webkit-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset
0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05)}.btn.disabled,.btn[disabled]{cursor:default;background-color:#e6e6e6;background-image:none;opacity:.65;filter:alpha(opacity=65);-webkit-box-shadow:none;-moz-box-shadow:none;box-shadow:none}.btn-large{padding:9px 14px;font-size:16px;line-height:normal;-webkit-border-radius:5px;-moz-border-radius:5px;border-radius:5px}.btn-large [class^="icon-"]{margin-top:2px}.btn-small{padding:3px 9px;font-size:12px;line-height:18px}.btn-small [class^="icon-"]{margin-top:0}.btn-mini{padding:2px 6px;font-size:11px;line-height:16px}.btn-block{display:block;width:100%;padding-right:0;padding-left:0;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.btn-block+.btn-block{margin-top:5px}.btn-primary.active,.btn-warning.active,.btn-danger.active,.btn-success.active,.btn-info.active,.btn-inverse.active{color:rgba(255,255,255,0.75)}.btn{border-color:#c5c5c5;border-color:rgba(0,0,0,0.15) rgba(0,0,0,0.15) rgba(0,0,0,0.25)}.btn-
primary{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#006dcc;*background-color:#04c;background-image:-webkit-gradient(linear,0 0,0 100%,from(#08c),to(#04c));background-image:-webkit-linear-gradient(top,#08c,#04c);background-image:-o-linear-gradient(top,#08c,#04c);background-image:linear-gradient(to bottom,#08c,#04c);background-image:-moz-linear-gradient(top,#08c,#04c);background-repeat:repeat-x;border-color:#04c #04c #002a80;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff0088cc',endColorstr='#ff0044cc',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-primary:hover,.btn-primary:active,.btn-primary.active,.btn-primary.disabled,.btn-primary[disabled]{color:#fff;background-color:#04c;*background-color:#003bb3}.btn-primary:active,.btn-primary.active{background-color:#039 \9}.btn-warning{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-col
or:#faa732;*background-color:#f89406;background-image:-webkit-gradient(linear,0 0,0 100%,from(#fbb450),to(#f89406));background-image:-webkit-linear-gradient(top,#fbb450,#f89406);background-image:-o-linear-gradient(top,#fbb450,#f89406);background-image:linear-gradient(to bottom,#fbb450,#f89406);background-image:-moz-linear-gradient(top,#fbb450,#f89406);background-repeat:repeat-x;border-color:#f89406 #f89406 #ad6704;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#fffbb450',endColorstr='#fff89406',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-warning:hover,.btn-warning:active,.btn-warning.active,.btn-warning.disabled,.btn-warning[disabled]{color:#fff;background-color:#f89406;*background-color:#df8505}.btn-warning:active,.btn-warning.active{background-color:#c67605 \9}.btn-danger{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#da4f49;*background-colo
r:#bd362f;background-image:-webkit-gradient(linear,0 0,0 100%,from(#ee5f5b),to(#bd362f));background-image:-webkit-linear-gradient(top,#ee5f5b,#bd362f);background-image:-o-linear-gradient(top,#ee5f5b,#bd362f);background-image:linear-gradient(to bottom,#ee5f5b,#bd362f);background-image:-moz-linear-gradient(top,#ee5f5b,#bd362f);background-repeat:repeat-x;border-color:#bd362f #bd362f #802420;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ffee5f5b',endColorstr='#ffbd362f',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-danger:hover,.btn-danger:active,.btn-danger.active,.btn-danger.disabled,.btn-danger[disabled]{color:#fff;background-color:#bd362f;*background-color:#a9302a}.btn-danger:active,.btn-danger.active{background-color:#942a25 \9}.btn-success{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#5bb75b;*background-color:#51a351;background-image:-webki
t-gradient(linear,0 0,0 100%,from(#62c462),to(#51a351));background-image:-webkit-linear-gradient(top,#62c462,#51a351);background-image:-o-linear-gradient(top,#62c462,#51a351);background-image:linear-gradient(to bottom,#62c462,#51a351);background-image:-moz-linear-gradient(top,#62c462,#51a351);background-repeat:repeat-x;border-color:#51a351 #51a351 #387038;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff62c462',endColorstr='#ff51a351',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-success:hover,.btn-success:active,.btn-success.active,.btn-success.disabled,.btn-success[disabled]{color:#fff;background-color:#51a351;*background-color:#499249}.btn-success:active,.btn-success.active{background-color:#408140 \9}.btn-info{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#49afcd;*background-color:#2f96b4;background-image:-webkit-gradient(linear,0 0,0 100%,
from(#5bc0de),to(#2f96b4));background-image:-webkit-linear-gradient(top,#5bc0de,#2f96b4);background-image:-o-linear-gradient(top,#5bc0de,#2f96b4);background-image:linear-gradient(to bottom,#5bc0de,#2f96b4);background-image:-moz-linear-gradient(top,#5bc0de,#2f96b4);background-repeat:repeat-x;border-color:#2f96b4 #2f96b4 #1f6377;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff5bc0de',endColorstr='#ff2f96b4',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-info:hover,.btn-info:active,.btn-info.active,.btn-info.disabled,.btn-info[disabled]{color:#fff;background-color:#2f96b4;*background-color:#2a85a0}.btn-info:active,.btn-info.active{background-color:#24748c \9}.btn-inverse{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#363636;*background-color:#222;background-image:-webkit-gradient(linear,0 0,0 100%,from(#444),to(#222));background-image:-webkit-line
ar-gradient(top,#444,#222);background-image:-o-linear-gradient(top,#444,#222);background-image:linear-gradient(to bottom,#444,#222);background-image:-moz-linear-gradient(top,#444,#222);background-repeat:repeat-x;border-color:#222 #222 #000;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff444444',endColorstr='#ff222222',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.btn-inverse:hover,.btn-inverse:active,.btn-inverse.active,.btn-inverse.disabled,.btn-inverse[disabled]{color:#fff;background-color:#222;*background-color:#151515}.btn-inverse:active,.btn-inverse.active{background-color:#080808 \9}button.btn,input[type="submit"].btn{*padding-top:3px;*padding-bottom:3px}button.btn::-moz-focus-inner,input[type="submit"].btn::-moz-focus-inner{padding:0;border:0}button.btn.btn-large,input[type="submit"].btn.btn-large{*padding-top:7px;*padding-bottom:7px}button.btn.btn-small,input
[type="submit"].btn.btn-small{*padding-top:3px;*padding-bottom:3px}button.btn.btn-mini,input[type="submit"].btn.btn-mini{*padding-top:1px;*padding-bottom:1px}.btn-link,.btn-link:active{background-color:transparent;background-image:none;-webkit-box-shadow:none;-moz-box-shadow:none;box-shadow:none}.btn-link{color:#08c;cursor:pointer;border-color:transparent;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.btn-link:hover{color:#005580;text-decoration:underline;background-color:transparent}.btn-group{position:relative;*margin-left:.3em;font-size:0;white-space:nowrap}.btn-group:first-child{*margin-left:0}.btn-group+.btn-group{margin-left:5px}.btn-toolbar{margin-top:10px;margin-bottom:10px;font-size:0}.btn-toolbar .btn-group{display:inline-block;*display:inline;*zoom:1}.btn-toolbar .btn+.btn,.btn-toolbar .btn-group+.btn,.btn-toolbar .btn+.btn-group{margin-left:5px}.btn-group>.btn{position:relative;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.btn-group>.btn+.b
tn{margin-left:-1px}.btn-group>.btn,.btn-group>.dropdown-menu{font-size:14px}.btn-group>.btn-mini{font-size:11px}.btn-group>.btn-small{font-size:12px}.btn-group>.btn-large{font-size:16px}.btn-group>.btn:first-child{margin-left:0;-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-webkit-border-top-left-radius:4px;border-top-left-radius:4px;-moz-border-radius-bottomleft:4px;-moz-border-radius-topleft:4px}.btn-group>.btn:last-child,.btn-group>.dropdown-toggle{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-moz-border-radius-topright:4px;-moz-border-radius-bottomright:4px}.btn-group>.btn.large:first-child{margin-left:0;-webkit-border-bottom-left-radius:6px;border-bottom-left-radius:6px;-webkit-border-top-left-radius:6px;border-top-left-radius:6px;-moz-border-radius-bottomleft:6px;-moz-border-radius-topleft:6px}.btn-group>.btn.large:last-child,.btn-group>.large.dropdown-toggle{-webkit
-border-top-right-radius:6px;border-top-right-radius:6px;-webkit-border-bottom-right-radius:6px;border-bottom-right-radius:6px;-moz-border-radius-topright:6px;-moz-border-radius-bottomright:6px}.btn-group>.btn:hover,.btn-group>.btn:focus,.btn-group>.btn:active,.btn-group>.btn.active{z-index:2}.btn-group .dropdown-toggle:active,.btn-group.open .dropdown-toggle{outline:0}.btn-group>.btn+.dropdown-toggle{*padding-top:5px;padding-right:8px;*padding-bottom:5px;padding-left:8px;-webkit-box-shadow:inset 1px 0 0 rgba(255,255,255,0.125),inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 1px 0 0 rgba(255,255,255,0.125),inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 1px 0 0 rgba(255,255,255,0.125),inset 0 1px 0 rgba(255,255,255,0.2),0 1px 2px rgba(0,0,0,0.05)}.btn-group>.btn-mini+.dropdown-toggle{*padding-top:2px;padding-right:5px;*padding-bottom:2px;padding-left:5px}.btn-group>.btn-small+.dropdown-toggle{*padding-top:5px;*padd
ing-bottom:4px}.btn-group>.btn-large+.dropdown-toggle{*padding-top:7px;padding-right:12px;*padding-bottom:7px;padding-left:12px}.btn-group.open .dropdown-toggle{background-image:none;-webkit-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05);box-shadow:inset 0 2px 4px rgba(0,0,0,0.15),0 1px 2px rgba(0,0,0,0.05)}.btn-group.open .btn.dropdown-toggle{background-color:#e6e6e6}.btn-group.open .btn-primary.dropdown-toggle{background-color:#04c}.btn-group.open .btn-warning.dropdown-toggle{background-color:#f89406}.btn-group.open .btn-danger.dropdown-toggle{background-color:#bd362f}.btn-group.open .btn-success.dropdown-toggle{background-color:#51a351}.btn-group.open .btn-info.dropdown-toggle{background-color:#2f96b4}.btn-group.open .btn-inverse.dropdown-toggle{background-color:#222}.btn .caret{margin-top:8px;margin-left:0}.btn-mini .caret,.btn-small .caret,.btn-large .caret{margin-top:6px}.btn-la
rge .caret{border-top-width:5px;border-right-width:5px;border-left-width:5px}.dropup .btn-large .caret{border-top:0;border-bottom:5px solid #000}.btn-primary .caret,.btn-warning .caret,.btn-danger .caret,.btn-info .caret,.btn-success .caret,.btn-inverse .caret{border-top-color:#fff;border-bottom-color:#fff}.btn-group-vertical{display:inline-block;*display:inline;*zoom:1}.btn-group-vertical .btn{display:block;float:none;width:100%;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.btn-group-vertical .btn+.btn{margin-top:-1px;margin-left:0}.btn-group-vertical .btn:first-child{-webkit-border-radius:4px 4px 0 0;-moz-border-radius:4px 4px 0 0;border-radius:4px 4px 0 0}.btn-group-vertical .btn:last-child{-webkit-border-radius:0 0 4px 4px;-moz-border-radius:0 0 4px 4px;border-radius:0 0 4px 4px}.btn-group-vertical .btn-large:first-child{-webkit-border-radius:6px 6px 0 0;-moz-border-radius:6px 6px 0 0;border-radius:6px 6px 0 0}.btn-group-vertical .btn-large:last-child{-webkit-bor
der-radius:0 0 6px 6px;-moz-border-radius:0 0 6px 6px;border-radius:0 0 6px 6px}.alert{padding:8px 35px 8px 14px;margin-bottom:20px;color:#c09853;text-shadow:0 1px 0 rgba(255,255,255,0.5);background-color:#fcf8e3;border:1px solid #fbeed5;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.alert h4{margin:0}.alert .close{position:relative;top:-2px;right:-21px;line-height:20px}.alert-success{color:#468847;background-color:#dff0d8;border-color:#d6e9c6}.alert-danger,.alert-error{color:#b94a48;background-color:#f2dede;border-color:#eed3d7}.alert-info{color:#3a87ad;background-color:#d9edf7;border-color:#bce8f1}.alert-block{padding-top:14px;padding-bottom:14px}.alert-block>p,.alert-block>ul{margin-bottom:0}.alert-block p+p{margin-top:5px}.nav{margin-bottom:20px;margin-left:0;list-style:none}.nav>li>a{display:block}.nav>li>a:hover{text-decoration:none;background-color:#eee}.nav>.pull-right{float:right}.nav-header{display:block;padding:3px 15px;font-size:11px;font-weight:bold
;line-height:20px;color:#999;text-shadow:0 1px 0 rgba(255,255,255,0.5);text-transform:uppercase}.nav li+.nav-header{margin-top:9px}.nav-list{padding-right:15px;padding-left:15px;margin-bottom:0}.nav-list>li>a,.nav-list .nav-header{margin-right:-15px;margin-left:-15px;text-shadow:0 1px 0 rgba(255,255,255,0.5)}.nav-list>li>a{padding:3px 15px}.nav-list>.active>a,.nav-list>.active>a:hover{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.2);background-color:#08c}.nav-list [class^="icon-"]{margin-right:2px}.nav-list .divider{*width:100%;height:1px;margin:9px 1px;*margin:-5px 0 5px;overflow:hidden;background-color:#e5e5e5;border-bottom:1px solid #fff}.nav-tabs,.nav-pills{*zoom:1}.nav-tabs:before,.nav-pills:before,.nav-tabs:after,.nav-pills:after{display:table;line-height:0;content:""}.nav-tabs:after,.nav-pills:after{clear:both}.nav-tabs>li,.nav-pills>li{float:left}.nav-tabs>li>a,.nav-pills>li>a{padding-right:12px;padding-left:12px;margin-right:2px;line-height:14px}.nav-tabs{border-bottom:1px s
olid #ddd}.nav-tabs>li{margin-bottom:-1px}.nav-tabs>li>a{padding-top:8px;padding-bottom:8px;line-height:20px;border:1px solid transparent;-webkit-border-radius:4px 4px 0 0;-moz-border-radius:4px 4px 0 0;border-radius:4px 4px 0 0}.nav-tabs>li>a:hover{border-color:#eee #eee #ddd}.nav-tabs>.active>a,.nav-tabs>.active>a:hover{color:#555;cursor:default;background-color:#fff;border:1px solid #ddd;border-bottom-color:transparent}.nav-pills>li>a{padding-top:8px;padding-bottom:8px;margin-top:2px;margin-bottom:2px;-webkit-border-radius:5px;-moz-border-radius:5px;border-radius:5px}.nav-pills>.active>a,.nav-pills>.active>a:hover{color:#fff;background-color:#08c}.nav-stacked>li{float:none}.nav-stacked>li>a{margin-right:0}.nav-tabs.nav-stacked{border-bottom:0}.nav-tabs.nav-stacked>li>a{border:1px solid #ddd;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.nav-tabs.nav-stacked>li:first-child>a{-webkit-border-top-right-radius:4px;border-top-right-radius:4px;-webkit-border-top-left-radi
us:4px;border-top-left-radius:4px;-moz-border-radius-topright:4px;-moz-border-radius-topleft:4px}.nav-tabs.nav-stacked>li:last-child>a{-webkit-border-bottom-right-radius:4px;border-bottom-right-radius:4px;-webkit-border-bottom-left-radius:4px;border-bottom-left-radius:4px;-moz-border-radius-bottomright:4px;-moz-border-radius-bottomleft:4px}.nav-tabs.nav-stacked>li>a:hover{z-index:2;border-color:#ddd}.nav-pills.nav-stacked>li>a{margin-bottom:3px}.nav-pills.nav-stacked>li:last-child>a{margin-bottom:1px}.nav-tabs .dropdown-menu{-webkit-border-radius:0 0 6px 6px;-moz-border-radius:0 0 6px 6px;border-radius:0 0 6px 6px}.nav-pills .dropdown-menu{-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.nav .dropdown-toggle .caret{margin-top:6px;border-top-color:#08c;border-bottom-color:#08c}.nav .dropdown-toggle:hover .caret{border-top-color:#005580;border-bottom-color:#005580}.nav-tabs .dropdown-toggle .caret{margin-top:8px}.nav .active .dropdown-toggle .caret{border-top-color:
#fff;border-bottom-color:#fff}.nav-tabs .active .dropdown-toggle .caret{border-top-color:#555;border-bottom-color:#555}.nav>.dropdown.active>a:hover{cursor:pointer}.nav-tabs .open .dropdown-toggle,.nav-pills .open .dropdown-toggle,.nav>li.dropdown.open.active>a:hover{color:#fff;background-color:#999;border-color:#999}.nav li.dropdown.open .caret,.nav li.dropdown.open.active .caret,.nav li.dropdown.open a:hover .caret{border-top-color:#fff;border-bottom-color:#fff;opacity:1;filter:alpha(opacity=100)}.tabs-stacked .open>a:hover{border-color:#999}.tabbable{*zoom:1}.tabbable:before,.tabbable:after{display:table;line-height:0;content:""}.tabbable:after{clear:both}.tab-content{overflow:auto}.tabs-below>.nav-tabs,.tabs-right>.nav-tabs,.tabs-left>.nav-tabs{border-bottom:0}.tab-content>.tab-pane,.pill-content>.pill-pane{display:none}.tab-content>.active,.pill-content>.active{display:block}.tabs-below>.nav-tabs{border-top:1px solid #ddd}.tabs-below>.nav-tabs>li{margin-top:-1px;margin-bottom:0
}.tabs-below>.nav-tabs>li>a{-webkit-border-radius:0 0 4px 4px;-moz-border-radius:0 0 4px 4px;border-radius:0 0 4px 4px}.tabs-below>.nav-tabs>li>a:hover{border-top-color:#ddd;border-bottom-color:transparent}.tabs-below>.nav-tabs>.active>a,.tabs-below>.nav-tabs>.active>a:hover{border-color:transparent #ddd #ddd #ddd}.tabs-left>.nav-tabs>li,.tabs-right>.nav-tabs>li{float:none}.tabs-left>.nav-tabs>li>a,.tabs-right>.nav-tabs>li>a{min-width:74px;margin-right:0;margin-bottom:3px}.tabs-left>.nav-tabs{float:left;margin-right:19px;border-right:1px solid #ddd}.tabs-left>.nav-tabs>li>a{margin-right:-1px;-webkit-border-radius:4px 0 0 4px;-moz-border-radius:4px 0 0 4px;border-radius:4px 0 0 4px}.tabs-left>.nav-tabs>li>a:hover{border-color:#eee #ddd #eee #eee}.tabs-left>.nav-tabs .active>a,.tabs-left>.nav-tabs .active>a:hover{border-color:#ddd transparent #ddd #ddd;*border-right-color:#fff}.tabs-right>.nav-tabs{float:right;margin-left:19px;border-left:1px solid #ddd}.tabs-right>.nav-tabs>li>a{marg
in-left:-1px;-webkit-border-radius:0 4px 4px 0;-moz-border-radius:0 4px 4px 0;border-radius:0 4px 4px 0}.tabs-right>.nav-tabs>li>a:hover{border-color:#eee #eee #eee #ddd}.tabs-right>.nav-tabs .active>a,.tabs-right>.nav-tabs .active>a:hover{border-color:#ddd #ddd #ddd transparent;*border-left-color:#fff}.nav>.disabled>a{color:#999}.nav>.disabled>a:hover{text-decoration:none;cursor:default;background-color:transparent}.navbar{*position:relative;*z-index:2;margin-bottom:20px;overflow:visible;color:#555}.navbar-inner{min-height:40px;padding-right:20px;padding-left:20px;background-color:#fafafa;background-image:-moz-linear-gradient(top,#fff,#e2f1f8);background-image:-webkit-gradient(linear,0 0,0 100%,from(#fff),to(#e2f1f8));background-image:-webkit-linear-gradient(top,#fff,#e2f1f8);background-image:-o-linear-gradient(top,#fff,#e2f1f8);background-image:linear-gradient(to bottom,#fff,#e2f1f8);background-repeat:repeat-x;border:1px solid #d4d4d4;-webkit-border-radius:4px;-moz-border-radius:4
px;border-radius:4px;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ffffffff',endColorstr='#ffe2f1f8',GradientType=0);-webkit-box-shadow:0 1px 4px rgba(0,0,0,0.065);-moz-box-shadow:0 1px 4px rgba(0,0,0,0.065);box-shadow:0 1px 4px rgba(0,0,0,0.065)}.navbar .container{width:auto}.nav-collapse.collapse{height:auto}.navbar .brand{display:block;float:left;padding:10px 20px 10px;margin-left:-20px;font-size:20px;font-weight:200;color:#555;text-shadow:0 1px 0 #fff}.navbar .brand:hover{text-decoration:none}.navbar-text{margin-bottom:0;line-height:40px}.navbar-link{color:#555}.navbar-link:hover{color:#333}.navbar .divider-vertical{height:40px;margin:0 9px;border-right:1px solid #fff;border-left:1px solid #f2f2f2}.navbar .btn,.navbar .btn-group{margin-top:6px}.navbar .btn-group .btn{margin:0}.navbar-form{margin-bottom:0;*zoom:1}.navbar-form:before,.navbar-form:after{display:table;line-height:0;content:""}.navbar-form:after{clear:both}.navbar-form input,.navbar-form select,.n
avbar-form .radio,.navbar-form .checkbox{margin-top:5px}.navbar-form input,.navbar-form select,.navbar-form .btn{display:inline-block;margin-bottom:0}.navbar-form input[type="image"],.navbar-form input[type="checkbox"],.navbar-form input[type="radio"]{margin-top:3px}.navbar-form .input-append,.navbar-form .input-prepend{margin-top:6px;white-space:nowrap}.navbar-form .input-append input,.navbar-form .input-prepend input{margin-top:0}.navbar-search{position:relative;float:left;margin-top:5px;margin-bottom:0}.navbar-search .search-query{padding:4px 14px;margin-bottom:0;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:13px;font-weight:normal;line-height:1;-webkit-border-radius:15px;-moz-border-radius:15px;border-radius:15px}.navbar-static-top{position:static;width:100%;margin-bottom:0}.navbar-static-top .navbar-inner{-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.navbar-fixed-top,.navbar-fixed-bottom{position:fixed;right:0;left:0;z-index:1030;margin-botto
m:0}.navbar-fixed-top .navbar-inner,.navbar-fixed-bottom .navbar-inner,.navbar-static-top .navbar-inner{border:0}.navbar-fixed-top .navbar-inner,.navbar-fixed-bottom .navbar-inner{padding-right:0;padding-left:0;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0}.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:940px}.navbar-fixed-top{top:0}.navbar-fixed-top .navbar-inner,.navbar-static-top .navbar-inner{-webkit-box-shadow:inset 0 -1px 0 rgba(0,0,0,0.1),0 1px 10px rgba(0,0,0,0.1);-moz-box-shadow:inset 0 -1px 0 rgba(0,0,0,0.1),0 1px 10px rgba(0,0,0,0.1);box-shadow:inset 0 -1px 0 rgba(0,0,0,0.1),0 1px 10px rgba(0,0,0,0.1)}.navbar-fixed-bottom{bottom:0}.navbar-fixed-bottom .navbar-inner{-webkit-box-shadow:inset 0 1px 0 rgba(0,0,0,0.1),0 -1px 10px rgba(0,0,0,0.1);-moz-box-shadow:inset 0 1px 0 rgba(0,0,0,0.1),0 -1px 10px rgba(0,0,0,0.1);box-shadow:inset 0 1px 0 rgba(0,0,0,0.1),0 -1px 10px rgba(0,0,0,0.1)}.navbar .nav{position:relati
ve;left:0;display:block;float:left;margin:0 10px 0 0}.navbar .nav.pull-right{float:right}.navbar .nav>li{float:left}.navbar .nav>li>a{float:none;padding:10px 15px 10px;color:#555;text-decoration:none;text-shadow:0 1px 0 #fff}.navbar .nav .dropdown-toggle .caret{margin-top:8px}.navbar .nav>li>a:focus,.navbar .nav>li>a:hover{color:#333;text-decoration:none;background-color:transparent}.navbar .nav>.active>a,.navbar .nav>.active>a:hover,.navbar .nav>.active>a:focus{color:#555;text-decoration:none;background-color:#e5e5e5;-webkit-box-shadow:inset 0 3px 8px rgba(0,0,0,0.125);-moz-box-shadow:inset 0 3px 8px rgba(0,0,0,0.125);box-shadow:inset 0 3px 8px rgba(0,0,0,0.125)}.navbar .btn-navbar{display:none;float:right;padding:7px 10px;margin-right:5px;margin-left:5px;color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#ededed;*background-color:#e5e5e5;background-image:-webkit-gradient(linear,0 0,0 100%,from(#f2f2f2),to(#e5e5e5));background-image:-webkit-linear-gradient(top,#f2f2f
2,#e5e5e5);background-image:-o-linear-gradient(top,#f2f2f2,#e5e5e5);background-image:linear-gradient(to bottom,#f2f2f2,#e5e5e5);background-image:-moz-linear-gradient(top,#f2f2f2,#e5e5e5);background-repeat:repeat-x;border-color:#e5e5e5 #e5e5e5 #bfbfbf;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#fff2f2f2',endColorstr='#ffe5e5e5',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false);-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.075);-moz-box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.075);box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.075)}.navbar .btn-navbar:hover,.navbar .btn-navbar:active,.navbar .btn-navbar.active,.navbar .btn-navbar.disabled,.navbar .btn-navbar[disabled]{color:#fff;background-color:#e5e5e5;*background-color:#d9d9d9}.navbar .btn-navbar:active,.navbar .btn-navbar.activ
e{background-color:#ccc \9}.navbar .btn-navbar .icon-bar{display:block;width:18px;height:2px;background-color:#f5f5f5;-webkit-border-radius:1px;-moz-border-radius:1px;border-radius:1px;-webkit-box-shadow:0 1px 0 rgba(0,0,0,0.25);-moz-box-shadow:0 1px 0 rgba(0,0,0,0.25);box-shadow:0 1px 0 rgba(0,0,0,0.25)}.btn-navbar .icon-bar+.icon-bar{margin-top:3px}.navbar .nav>li>.dropdown-menu:before{position:absolute;top:-7px;left:9px;display:inline-block;border-right:7px solid transparent;border-bottom:7px solid #ccc;border-left:7px solid transparent;border-bottom-color:rgba(0,0,0,0.2);content:''}.navbar .nav>li>.dropdown-menu:after{position:absolute;top:-6px;left:10px;display:inline-block;border-right:6px solid transparent;border-bottom:6px solid #fff;border-left:6px solid transparent;content:''}.navbar-fixed-bottom .nav>li>.dropdown-menu:before{top:auto;bottom:-7px;border-top:7px solid #ccc;border-bottom:0;border-top-color:rgba(0,0,0,0.2)}.navbar-fixed-bottom .nav>li>.dropdown-menu:after{top
:auto;bottom:-6px;border-top:6px solid #fff;border-bottom:0}.navbar .nav li.dropdown.open>.dropdown-toggle,.navbar .nav li.dropdown.active>.dropdown-toggle,.navbar .nav li.dropdown.open.active>.dropdown-toggle{color:#555;background-color:#e5e5e5}.navbar .nav li.dropdown>.dropdown-toggle .caret{border-top-color:#555;border-bottom-color:#555}.navbar .nav li.dropdown.open>.dropdown-toggle .caret,.navbar .nav li.dropdown.active>.dropdown-toggle .caret,.navbar .nav li.dropdown.open.active>.dropdown-toggle .caret{border-top-color:#555;border-bottom-color:#555}.navbar .pull-right>li>.dropdown-menu,.navbar .nav>li>.dropdown-menu.pull-right{right:0;left:auto}.navbar .pull-right>li>.dropdown-menu:before,.navbar .nav>li>.dropdown-menu.pull-right:before{right:12px;left:auto}.navbar .pull-right>li>.dropdown-menu:after,.navbar .nav>li>.dropdown-menu.pull-right:after{right:13px;left:auto}.navbar .pull-right>li>.dropdown-menu .dropdown-menu,.navbar .nav>li>.dropdown-menu.pull-right .dropdown-menu{r
ight:100%;left:auto;margin-right:-1px;margin-left:0;-webkit-border-radius:6px 0 6px 6px;-moz-border-radius:6px 0 6px 6px;border-radius:6px 0 6px 6px}.navbar-inverse{color:#999}.navbar-inverse .navbar-inner{background-color:#1b1b1b;background-image:-moz-linear-gradient(top,#222,#111);background-image:-webkit-gradient(linear,0 0,0 100%,from(#222),to(#111));background-image:-webkit-linear-gradient(top,#222,#111);background-image:-o-linear-gradient(top,#222,#111);background-image:linear-gradient(to bottom,#222,#111);background-repeat:repeat-x;border-color:#252525;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff222222',endColorstr='#ff111111',GradientType=0)}.navbar-inverse .brand,.navbar-inverse .nav>li>a{color:#999;text-shadow:0 -1px 0 rgba(0,0,0,0.25)}.navbar-inverse .brand:hover,.navbar-inverse .nav>li>a:hover{color:#fff}.navbar-inverse .nav>li>a:focus,.navbar-inverse .nav>li>a:hover{color:#fff;background-color:transparent}.navbar-inverse .nav .active>a,.navbar-in
verse .nav .active>a:hover,.navbar-inverse .nav .active>a:focus{color:#fff;background-color:#111}.navbar-inverse .navbar-link{color:#999}.navbar-inverse .navbar-link:hover{color:#fff}.navbar-inverse .divider-vertical{border-right-color:#222;border-left-color:#111}.navbar-inverse .nav li.dropdown.open>.dropdown-toggle,.navbar-inverse .nav li.dropdown.active>.dropdown-toggle,.navbar-inverse .nav li.dropdown.open.active>.dropdown-toggle{color:#fff;background-color:#111}.navbar-inverse .nav li.dropdown>.dropdown-toggle .caret{border-top-color:#999;border-bottom-color:#999}.navbar-inverse .nav li.dropdown.open>.dropdown-toggle .caret,.navbar-inverse .nav li.dropdown.active>.dropdown-toggle .caret,.navbar-inverse .nav li.dropdown.open.active>.dropdown-toggle .caret{border-top-color:#fff;border-bottom-color:#fff}.navbar-inverse .navbar-search .search-query{color:#fff;background-color:#515151;border-color:#111;-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,0.1),0 1px 0 rgba(255,255,255,0.15)
;-moz-box-shadow:inset 0 1px 2px rgba(0,0,0,0.1),0 1px 0 rgba(255,255,255,0.15);box-shadow:inset 0 1px 2px rgba(0,0,0,0.1),0 1px 0 rgba(255,255,255,0.15);-webkit-transition:none;-moz-transition:none;-o-transition:none;transition:none}.navbar-inverse .navbar-search .search-query:-moz-placeholder{color:#ccc}.navbar-inverse .navbar-search .search-query:-ms-input-placeholder{color:#ccc}.navbar-inverse .navbar-search .search-query::-webkit-input-placeholder{color:#ccc}.navbar-inverse .navbar-search .search-query:focus,.navbar-inverse .navbar-search .search-query.focused{padding:5px 15px;color:#333;text-shadow:0 1px 0 #fff;background-color:#fff;border:0;outline:0;-webkit-box-shadow:0 0 3px rgba(0,0,0,0.15);-moz-box-shadow:0 0 3px rgba(0,0,0,0.15);box-shadow:0 0 3px rgba(0,0,0,0.15)}.navbar-inverse .btn-navbar{color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#0e0e0e;*background-color:#040404;background-image:-webkit-gradient(linear,0 0,0 100%,from(#151515),to(#040404));bac
kground-image:-webkit-linear-gradient(top,#151515,#040404);background-image:-o-linear-gradient(top,#151515,#040404);background-image:linear-gradient(to bottom,#151515,#040404);background-image:-moz-linear-gradient(top,#151515,#040404);background-repeat:repeat-x;border-color:#040404 #040404 #000;border-color:rgba(0,0,0,0.1) rgba(0,0,0,0.1) rgba(0,0,0,0.25);filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff151515',endColorstr='#ff040404',GradientType=0);filter:progid:dximagetransform.microsoft.gradient(enabled=false)}.navbar-inverse .btn-navbar:hover,.navbar-inverse .btn-navbar:active,.navbar-inverse .btn-navbar.active,.navbar-inverse .btn-navbar.disabled,.navbar-inverse .btn-navbar[disabled]{color:#fff;background-color:#040404;*background-color:#000}.navbar-inverse .btn-navbar:active,.navbar-inverse .btn-navbar.active{background-color:#000 \9}.breadcrumb{padding:8px 15px;margin:0 0 20px;list-style:none;background-color:#f5f5f5;-webkit-border-radius:4px;-moz-border-r
adius:4px;border-radius:4px}.breadcrumb li{display:inline-block;*display:inline;text-shadow:0 1px 0 #fff;*zoom:1}.breadcrumb .divider{padding:0 5px;color:#ccc}.breadcrumb .active{color:#999}.pagination{height:40px;margin:20px 0}.pagination ul{display:inline-block;*display:inline;margin-bottom:0;margin-left:0;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px;*zoom:1;-webkit-box-shadow:0 1px 2px rgba(0,0,0,0.05);-moz-box-shadow:0 1px 2px rgba(0,0,0,0.05);box-shadow:0 1px 2px rgba(0,0,0,0.05)}.pagination li{display:inline}.pagination a,.pagination span{float:left;padding:0 14px;line-height:38px;text-decoration:none;background-color:#fff;border:1px solid #ddd;border-left-width:0}.pagination a:hover,.pagination .active a,.pagination .active span{background-color:#f5f5f5}.pagination .active a,.pagination .active span{color:#999;cursor:default}.pagination .disabled span,.pagination .disabled a,.pagination .disabled a:hover{color:#999;cursor:default;background-color:transpa
rent}.pagination li:first-child a,.pagination li:first-child span{border-left-width:1px;-webkit-border-radius:3px 0 0 3px;-moz-border-radius:3px 0 0 3px;border-radius:3px 0 0 3px}.pagination li:last-child a,.pagination li:last-child span{-webkit-border-radius:0 3px 3px 0;-moz-border-radius:0 3px 3px 0;border-radius:0 3px 3px 0}.pagination-centered{text-align:center}.pagination-right{text-align:right}.pager{margin:20px 0;text-align:center;list-style:none;*zoom:1}.pager:before,.pager:after{display:table;line-height:0;content:""}.pager:after{clear:both}.pager li{display:inline}.pager a{display:inline-block;padding:5px 14px;background-color:#fff;border:1px solid #ddd;-webkit-border-radius:15px;-moz-border-radius:15px;border-radius:15px}.pager a:hover{text-decoration:none;background-color:#f5f5f5}.pager .next a{float:right}.pager .previous a{float:left}.pager .disabled a,.pager .disabled a:hover{color:#999;cursor:default;background-color:#fff}.modal-open .dropdown-menu{z-index:2050}.moda
l-open .dropdown.open{*z-index:2050}.modal-open .popover{z-index:2060}.modal-open .tooltip{z-index:2080}.modal-backdrop{position:fixed;top:0;right:0;bottom:0;left:0;z-index:1040;background-color:#000}.modal-backdrop.fade{opacity:0}.modal-backdrop,.modal-backdrop.fade.in{opacity:.8;filter:alpha(opacity=80)}.modal{position:fixed;top:50%;left:50%;z-index:1050;width:560px;margin:-250px 0 0 -280px;overflow:auto;background-color:#fff;border:1px solid #999;border:1px solid rgba(0,0,0,0.3);*border:1px solid #999;-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px;-webkit-box-shadow:0 3px 7px rgba(0,0,0,0.3);-moz-box-shadow:0 3px 7px rgba(0,0,0,0.3);box-shadow:0 3px 7px rgba(0,0,0,0.3);-webkit-background-clip:padding-box;-moz-background-clip:padding-box;background-clip:padding-box}.modal.fade{top:-25%;-webkit-transition:opacity .3s linear,top .3s ease-out;-moz-transition:opacity .3s linear,top .3s ease-out;-o-transition:opacity .3s linear,top .3s ease-out;transition:opacity .3
s linear,top .3s ease-out}.modal.fade.in{top:50%}.modal-header{padding:9px 15px;border-bottom:1px solid #eee}.modal-header .close{margin-top:2px}.modal-header h3{margin:0;line-height:30px}.modal-body{max-height:400px;padding:15px;overflow-y:auto}.modal-form{margin-bottom:0}.modal-footer{padding:14px 15px 15px;margin-bottom:0;text-align:right;background-color:#f5f5f5;border-top:1px solid #ddd;-webkit-border-radius:0 0 6px 6px;-moz-border-radius:0 0 6px 6px;border-radius:0 0 6px 6px;*zoom:1;-webkit-box-shadow:inset 0 1px 0 #fff;-moz-box-shadow:inset 0 1px 0 #fff;box-shadow:inset 0 1px 0 #fff}.modal-footer:before,.modal-footer:after{display:table;line-height:0;content:""}.modal-footer:after{clear:both}.modal-footer .btn+.btn{margin-bottom:0;margin-left:5px}.modal-footer .btn-group .btn+.btn{margin-left:-1px}.tooltip{position:absolute;z-index:1030;display:block;padding:5px;font-size:11px;opacity:0;filter:alpha(opacity=0);visibility:visible}.tooltip.in{opacity:.8;filter:alpha(opacity=80)
}.tooltip.top{margin-top:-3px}.tooltip.right{margin-left:3px}.tooltip.bottom{margin-top:3px}.tooltip.left{margin-left:-3px}.tooltip-inner{max-width:200px;padding:3px 8px;color:#fff;text-align:center;text-decoration:none;background-color:#000;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.tooltip-arrow{position:absolute;width:0;height:0;border-color:transparent;border-style:solid}.tooltip.top .tooltip-arrow{bottom:0;left:50%;margin-left:-5px;border-top-color:#000;border-width:5px 5px 0}.tooltip.right .tooltip-arrow{top:50%;left:0;margin-top:-5px;border-right-color:#000;border-width:5px 5px 5px 0}.tooltip.left .tooltip-arrow{top:50%;right:0;margin-top:-5px;border-left-color:#000;border-width:5px 0 5px 5px}.tooltip.bottom .tooltip-arrow{top:0;left:50%;margin-left:-5px;border-bottom-color:#000;border-width:0 5px 5px}.popover{position:absolute;top:0;left:0;z-index:1010;display:none;width:236px;padding:1px;background-color:#fff;border:1px solid #ccc;border:1px solid r
gba(0,0,0,0.2);-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px;-webkit-box-shadow:0 5px 10px rgba(0,0,0,0.2);-moz-box-shadow:0 5px 10px rgba(0,0,0,0.2);box-shadow:0 5px 10px rgba(0,0,0,0.2);-webkit-background-clip:padding-box;-moz-background-clip:padding;background-clip:padding-box}.popover.top{margin-bottom:10px}.popover.right{margin-left:10px}.popover.bottom{margin-top:10px}.popover.left{margin-right:10px}.popover-title{padding:8px 14px;margin:0;font-size:14px;font-weight:normal;line-height:18px;background-color:#f7f7f7;border-bottom:1px solid #ebebeb;-webkit-border-radius:5px 5px 0 0;-moz-border-radius:5px 5px 0 0;border-radius:5px 5px 0 0}.popover-content{padding:9px 14px}.popover-content p,.popover-content ul,.popover-content ol{margin-bottom:0}.popover .arrow,.popover .arrow:after{position:absolute;display:inline-block;width:0;height:0;border-color:transparent;border-style:solid}.popover .arrow:after{z-index:-1;content:""}.popover.top .arrow{bottom:-10px;lef
t:50%;margin-left:-10px;border-top-color:#fff;border-width:10px 10px 0}.popover.top .arrow:after{bottom:-1px;left:-11px;border-top-color:rgba(0,0,0,0.25);border-width:11px 11px 0}.popover.right .arrow{top:50%;left:-10px;margin-top:-10px;border-right-color:#fff;border-width:10px 10px 10px 0}.popover.right .arrow:after{bottom:-11px;left:-1px;border-right-color:rgba(0,0,0,0.25);border-width:11px 11px 11px 0}.popover.bottom .arrow{top:-10px;left:50%;margin-left:-10px;border-bottom-color:#fff;border-width:0 10px 10px}.popover.bottom .arrow:after{top:-1px;left:-11px;border-bottom-color:rgba(0,0,0,0.25);border-width:0 11px 11px}.popover.left .arrow{top:50%;right:-10px;margin-top:-10px;border-left-color:#fff;border-width:10px 0 10px 10px}.popover.left .arrow:after{right:-1px;bottom:-11px;border-left-color:rgba(0,0,0,0.25);border-width:11px 0 11px 11px}.thumbnails{margin-left:-20px;list-style:none;*zoom:1}.thumbnails:before,.thumbnails:after{display:table;line-height:0;content:""}.thumbnails
:after{clear:both}.row-fluid .thumbnails{margin-left:0}.thumbnails>li{float:left;margin-bottom:20px;margin-left:20px}.thumbnail{display:block;padding:4px;line-height:20px;border:1px solid #ddd;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;-webkit-box-shadow:0 1px 3px rgba(0,0,0,0.055);-moz-box-shadow:0 1px 3px rgba(0,0,0,0.055);box-shadow:0 1px 3px rgba(0,0,0,0.055);-webkit-transition:all .2s ease-in-out;-moz-transition:all .2s ease-in-out;-o-transition:all .2s ease-in-out;transition:all .2s ease-in-out}a.thumbnail:hover{border-color:#08c;-webkit-box-shadow:0 1px 4px rgba(0,105,214,0.25);-moz-box-shadow:0 1px 4px rgba(0,105,214,0.25);box-shadow:0 1px 4px rgba(0,105,214,0.25)}.thumbnail>img{display:block;max-width:100%;margin-right:auto;margin-left:auto}.thumbnail .caption{padding:9px;color:#555}.label,.badge{font-size:11.844px;font-weight:bold;line-height:14px;color:#fff;text-shadow:0 -1px 0 rgba(0,0,0,0.25);white-space:nowrap;vertical-align:baseline;background-
color:#999}.label{padding:1px 4px 2px;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}.badge{padding:1px 9px 2px;-webkit-border-radius:9px;-moz-border-radius:9px;border-radius:9px}a.label:hover,a.badge:hover{color:#fff;text-decoration:none;cursor:pointer}.label-important,.badge-important{background-color:#b94a48}.label-important[href],.badge-important[href]{background-color:#953b39}.label-warning,.badge-warning{background-color:#f89406}.label-warning[href],.badge-warning[href]{background-color:#c67605}.label-success,.badge-success{background-color:#468847}.label-success[href],.badge-success[href]{background-color:#356635}.label-info,.badge-info{background-color:#3a87ad}.label-info[href],.badge-info[href]{background-color:#2d6987}.label-inverse,.badge-inverse{background-color:#333}.label-inverse[href],.badge-inverse[href]{background-color:#1a1a1a}.btn .label,.btn .badge{position:relative;top:-1px}.btn-mini .label,.btn-mini .badge{top:0}@-webkit-keyframes progress-b
ar-stripes{from{background-position:40px 0}to{background-position:0 0}}@-moz-keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}@-ms-keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}@-o-keyframes progress-bar-stripes{from{background-position:0 0}to{background-position:40px 0}}@keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}.progress{height:20px;margin-bottom:20px;overflow:hidden;background-color:#f7f7f7;background-image:-moz-linear-gradient(top,#f5f5f5,#f9f9f9);background-image:-webkit-gradient(linear,0 0,0 100%,from(#f5f5f5),to(#f9f9f9));background-image:-webkit-linear-gradient(top,#f5f5f5,#f9f9f9);background-image:-o-linear-gradient(top,#f5f5f5,#f9f9f9);background-image:linear-gradient(to bottom,#f5f5f5,#f9f9f9);background-repeat:repeat-x;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;filter:progid:dximagetransform.microsoft.gradient(sta
rtColorstr='#fff5f5f5',endColorstr='#fff9f9f9',GradientType=0);-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,0.1);-moz-box-shadow:inset 0 1px 2px rgba(0,0,0,0.1);box-shadow:inset 0 1px 2px rgba(0,0,0,0.1)}.progress .bar{float:left;width:0;height:100%;font-size:12px;color:#fff;text-align:center;text-shadow:0 -1px 0 rgba(0,0,0,0.25);background-color:#0e90d2;background-image:-moz-linear-gradient(top,#149bdf,#0480be);background-image:-webkit-gradient(linear,0 0,0 100%,from(#149bdf),to(#0480be));background-image:-webkit-linear-gradient(top,#149bdf,#0480be);background-image:-o-linear-gradient(top,#149bdf,#0480be);background-image:linear-gradient(to bottom,#149bdf,#0480be);background-repeat:repeat-x;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ff149bdf',endColorstr='#ff0480be',GradientType=0);-webkit-box-shadow:inset 0 -1px 0 rgba(0,0,0,0.15);-moz-box-shadow:inset 0 -1px 0 rgba(0,0,0,0.15);box-shadow:inset 0 -1px 0 rgba(0,0,0,0.15);-webkit-box-sizing:border-box;-moz-bo
x-sizing:border-box;box-sizing:border-box;-webkit-transition:width .6s ease;-moz-transition:width .6s ease;-o-transition:width .6s ease;transition:width .6s ease}.progress .bar+.bar{-webkit-box-shadow:inset 1px 0 0 rgba(0,0,0,0.15),inset 0 -1px 0 rgba(0,0,0,0.15);-moz-box-shadow:inset 1px 0 0 rgba(0,0,0,0.15),inset 0 -1px 0 rgba(0,0,0,0.15);box-shadow:inset 1px 0 0 rgba(0,0,0,0.15),inset 0 -1px 0 rgba(0,0,0,0.15)}.progress-striped .bar{background-color:#149bdf;background-image:-webkit-gradient(linear,0 100%,100% 0,color-stop(0.25,rgba(255,255,255,0.15)),color-stop(0.25,transparent),color-stop(0.5,transparent),color-stop(0.5,rgba(255,255,255,0.15)),color-stop(0.75,rgba(255,255,255,0.15)),color-stop(0.75,transparent),to(transparent));background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:-moz-linear-gradient(45deg,rgba(255,255,255,0.15)
25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);-webkit-background-size:40px 40px;-moz-background-size:40px 40px;-o-background-size:40px 40px;background-size:40px 40px}.progress.active .bar{-webkit-animation:progress-bar-stripes 2s linear infinite;-moz-animation:progress-bar-stripes 2s linear infinite;-ms-animation:progress-bar-stripes 2s linear infinite;-o-animation:progress-bar-stripes 2s linear infinite;animation:progress-bar-stripes 2s linear infinite}.progress-danger .bar,.progress .bar-danger{background-color:#dd514c;background-image:-moz-linear-gr
adient(top,#ee5f5b,#c43c35);background-image:-webkit-gradient(linear,0 0,0 100%,from(#ee5f5b),to(#c43c35));background-image:-webkit-linear-gradient(top,#ee5f5b,#c43c35);background-image:-o-linear-gradient(top,#ee5f5b,#c43c35);background-image:linear-gradient(to bottom,#ee5f5b,#c43c35);background-repeat:repeat-x;filter:progid:dximagetransform.microsoft.gradient(startColorstr='#ffee5f5b',endColorstr='#ffc43c35',GradientType=0)}.progress-danger.progress-striped .bar,.progress-striped .bar-danger{background-color:#ee5f5b;background-image:-webkit-gradient(linear,0 100%,100% 0,color-stop(0.25,rgba(255,255,255,0.15)),color-stop(0.25,transparent),color-stop(0.5,transparent),color-stop(0.5,rgba(255,255,255,0.15)),color-stop(0.75,rgba(255,255,255,0.15)),color-stop(0.75,transparent),to(transparent));background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);backgrou
nd-image:-moz-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,0.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,0.15) 50%,rgba(255,255,255,0.15) 75%,transparent 75%,transparent)}.progress-success .bar,.progress .bar-success{background-color:#5eb95e;background-image:-moz-linear-gradient(top,#62c462,#57a957);background-image:-webkit-gradient(linear,0 0,0 100%,from(#62c462),to(#57a957));background-image:-webkit-linear-gradient(top,#62c462,#57a957);background-image:-o-linear-gradient(top,#62c462,#57a957);background-image:linear-gradient(to bottom,#62c462,#57a957);background-repeat:repeat-x;filter:progid:dximagetransform.microsof
t.gradient(startColorstr='#ff62c462',endColorstr='#ff57a957',GradientType=0)}.progress-success.progress-striped .bar,.progress-striped .bar-success{background-color:#62c462;background-image:-
<TRUNCATED>
[15/47] incubator-systemml git commit: initial port of documentation
to md, algorithms reference converted to md, template based on spark docs
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/algorithms-regression.md
----------------------------------------------------------------------
diff --git a/algorithms-regression.md b/algorithms-regression.md
new file mode 100644
index 0000000..be39a27
--- /dev/null
+++ b/algorithms-regression.md
@@ -0,0 +1,1404 @@
+---
+layout: global
+title: SystemML Algorithms Reference - Regression
+displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
+---
+
+
+# 4. Regression
+
+
+## 4.1. Linear Regression
+
+### Description
+
+Linear Regression scripts are used to model the relationship between one
+numerical response variable and one or more explanatory (feature)
+variables. The scripts are given a dataset $(X, Y) = (x_i, y_i)_{i=1}^n$
+where $x_i$ is a numerical vector of feature variables and $y_i$ is a
+numerical response value for each training data record. The feature
+vectors are provided as a matrix $X$ of size $n\,{\times}\,m$, where $n$
+is the number of records and $m$ is the number of features. The observed
+response values are provided as a 1-column matrix $Y$, with a numerical
+value $y_i$ for each $x_i$ in the corresponding row of matrix $X$.
+
+In linear regression, we predict the distribution of the response $y_i$
+based on a fixed linear combination of the features in $x_i$. We assume
+that there exist constant regression coefficients
+$\beta_0, \beta_1, \ldots, \beta_m$ and a constant residual
+variance $\sigma^2$ such that
+
+$$
+\begin{equation}
+y_i \sim Normal(\mu_i, \sigma^2) \,\,\,\,\textrm{where}\,\,\,\,
+\mu_i \,=\, \beta_0 + \beta_1 x_{i,1} + \ldots + \beta_m x_{i,m}
+\end{equation}
+$$
+
+Distribution
+$y_i \sim Normal(\mu_i, \sigma^2)$
+models the “unexplained” residual noise and is assumed independent
+across different records.
+
+The goal is to estimate the regression coefficients and the residual
+variance. Once they are accurately estimated, we can make predictions
+about $y_i$ given $x_i$ in new records. We can also use the $\beta_j$’s
+to analyze the influence of individual features on the response value,
+and assess the quality of this model by comparing residual variance in
+the response, left after prediction, with its total variance.
+
+There are two scripts in our library, both doing the same estimation,
+but using different computational methods. Depending on the size and the
+sparsity of the feature matrix $X$, one or the other script may be more
+efficient. The “direct solve” script `LinearRegDS.dml` is more
+efficient when the number of features $m$ is relatively small
+($m \sim 1000$ or less) and matrix $X$ is either tall or fairly dense
+(has ${\gg}\:m^2$ nonzeros); otherwise, the “conjugate gradient” script
+`LinearRegCG.dml` is more efficient. If $m > 50000$, use only
+`LinearRegCG.dml`.
+
+
+### Usage
+
+Linear Regression - Direct Solve
+
+ hadoop jar SystemML.jar -f LinearRegDS.dml
+ -nvargs X=file
+ Y=file
+ B=file
+ O=file
+ icpt=int
+ reg=double
+ fmt=format
+
+
+Linear Regression - Conjugate Gradient
+
+ hadoop jar SystemML.jar -f LinearRegCG.dml
+ -nvargs X=file
+ Y=file
+ B=file
+ O=file
+ Log=file
+ icpt=int
+ reg=double
+ tol=double
+ maxi=int
+ fmt=format
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read the matrix of feature vectors, each row
+constitutes one feature vector
+
+**Y**: Location to read the 1-column matrix of response values
+
+**B**: Location to store the estimated regression parameters (the $\beta_j$’s),
+with the intercept parameter $\beta_0$ at position
+B\[$m\,{+}\,1$, 1\] if available
+
+**O**: (default: `" "`) Location to store the CSV-file of summary statistics defined
+in [**Table 7**](algorithms-regression.html#table7), the default is to print it to the
+standard output
+
+**Log**: (default: `" "`, `LinearRegCG.dml` only) Location to store
+iteration-specific variables for monitoring and debugging purposes, see
+[**Table 8**](algorithms-regression.html#table8)
+for details.
+
+**icpt**: (default: 0) Intercept presence and shifting/rescaling the features
+in $X$:
+<ul>
+<li>0 = no intercept (hence no $\beta_0$), no shifting or
+rescaling of the features</li>
+<li>1 = add intercept, but do not shift/rescale the features
+in $X$</li>
+<li>2 = add intercept, shift/rescale the features in $X$ to
+mean 0, variance 1</li>
+</ul>
+
+**reg**: (default: 0.000001) L2-regularization parameter $\lambda\geq 0$; set to nonzero for highly
+dependent, sparse, or numerous ($m \gtrsim n/10$) features
+
+**tol**: (default: 0.000001, `LinearRegCG.dml` only) Tolerance $\varepsilon\geq 0$ used in the
+convergence criterion: we terminate conjugate gradient iterations when
+the $\beta$-residual reduces in L2-norm by this factor
+
+**maxi**: (default: 0, `LinearRegCG.dml` only) Maximum number of conjugate
+gradient iterations, or 0 if no maximum limit provided
+
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
+
+
+### Examples
+
+Linear Regression - Direct Solve
+
+ hadoop jar SystemML.jar -f LinearRegDS.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ fmt=csv
+ O=/user/ml/stats.csv
+ icpt=2
+ reg=1.0
+
+
+Linear Regression - Conjugate Gradient
+
+ hadoop jar SystemML.jar -f LinearRegCG.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ fmt=csv
+ O=/user/ml/stats.csv
+ icpt=2
+ reg=1.0
+ tol=0.00000001
+ maxi=100
+ Log=/user/ml/log.csv
+
+
+
+* * *
+
+<a name="table7" />
+**Table 7**: Besides $\beta$, linear regression scripts compute a few summary statistics
+listed below. The statistics are provided in CSV format, one comma-separated name-value
+pair per each line.
+
+| Name | Meaning |
+| --------------------- | ------- |
+| AVG\_TOT\_Y | Average of the response value $Y$
+| STDEV\_TOT\_Y | Standard Deviation of the response value $Y$
+| AVG\_RES\_Y | Average of the residual $Y - \mathop{\mathrm{pred}}(Y \mid X)$, i.e. residual bias
+| STDEV\_RES\_Y | Standard Deviation of the residual $Y - \mathop{\mathrm{pred}}(Y \mid X)$
+| DISPERSION | GLM-style dispersion, i.e. residual sum of squares / \#deg. fr.
+| PLAIN\_R2 | Plain $R^2$ of residual with bias included vs. total average
+| ADJUSTED\_R2 | Adjusted $R^2$ of residual with bias included vs. total average
+| PLAIN\_R2\_NOBIAS | Plain $R^2$ of residual with bias subtracted vs. total average
+| ADJUSTED\_R2\_NOBIAS | Adjusted $R^2$ of residual with bias subtracted vs. total average
+| PLAIN\_R2\_VS\_0 | * Plain $R^2$ of residual with bias included vs. zero constant
+| ADJUSTED\_R2\_VS\_0 | * Adjusted $R^2$ of residual with bias included vs. zero constant
+
+\* The last two statistics are only printed if there is no intercept (`icpt=0`)
+
+* * *
+
+<a name="table8" />
+**Table 8**: The `Log` file for the `LinearRegCG.dml` script
+contains the above iteration variables in CSV format, each line
+containing a triple (Name, Iteration\#, Value) with Iteration\# being 0
+for initial values.
+
+
+| Name | Meaning |
+| --------------------- | ------- |
+| CG\_RESIDUAL\_NORM | L2-norm of conjug. grad. residual, which is $A$ %\*% $\beta - t(X)$ %\*% $y$ where $A = t(X)$ %\*% $X + diag(\lambda)$, or a similar quantity
+| CG\_RESIDUAL\_RATIO | Ratio of current L2-norm of conjug. grad. residual over the initial
+
+* * *
+
+
+### Details
+
+To solve a linear regression problem over feature matrix $X$ and
+response vector $Y$, we can find coefficients
+$\beta_0, \beta_1, \ldots, \beta_m$ and $\sigma^2$ that maximize the
+joint likelihood of all $y_i$ for $i=1\ldots n$, defined by the assumed
+statistical model (1). Since the joint likelihood of the
+independent
+$y_i \sim Normal(\mu_i, \sigma^2)$
+is proportional to the product of
+$\exp\big({-}\,(y_i - \mu_i)^2 / (2\sigma^2)\big)$, we can take the
+logarithm of this product, then multiply by $-2\sigma^2 < 0$ to obtain a
+least squares problem:
+
+$$
+\begin{equation}
+\sum_{i=1}^n \, (y_i - \mu_i)^2 \,\,=\,\,
+\sum_{i=1}^n \Big(y_i - \beta_0 - \sum_{j=1}^m \beta_j x_{i,j}\Big)^2
+\,\,\to\,\,\min
+\end{equation}
+$$
+
+This may not be enough, however. The minimum may
+sometimes be attained over infinitely many $\beta$-vectors, for example
+if $X$ has an all-0 column, or has linearly dependent columns, or has
+fewer rows than columns . Even if (2) has a unique
+solution, other $\beta$-vectors may be just a little suboptimal[^1], yet
+give significantly different predictions for new feature vectors. This
+results in *overfitting*: prediction error for the training data ($X$
+and $Y$) is much smaller than for the test data (new records).
+
+Overfitting and degeneracy in the data is commonly mitigated by adding a
+regularization penalty term to the least squares function:
+
+$$
+\begin{equation}
+\sum_{i=1}^n \Big(y_i - \beta_0 - \sum_{j=1}^m \beta_j x_{i,j}\Big)^2
+\,+\,\, \lambda \sum_{j=1}^m \beta_j^2
+\,\,\to\,\,\min
+\end{equation}
+$$
+
+The choice of $\lambda>0$, the regularization
+constant, typically involves cross-validation where the dataset is
+repeatedly split into a training part (to estimate the $\beta_j$’s) and
+a test part (to evaluate prediction accuracy), with the goal of
+maximizing the test accuracy. In our scripts, $\lambda$ is provided as
+input parameter `reg`.
+
+The solution to the least squares problem (3), through
+taking the derivative and setting it to 0, has the matrix linear
+equation form
+
+$$
+\begin{equation}
+A\left[\textstyle\beta_{1:m}\atop\textstyle\beta_0\right] \,=\, \big[X,\,1\big]^T Y,\,\,\,
+\textrm{where}\,\,\,
+A \,=\, \big[X,\,1\big]^T \big[X,\,1\big]\,+\,\hspace{0.5pt} diag(\hspace{0.5pt}
+\underbrace{\lambda,\ldots, \lambda}_{\scriptstyle m}, 0)
+\end{equation}
+$$
+
+where $[X,\,1]$ is $X$ with an extra column of 1s
+appended on the right, and the diagonal matrix of $\lambda$’s has a zero
+to keep the intercept $\beta_0$ unregularized. If the intercept is
+disabled by setting $icpt=0$, the equation is simply $X^T X \beta = X^T Y$.
+
+We implemented two scripts for solving equation (4): one
+is a “direct solver” that computes $A$ and then solves
+$A\beta = [X,\,1]^T Y$ by calling an external package, the other
+performs linear conjugate gradient (CG) iterations without ever
+materializing $A$. The CG algorithm closely follows Algorithm 5.2 in
+Chapter 5 of [[Nocedal2006]](algorithms-bibliography.html). Each step in the CG algorithm
+computes a matrix-vector multiplication $q = Ap$ by first computing
+$[X,\,1]\, p$ and then $[X,\,1]^T [X,\,1]\, p$. Usually the number of
+such multiplications, one per CG iteration, is much smaller than $m$.
+The user can put a hard bound on it with input
+parameter `maxi`, or use the default maximum of $m+1$ (or $m$
+if no intercept) by having `maxi=0`. The CG iterations
+terminate when the L2-norm of vector $r = A\beta - [X,\,1]^T Y$
+decreases from its initial value (for $\beta=0$) by the tolerance factor
+specified in input parameter `tol`.
+
+The CG algorithm is more efficient if computing
+$[X,\,1]^T \big([X,\,1]\, p\big)$ is much faster than materializing $A$,
+an $(m\,{+}\,1)\times(m\,{+}\,1)$ matrix. The Direct Solver (DS) is more
+efficient if $X$ takes up a lot more memory than $A$ (i.e. $X$ has a lot
+more nonzeros than $m^2$) and if $m^2$ is small enough for the external
+solver ($m \lesssim 50000$). A more precise determination between CG
+and DS is subject to further research.
+
+In addition to the $\beta$-vector, the scripts estimate the residual
+standard deviation $\sigma$ and the $R^2$, the ratio of “explained”
+variance to the total variance of the response variable. These
+statistics only make sense if the number of degrees of freedom
+$n\,{-}\,m\,{-}\,1$ is positive and the regularization constant
+$\lambda$ is negligible or zero. The formulas for $\sigma$ and
+$R^2$ are:
+
+$$R^2_{\textrm{plain}} = 1 - \frac{\mathrm{RSS}}{\mathrm{TSS}},\quad
+\sigma \,=\, \sqrt{\frac{\mathrm{RSS}}{n - m - 1}},\quad
+R^2_{\textrm{adj.}} = 1 - \frac{\sigma^2 (n-1)}{\mathrm{TSS}}$$
+
+where
+
+$$\mathrm{RSS} \,=\, \sum_{i=1}^n \Big(y_i - \hat{\mu}_i -
+\frac{1}{n} \sum_{i'=1}^n \,(y_{i'} - \hat{\mu}_{i'})\Big)^2; \quad
+\mathrm{TSS} \,=\, \sum_{i=1}^n \Big(y_i - \frac{1}{n} \sum_{i'=1}^n y_{i'}\Big)^2$$
+
+Here $\hat{\mu}_i$ are the predicted means for $y_i$ based on the
+estimated regression coefficients and the feature vectors. They may be
+biased when no intercept is present, hence the RSS formula subtracts the
+bias.
+
+Lastly, note that by choosing the input option `icpt=2` the
+user can shift and rescale the columns of $X$ to have zero average and
+the variance of 1. This is particularly important when using
+regularization over highly disbalanced features, because regularization
+tends to penalize small-variance columns (which need large $\beta_j$’s)
+more than large-variance columns (with small $\beta_j$’s). At the end,
+the estimated regression coefficients are shifted and rescaled to apply
+to the original features.
+
+
+### Returns
+
+The estimated regression coefficients (the $\hat{\beta}_j$’s) are
+populated into a matrix and written to an HDFS file whose path/name was
+provided as the `B` input argument. What this matrix
+contains, and its size, depends on the input argument `icpt`,
+which specifies the user’s intercept and rescaling choice:
+
+ * **icpt=0**: No intercept, matrix $B$ has size $m\,{\times}\,1$, with
+$B[j, 1] = \hat{\beta}_j$ for each $j$ from 1 to $m = {}$ncol$(X)$.
+ * **icpt=1**: There is intercept, but no shifting/rescaling of $X$; matrix $B$ has
+size $(m\,{+}\,1) \times 1$, with $B[j, 1] = \hat{\beta}_j$ for $j$ from
+1 to $m$, and $B[m\,{+}\,1, 1] = \hat{\beta}_0$, the estimated intercept
+coefficient.
+ * **icpt=2**: There is intercept, and the features in $X$ are shifted to mean$ = 0$
+and rescaled to variance$ = 1$; then there are two versions of
+the $\hat{\beta}_j$’s, one for the original features and another for the
+shifted/rescaled features. Now matrix $B$ has size
+$(m\,{+}\,1) \times 2$, with $B[\cdot, 1]$ for the original features and
+$B[\cdot, 2]$ for the shifted/rescaled features, in the above format.
+Note that $B[\cdot, 2]$ are iteratively estimated and $B[\cdot, 1]$ are
+obtained from $B[\cdot, 2]$ by complementary shifting and rescaling.
+
+The estimated summary statistics, including residual standard
+deviation $\sigma$ and the $R^2$, are printed out or sent into a file
+(if specified) in CSV format as defined in [**Table 7**](algorithms-regression.html#table7).
+For conjugate gradient iterations, a log file with monitoring variables
+can also be made available, see [**Table 8**](algorithms-regression.html#table8).
+
+
+* * *
+
+## 4.2. Generalized Linear Models
+
+### Description
+
+Generalized Linear Models
+[[Gill2000](algorithms-bibliography.html),
+[McCullagh1989](algorithms-bibliography.html),
+[Nelder1972](algorithms-bibliography.html)]
+extend the methodology of linear
+and logistic regression to a variety of distributions commonly assumed
+as noise effects in the response variable. As before, we are given a
+collection of records $(x_1, y_1)$, …, $(x_n, y_n)$ where $x_i$ is a
+numerical vector of explanatory (feature) variables of size $\dim x_i = m$, and $y_i$
+is the response (dependent) variable observed for this vector. GLMs
+assume that some linear combination of the features in $x_i$ determines
+the *mean* $\mu_i$ of $y_i$, while the observed $y_i$ is a random
+outcome of a noise distribution
+$Prob[y\mid \mu_i]\,$[^2]
+with that mean $\mu_i$:
+
+$$x_i \,\,\,\,\mapsto\,\,\,\, \eta_i = \beta_0 + \sum\nolimits_{j=1}^m \beta_j x_{i,j}
+\,\,\,\,\mapsto\,\,\,\, \mu_i \,\,\,\,\mapsto \,\,\,\, y_i \sim Prob[y\mid \mu_i]$$
+
+In linear regression the response mean $\mu_i$ *equals* some linear
+combination over $x_i$, denoted above by $\eta_i$. In logistic
+regression with $y\in\{0, 1\}$ (Bernoulli) the mean of $y$ is the same
+as $Prob[y=1]$
+and equals $1/(1+e^{-\eta_i})$, the logistic function of $\eta_i$. In
+GLM, $\mu_i$ and $\eta_i$ can be related via any given smooth monotone
+function called the *link function*: $\eta_i = g(\mu_i)$. The unknown
+linear combination parameters $\beta_j$ are assumed to be the same for
+all records.
+
+The goal of the regression is to estimate the parameters $\beta_j$ from
+the observed data. Once the $\beta_j$’s are accurately estimated, we can
+make predictions about $y$ for a new feature vector $x$. To do so,
+compute $\eta$ from $x$ and use the inverted link function
+$\mu = g^{-1}(\eta)$ to compute the mean $\mu$ of $y$; then use the
+distribution $Prob[y\mid \mu]$
+to make predictions about $y$. Both $g(\mu)$ and
+$Prob[y\mid \mu]$
+are user-provided. Our GLM script supports a standard set of
+distributions and link functions, see below for details.
+
+
+### Usage
+
+ hadoop jar SystemML.jar -f GLM.dml
+ -nvargs X=file
+ Y=file
+ B=file
+ fmt=format
+ O=file
+ Log=file
+ dfam=int
+ vpow=double
+ link=int
+ lpow=double
+ yneg=double
+ icpt=int
+ reg=double
+ tol=double
+ disp=double
+ moi=int
+ mii=int
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read the matrix of feature vectors; each row
+constitutes an example.
+
+**Y**: Location to read the response matrix, which may have 1 or 2 columns
+
+**B**: Location to store the estimated regression parameters (the $\beta_j$’s),
+with the intercept parameter $\beta_0$ at position
+B\[$m\,{+}\,1$, 1\] if available
+
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
+
+**O**: (default: `" "`) Location to write certain summary statistics described
+in [**Table 9**](algorithms-regression.html#table9),
+by default it is standard output.
+
+**Log**: (default: `" "`) Location to store iteration-specific variables for monitoring
+and debugging purposes, see [**Table 10**](algorithms-regression.html#table10) for details.
+
+**dfam**: (default: 1) Distribution family code to specify
+$Prob[y\mid \mu]$,
+see [**Table 11**](algorithms-regression.html#table11):
+
+ * 1 = power distributions with $Var(y) = \mu^{\alpha}$
+ * 2 = binomial or Bernoulli
+
+**vpow**: (default: 0.0) When `dfam=1`, this provides the $q$ in
+$Var(y) = a\mu^q$, the power
+dependence of the variance of $y$ on its mean. In particular, use:
+
+ * 0.0 = Gaussian
+ * 1.0 = Poisson
+ * 2.0 = Gamma
+ * 3.0 = inverse Gaussian
+
+**link**: (default: 0) Link function code to determine the link
+function $\eta = g(\mu)$:
+
+ * 0 = canonical link (depends on the distribution family),
+see [**Table 11**](algorithms-regression.html#table11)
+ * 1 = power functions
+ * 2 = logit
+ * 3 = probit
+ * 4 = cloglog
+ * 5 = cauchit
+
+**lpow**: (default: 1.0) When link=1, this provides the $s$ in
+$\eta = \mu^s$, the power link function; `lpow=0.0` gives the
+log link $\eta = \log\mu$. Common power links:
+
+ * -2.0 = $1/\mu^2$
+ * -1.0 = reciprocal
+ * 0.0 = log
+ * 0.5 = sqrt
+ * 1.0 = identity
+
+**yneg**: (default: 0.0) When `dfam=2` and the response matrix $Y$ has
+1 column, this specifies the $y$-value used for Bernoulli “No” label
+(“Yes” is $1$):
+0.0 when $y\in\\{0, 1\\}$; -1.0 when
+$y\in\\{-1, 1\\}$
+
+**icpt**: (default: 0) Intercept and shifting/rescaling of the features in $X$:
+
+ * 0 = no intercept (hence no $\beta_0$), no
+shifting/rescaling of the features
+ * 1 = add intercept, but do not shift/rescale the features
+in $X$
+ * 2 = add intercept, shift/rescale the features in $X$ to
+mean 0, variance 1
+
+**reg**: (default: 0.0) L2-regularization parameter ($\lambda$)
+
+**tol**: (default: 0.000001) Tolerance ($\varepsilon$) used in the convergence criterion: we
+terminate the outer iterations when the deviance changes by less than
+this factor; see below for details
+
+**disp**: (default: 0.0) Dispersion parameter, or 0.0 to estimate it from
+data
+
+**moi**: (default: 200) Maximum number of outer (Fisher scoring) iterations
+
+**mii**: (default: 0) Maximum number of inner (conjugate gradient) iterations, or 0
+if no maximum limit provided
+
+* * *
+
+### Examples
+
+ hadoop jar SystemML.jar -f GLM.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ fmt=csv
+ dfam=2
+ link=2
+ yneg=-1.0
+ icpt=2
+ reg=0.01
+ tol=0.00000001
+ disp=1.0
+ moi=100
+ mii=10
+ O=/user/ml/stats.csv
+ Log=/user/ml/log.csv
+
+
+* * *
+
+<a name="table9" />
+**Table 9**: Besides $\beta$, GLM regression script computes a few summary
+statistics listed below. They are provided in CSV format, one
+comma-separated name-value pair per each line.
+
+| Name | Meaning |
+| --------------------- | ------- |
+| TERMINATION\_CODE | A positive integer indicating success/failure as follows: <br />1 = Converged successfully; 2 = Maximum \# of iterations reached; 3 = Input (X, Y) out of range; 4 = Distribution/link not supported
+| BETA\_MIN | Smallest beta value (regression coefficient), excluding the intercept
+| BETA\_MIN\_INDEX | Column index for the smallest beta value
+| BETA\_MAX | Largest beta value (regression coefficient), excluding the intercept
+| BETA\_MAX\_INDEX | Column index for the largest beta value
+| INTERCEPT | Intercept value, or NaN if there is no intercept (if `icpt=0`)
+| DISPERSION | Dispersion used to scale deviance, provided in disp input argument or estimated (same as DISPERSION\_EST) if disp argument is $\leq 0$
+| DISPERSION\_EST | Dispersion estimated from the dataset
+| DEVIANCE\_UNSCALED | Deviance from the saturated model, assuming dispersion $= 1.0$
+| DEVIANCE\_SCALED | Deviance from the saturated model, scaled by DISPERSION value
+
+
+* * *
+
+<a name="table10" />
+**Table 10**: The Log file for GLM regression contains the below
+iteration variables in CSV format, each line containing a triple (Name,
+Iteration\#, Value) with Iteration\# being 0 for initial values.
+
+| Name | Meaning |
+| --------------------- | ------- |
+| NUM\_CG\_ITERS | Number of inner (Conj. Gradient) iterations in this outer iteration
+| IS\_TRUST\_REACHED | 1 = trust region boundary was reached, 0 = otherwise
+| POINT\_STEP\_NORM | L2-norm of iteration step from old point ($\beta$-vector) to new point
+| OBJECTIVE | The loss function we minimize (negative partial log-likelihood)
+| OBJ\_DROP\_REAL | Reduction in the objective during this iteration, actual value
+| OBJ\_DROP\_PRED | Reduction in the objective predicted by a quadratic approximation
+| OBJ\_DROP\_RATIO | Actual-to-predicted reduction ratio, used to update the trust region
+| GRADIENT\_NORM | L2-norm of the loss function gradient (omitted if point is rejected)
+| LINEAR\_TERM\_MIN | The minimum value of $X$ %\*% $\beta$, used to check for overflows
+| LINEAR\_TERM\_MAX | The maximum value of $X$ %\*% $\beta$, used to check for overflows
+| IS\_POINT\_UPDATED | 1 = new point accepted; 0 = new point rejected, old point restored
+| TRUST\_DELTA | Updated trust region size, the “delta”
+
+
+* * *
+
+<a name="table11" />
+**Table 11**: Common GLM distribution families and link functions. (Here “\*” stands
+for “any value.”)
+
+| dfam | vpow | link | lpow | Distribution<br />Family | Link<br /> Function | Canonical |
+| :--: | :--: | :--: | :--: | :------------: | :--------: | :----: |
+| 1 | 0.0 | 1 | -1.0 | Gaussian | inverse |
+| 1 | 0.0 | 1 | 0.0 | Gaussian | log |
+| 1 | 0.0 | 1 | 1.0 | Gaussian | identity | Yes
+| 1 | 1.0 | 1 | 0.0 | Poisson | log | Yes
+| 1 | 1.0 | 1 | 0.5 | Poisson | sq.root |
+| 1 | 1.0 | 1 | 1.0 | Poisson | identity |
+| 1 | 2.0 | 1 | -1.0 | Gamma | inverse | Yes
+| 1 | 2.0 | 1 | 0.0 | Gamma | log |
+| 1 | 2.0 | 1 | 1.0 | Gamma | identity |
+| 1 | 3.0 | 1 | -2.0 | Inverse Gauss | $1/\mu^2$ | Yes
+| 1 | 3.0 | 1 | -1.0 | Inverse Gauss | inverse |
+| 1 | 3.0 | 1 | 0.0 | Inverse Gauss | log |
+| 1 | 3.0 | 1 | 1.0 | Inverse Gauss | identity |
+| 2 | \* | 1 | 0.0 | Binomial | log |
+| 2 | \* | 1 | 0.5 | Binomial | sq.root |
+| 2 | \* | 2 | \* | Binomial | logit | Yes
+| 2 | \* | 3 | \* | Binomial | probit |
+| 2 | \* | 4 | \* | Binomial | cloglog |
+| 2 | \* | 5 | \* | Binomial | cauchit |
+
+
+* * *
+
+<a name="table12" />
+**Table 12**: The supported non-power link functions for the Bernoulli and the
+binomial distributions. Here $\mu$ is the Bernoulli mean.
+
+| Name | Link Function |
+| --------------------- | ------------- |
+| Logit | $\displaystyle \eta = 1 / \big(1 + e^{-\mu}\big)^{\mathstrut}$
+| Probit | $\displaystyle \mu = \frac{1}{\sqrt{2\pi}}\int\nolimits_{-\infty_{\mathstrut}}^{\,\eta\mathstrut} e^{-\frac{t^2}{2}} dt$
+| Cloglog | $\displaystyle \eta = \log \big(- \log(1 - \mu)\big)^{\mathstrut}$
+| Cauchit | $\displaystyle \eta = \tan\pi(\mu - 1/2)$
+
+
+* * *
+
+
+### Details
+
+In GLM, the noise distribution
+$Prob[y\mid \mu]$
+of the response variable $y$ given its mean $\mu$ is restricted to have
+the *exponential family* form
+
+$$
+\begin{equation}
+Y \sim\, Prob[y\mid \mu] \,=\, \exp\left(\frac{y\theta - b(\theta)}{a}
++ c(y, a)\right),\,\,\textrm{where}\,\,\,\mu = E(Y) = b'(\theta).
+\end{equation}
+$$
+
+Changing the mean in such a distribution simply
+multiplies all
+$Prob[y\mid \mu]$
+by $e^{\,y\hspace{0.2pt}\theta/a}$ and rescales them so
+that they again integrate to 1. Parameter $\theta$ is called
+*canonical*, and the function $\theta = b'^{\,-1}(\mu)$ that relates it
+to the mean is called the *canonical link*; constant $a$ is called
+*dispersion* and rescales the variance of $y$. Many common distributions
+can be put into this form, see [**Table 11**](algorithms-regression.html#table11). The canonical
+parameter $\theta$ is often chosen to coincide with $\eta$, the linear
+combination of the regression features; other choices for $\eta$ are
+possible too.
+
+Rather than specifying the canonical link, GLM distributions are
+commonly defined by their variance
+$Var(y)$ as the function of
+the mean $\mu$. It can be shown from Eq. 5 that
+$Var(y) = a\,b''(\theta) = a\,b''(b'^{\,-1}(\mu))$.
+For example, for the Bernoulli distribution
+$Var(y) = \mu(1-\mu)$, for the
+Poisson distribution
+$Var(y) = \mu$, and for the Gaussian distribution
+$Var(y) = a\cdot 1 = \sigma^2$.
+It turns out that for many common distributions
+$Var(y) = a\mu^q$, a power
+function. We support all distributions where
+$Var(y) = a\mu^q$, as well as
+the Bernoulli and the binomial distributions.
+
+For distributions with
+$Var(y) = a\mu^q$ the
+canonical link is also a power function, namely
+$\theta = \mu^{1-q}/(1-q)$, except for the Poisson ($q = 1$) whose
+canonical link is $\theta = \log\mu$. We support all power link
+functions in the form $\eta = \mu^s$, dropping any constant factor, with
+$\eta = \log\mu$ for $s=0$. The binomial distribution has its own family
+of link functions, which includes logit (the canonical link), probit,
+cloglog, and cauchit (see [**Table 12**](algorithms-regression.html#table12)); we support
+these only for the binomial and Bernoulli distributions. Links and
+distributions are specified via four input parameters:
+`dfam`, `vpow`, `link`, and
+`lpow` (see [**Table 11**](algorithms-regression.html#table11)).
+
+The observed response values are provided to the regression script as a
+matrix $Y$ having 1 or 2 columns. If a power distribution family is
+selected (`dfam=1`), matrix $Y$ must have 1 column that
+provides $y_i$ for each $x_i$ in the corresponding row of matrix $X$.
+When dfam=2 and $Y$ has 1 column, we assume the Bernoulli
+distribution for $y_i\in\{y_{\mathrm{neg}}, 1\}$ with $y_{\mathrm{neg}}$
+from the input parameter `yneg`. When `dfam=2` and
+$Y$ has 2 columns, we assume the binomial distribution; for each row $i$
+in $X$, cells $Y[i, 1]$ and $Y[i, 2]$ provide the positive and the
+negative binomial counts respectively. Internally we convert the
+1-column Bernoulli into the 2-column binomial with 0-versus-1 counts.
+
+We estimate the regression parameters via L2-regularized negative
+log-likelihood minimization:
+
+$$f(\beta; X, Y) \,\,=\,\, -\sum\nolimits_{i=1}^n \big(y_i\theta_i - b(\theta_i)\big)
+\,+\,(\lambda/2) \sum\nolimits_{j=1}^m \beta_j^2\,\,\to\,\,\min$$
+
+where
+$\theta_i$ and $b(\theta_i)$ are from (5); note that $a$ and
+$c(y, a)$ are constant w.r.t. $\beta$ and can be ignored here. The
+canonical parameter $\theta_i$ depends on both $\beta$ and $x_i$:
+
+$$\theta_i \,\,=\,\, b'^{\,-1}(\mu_i) \,\,=\,\, b'^{\,-1}\big(g^{-1}(\eta_i)\big) \,\,=\,\,
+\big(b'^{\,-1}\circ g^{-1}\big)\left(\beta_0 + \sum\nolimits_{j=1}^m \beta_j x_{i,j}\right)$$
+
+The user-provided (via `reg`) regularization coefficient
+$\lambda\geq 0$ can be used to mitigate overfitting and degeneracy in
+the data. Note that the intercept is never regularized.
+
+Our iterative minimizer for $f(\beta; X, Y)$ uses the Fisher scoring
+approximation to the difference
+$\varDelta f(z; \beta) = f(\beta + z; X, Y) \,-\, f(\beta; X, Y)$,
+recomputed at each iteration:
+
+$$\begin{gathered}
+\varDelta f(z; \beta) \,\,\,\approx\,\,\, 1/2 \cdot z^T A z \,+\, G^T z,
+\,\,\,\,\textrm{where}\,\,\,\, A \,=\, X^T\!diag(w) X \,+\, \lambda I\\
+\textrm{and}\,\,\,\,G \,=\, - X^T u \,+\, \lambda\beta,
+\,\,\,\textrm{with $n\,{\times}\,1$ vectors $w$ and $u$ given by}\\
+\forall\,i = 1\ldots n: \,\,\,\,
+w_i = \big[v(\mu_i)\,g'(\mu_i)^2\big]^{-1}
+\!\!\!\!\!\!,\,\,\,\,\,\,\,\,\,
+u_i = (y_i - \mu_i)\big[v(\mu_i)\,g'(\mu_i)\big]^{-1}
+\!\!\!\!\!\!.\,\,\,\,\end{gathered}$$
+
+Here
+$v(\mu_i)=Var(y_i)/a$, the
+variance of $y_i$ as the function of the mean, and
+$g'(\mu_i) = d \eta_i/d \mu_i$ is the link function derivative. The
+Fisher scoring approximation is minimized by trust-region conjugate
+gradient iterations (called the *inner* iterations, with the Fisher
+scoring iterations as the *outer* iterations), which approximately solve
+the following problem:
+
+$$1/2 \cdot z^T A z \,+\, G^T z \,\,\to\,\,\min\,\,\,\,\textrm{subject to}\,\,\,\,
+\|z\|_2 \leq \delta$$
+
+The conjugate gradient algorithm closely follows
+Algorithm 7.2 on page 171 of [[Nocedal2006]](algorithms-bibliography.html). The trust region
+size $\delta$ is initialized as
+$0.5\sqrt{m}\,/ \max\nolimits_i \|x_i\|_2$ and updated as described
+in [[Nocedal2006]](algorithms-bibliography.html).
+The user can specify the maximum number of
+the outer and the inner iterations with input parameters
+moi and mii, respectively. The Fisher scoring
+algorithm terminates successfully if
+$2|\varDelta f(z; \beta)| < (D_1(\beta) + 0.1)\hspace{0.5pt}{\varepsilon}$
+where ${\varepsilon}> 0$ is a tolerance supplied by the user via
+`tol`, and $D_1(\beta)$ is the unit-dispersion deviance
+estimated as
+
+$$D_1(\beta) \,\,=\,\, 2 \cdot \big(Prob[Y \mid \!
+\begin{smallmatrix}\textrm{saturated}\\\textrm{model}\end{smallmatrix}, a\,{=}\,1]
+\,\,-\,\,Prob[Y \mid X, \beta, a\,{=}\,1]\,\big)$$
+
+The deviance estimate is also produced as part of the output. Once the
+Fisher scoring algorithm terminates, if requested by the user, we
+estimate the dispersion $a$ from Eq. 5 using Pearson residuals
+
+$$
+\begin{equation}
+\hat{a} \,\,=\,\, \frac{1}{n-m}\cdot \sum_{i=1}^n \frac{(y_i - \mu_i)^2}{v(\mu_i)}
+\end{equation}
+$$
+
+and use it to adjust our deviance estimate:
+$D_{\hat{a}}(\beta) = D_1(\beta)/\hat{a}$. If input argument
+disp is 0.0 we estimate $\hat{a}$, otherwise
+we use its value as $a$. Note that in (6) $m$ counts
+the intercept ($m \leftarrow m+1$) if it is present.
+
+
+### Returns
+
+The estimated regression parameters (the $\hat{\beta}_j$’s) are
+populated into a matrix and written to an HDFS file whose path/name was
+provided as the `B` input argument. What this matrix
+contains, and its size, depends on the input argument `icpt`,
+which specifies the user’s intercept and rescaling choice:
+
+ * **icpt=0**: No intercept, matrix $B$ has size $m\,{\times}\,1$, with
+$B[j, 1] = \hat{\beta}_j$ for each $j$ from 1 to $m = {}$ncol$(X)$.
+ * **icpt=1**: There is intercept, but no shifting/rescaling of $X$; matrix $B$ has
+size $(m\,{+}\,1) \times 1$, with $B[j, 1] = \hat{\beta}_j$ for $j$ from
+1 to $m$, and $B[m\,{+}\,1, 1] = \hat{\beta}_0$, the estimated intercept
+coefficient.
+ * **icpt=2**: There is intercept, and the features in $X$ are shifted to mean${} = 0$
+and rescaled to variance${} = 1$; then there are two versions of
+the $\hat{\beta}_j$’s, one for the original features and another for the
+shifted/rescaled features. Now matrix $B$ has size
+$(m\,{+}\,1) \times 2$, with $B[\cdot, 1]$ for the original features and
+$B[\cdot, 2]$ for the shifted/rescaled features, in the above format.
+Note that $B[\cdot, 2]$ are iteratively estimated and $B[\cdot, 1]$ are
+obtained from $B[\cdot, 2]$ by complementary shifting and rescaling.
+
+Our script also estimates the dispersion $\hat{a}$ (or takes it from the
+user’s input) and the deviances $D_1(\hat{\beta})$ and
+$D_{\hat{a}}(\hat{\beta})$, see [**Table 9**](algorithms-regression.html#table9) for details. A
+log file with variables monitoring progress through the iterations can
+also be made available, see [**Table 10**](algorithms-regression.html#table10).
+
+
+### See Also
+
+In case of binary classification problems, consider using L2-SVM or
+binary logistic regression; for multiclass classification, use
+multiclass SVM or multinomial logistic regression. For the special cases
+of linear regression and logistic regression, it may be more efficient
+to use the corresponding specialized scripts instead of GLM.
+
+
+* * *
+
+## 4.3. Regression Scoring and Prediction
+
+### Description
+
+Script `GLM-predict.dml` is intended to cover all linear
+model based regressions, including linear regression, binomial and
+multinomial logistic regression, and GLM regressions (Poisson, gamma,
+binomial with probit link etc.). Having just one scoring script for all
+these regressions simplifies maintenance and enhancement while ensuring
+compatible interpretations for output statistics.
+
+The script performs two functions, prediction and scoring. To perform
+prediction, the script takes two matrix inputs: a collection of records
+$X$ (without the response attribute) and the estimated regression
+parameters $B$, also known as $\beta$. To perform scoring, in addition
+to $X$ and $B$, the script takes the matrix of actual response
+values $Y$ that are compared to the predictions made with $X$ and $B$.
+Of course there are other, non-matrix, input arguments that specify the
+model and the output format, see below for the full list.
+
+We assume that our test/scoring dataset is given by
+$n\,{\times}\,m$-matrix $X$ of numerical feature vectors, where each
+row $x_i$ represents one feature vector of one record; we have $\dim x_i = m$. Each
+record also includes the response variable $y_i$ that may be numerical,
+single-label categorical, or multi-label categorical. A single-label
+categorical $y_i$ is an integer category label, one label per record; a
+multi-label $y_i$ is a vector of integer counts, one count for each
+possible label, which represents multiple single-label events
+(observations) for the same $x_i$. Internally we convert single-label
+categoricals into multi-label categoricals by replacing each label $l$
+with an indicator vector $(0,\ldots,0,1_l,0,\ldots,0)$. In
+prediction-only tasks the actual $y_i$’s are not needed to the script,
+but they are needed for scoring.
+
+To perform prediction, the script matrix-multiplies $X$ and $B$, adding
+the intercept if available, then applies the inverse of the model’s link
+function. All GLMs assume that the linear combination of the features
+in $x_i$ and the betas in $B$ determines the means $\mu_i$ of
+the $y_i$’s (in numerical or multi-label categorical form) with
+$\dim \mu_i = \dim y_i$. The observed $y_i$ is assumed to follow a
+specified GLM family distribution
+$Prob[y\mid \mu_i]$
+with mean(s) $\mu_i$:
+
+$$x_i \,\,\,\,\mapsto\,\,\,\, \eta_i = \beta_0 + \sum\nolimits_{j=1}^m \beta_j x_{i,j}
+\,\,\,\,\mapsto\,\,\,\, \mu_i \,\,\,\,\mapsto \,\,\,\, y_i \sim Prob[y\mid \mu_i]$$
+
+If $y_i$ is numerical, the predicted mean $\mu_i$ is a real number. Then
+our script’s output matrix $M$ is the $n\,{\times}\,1$-vector of these
+means $\mu_i$. Note that $\mu_i$ predicts the mean of $y_i$, not the
+actual $y_i$. For example, in Poisson distribution, the mean is usually
+fractional, but the actual $y_i$ is always integer.
+
+If $y_i$ is categorical, i.e. a vector of label counts for record $i$,
+then $\mu_i$ is a vector of non-negative real numbers, one number
+$\mu_{i,l}$ per each label $l$. In this case we divide the $\mu_{i,l}$
+by their sum $\sum_l \mu_{i,l}$ to obtain predicted label
+probabilities . The output matrix $M$ is the
+$n \times (k\,{+}\,1)$-matrix of these probabilities, where $n$ is the
+number of records and $k\,{+}\,1$ is the number of categories[^3]. Note
+again that we do not predict the labels themselves, nor their actual
+counts per record, but we predict the labels’ probabilities.
+
+Going from predicted probabilities to predicted labels, in the
+single-label categorical case, requires extra information such as the
+cost of false positive versus false negative errors. For example, if
+there are 5 categories and we *accurately* predicted their probabilities
+as $(0.1, 0.3, 0.15, 0.2, 0.25)$, just picking the highest-probability
+label would be wrong 70% of the time, whereas picking the
+lowest-probability label might be right if, say, it represents a
+diagnosis of cancer or another rare and serious outcome. Hence, we keep
+this step outside the scope of `GLM-predict.dml` for now.
+
+
+### Usage
+
+ hadoop jar SystemML.jar -f GLM-predict.dml
+ -nvargs X=file
+ Y=file
+ B=file
+ M=file
+ O=file
+ dfam=int
+ vpow=double
+ link=int
+ lpow=double
+ disp=double
+ fmt=format
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read the $n\,{\times}\,m$-matrix $X$ of feature
+vectors, each row constitutes one feature vector (one record)
+
+**Y**: (default: `" "`) Location to read the response matrix $Y$ needed for scoring
+(but optional for prediction), with the following dimensions:
+
+ * $n {\times} 1$: acceptable for all distributions
+(`dfam=1` or `2` or `3`)
+ * $n {\times} 2$: for binomial (`dfam=2`) if given by
+(\#pos, \#neg) counts
+ * $n {\times} k\,{+}\,1$: for multinomial (`dfam=3`) if
+given by category counts
+
+**M**: (default: `" "`) Location to write, if requested, the matrix of predicted
+response means (for `dfam=1`) or probabilities (for
+`dfam=2` or `3`):
+
+ * $n {\times} 1$: for power-type distributions (`dfam=1`)
+ * $n {\times} 2$: for binomial distribution (`dfam=2`),
+col\# 2 is the “No” probability
+ * $n {\times} k\,{+}\,1$: for multinomial logit (`dfam=3`),
+col\# $k\,{+}\,1$ is for the baseline
+
+**B**: Location to read matrix $B$ of the betas, i.e. estimated GLM regression
+parameters, with the intercept at row\# $m\,{+}\,1$ if available:
+
+ * $\dim(B) \,=\, m {\times} k$: do not add intercept
+ * $\dim(B) \,=\, m\,{+}\,1 {\times} k$: add intercept as given by the
+last $B$-row
+ * if $k > 1$, use only $B[, 1]$ unless it is Multinomial Logit
+(`dfam=3`)
+
+**O**: (default: `" "`) Location to store the CSV-file with goodness-of-fit
+statistics defined in [**Table 13**](algorithms-regression.html#table13),
+the default is to
+print them to the standard output
+
+**dfam**: (default: 1) GLM distribution family code to specify the type of
+distribution
+$Prob[y\,|\,\mu]$
+that we assume:
+
+ * 1 = power distributions with
+$Var(y) = \mu^{\alpha}$, see
+[**Table 11**](algorithms-regression.html#table11)
+ * 2 = binomial
+ * 3 = multinomial logit
+
+**vpow**: (default: 0.0) Power for variance defined as (mean)$^{\textrm{power}}$
+(ignored if `dfam`$\,{\neq}\,1$): when `dfam=1`,
+this provides the $q$ in
+$Var(y) = a\mu^q$, the power
+dependence of the variance of $y$ on its mean. In particular, use:
+
+ * 0.0 = Gaussian
+ * 1.0 = Poisson
+ * 2.0 = Gamma
+ * 3.0 = inverse Gaussian
+
+**link**: (default: 0) Link function code to determine the link
+function $\eta = g(\mu)$, ignored for multinomial logit
+(`dfam=3`):
+
+ * 0 = canonical link (depends on the distribution family),
+see [**Table 11**](algorithms-regression.html#table11)
+ * 1 = power functions
+ * 2 = logit
+ * 3 = probit
+ * 4 = cloglog
+ * 5 = cauchit
+
+**lpow**: (default: 1.0) Power for link function defined as
+(mean)$^{\textrm{power}}$ (ignored if `link`$\,{\neq}\,1$):
+when `link=1`, this provides the $s$ in $\eta = \mu^s$, the
+power link function; `lpow=0.0` gives the log link
+$\eta = \log\mu$. Common power links:
+
+ * -2.0 = $1/\mu^2$
+ * -1.0 = reciprocal
+ * 0.0 = log
+ * 0.5 = sqrt
+ * 1.0 = identity
+
+**disp**: (default: 1.0) Dispersion value, when available; must be positive
+
+**fmt**: (default: `"text"`) Matrix M file output format, such as
+`text`, `mm`, or `csv`; see read/write
+functions in SystemML Language Reference for details.
+
+
+### Examples
+
+Note that in the examples below the value for the `disp` input
+argument is set arbitrarily. The correct dispersion value should be
+computed from the training data during model estimation, or omitted if
+unknown (which sets it to 1.0).
+
+Linear regression example:
+
+ hadoop jar SystemML.jar -f GLM-predict.dml
+ -nvargs dfam=1
+ vpow=0.0
+ link=1
+ lpow=1.0
+ disp=5.67
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Means.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+
+
+Linear regression example, prediction only (no Y given):
+
+ hadoop jar SystemML.jar -f GLM-predict.dml
+ -nvargs dfam=1
+ vpow=0.0
+ link=1
+ lpow=1.0
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Means.mtx
+ fmt=csv
+
+Binomial logistic regression example:
+
+ hadoop jar SystemML.jar -f GLM-predict.dml
+ -nvargs dfam=2
+ link=2
+ disp=3.0004464
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Probabilities.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+
+Binomial probit regression example:
+
+ hadoop jar SystemML.jar -f GLM-predict.dml
+ -nvargs dfam=2
+ link=3
+ disp=3.0004464
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Probabilities.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+
+Multinomial logistic regression example:
+
+ hadoop jar SystemML.jar -f GLM-predict.dml
+ -nvargs dfam=3
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Probabilities.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+
+Poisson regression with the log link example:
+
+ hadoop jar SystemML.jar -f GLM-predict.dml
+ -nvargs dfam=1
+ vpow=1.0
+ link=1
+ lpow=0.0
+ disp=3.45
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Means.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+
+Gamma regression with the inverse (reciprocal) link example:
+
+ hadoop jar SystemML.jar -f GLM-predict.dml
+ -nvargs dfam=1
+ vpow=2.0
+ link=1
+ lpow=-1.0
+ disp=1.99118
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Means.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+
+
+* * *
+
+<a name="table13" />
+**Table 13**: The goodness-of-fit statistics are provided in CSV format, one per each line, with four
+columns: (Name, CID, Disp?, Value). The columns are:
+“Name” is the string identifier for the statistic;
+“CID” is an optional integer value that specifies the $Y$-column index for per-column statistics
+(note that a bi/multinomial one-column Y-input is converted into multi-column);
+“Disp?” is an optional Boolean value ($TRUE$ or $FALSE$) that tells us
+whether or not scaling by the input dispersion parameter `disp` has been applied to this
+statistic;
+“Value” is the value of the statistic.
+
+| Name | CID | Disp? | Meaning |
+| -------------------- | :-: | :---: | ------- |
+| LOGLHOOD\_Z | | + | Log-likelihood $Z$-score (in st. dev.'s from the mean) |
+| LOGLHOOD\_Z\_PVAL | | + | Log-likelihood $Z$-score p-value, two-sided |
+| PEARSON\_X2 | | + | Pearson residual $X^2$-statistic |
+| PEARSON\_X2\_BY\_DF | | + | Pearson $X^2$ divided by degrees of freedom |
+| PEARSON\_X2\_PVAL | | + | Pearson $X^2$ p-value |
+| DEVIANCE\_G2 | | + | Deviance from the saturated model $G^2$-statistic |
+| DEVIANCE\_G2\_BY\_DF | | + | Deviance $G^2$ divided by degrees of freedom |
+| DEVIANCE\_G2\_PVAL | | + | Deviance $G^2$ p-value |
+| AVG\_TOT\_Y | + | | $Y$-column average for an individual response value |
+| STDEV\_TOT\_Y | + | | $Y$-column st. dev. for an individual response value |
+| AVG\_RES\_Y | + | | $Y$-column residual average of $Y - pred. mean(Y\\|X)$ |
+| STDEV\_RES\_Y | + | | $Y$-column residual st. dev. of $Y - pred. mean(Y\\|X)$ |
+| PRED\_STDEV\_RES | + | + | Model-predicted $Y$-column residual st. deviation|
+| PLAIN\_R2 | + | | Plain $R^2$ of $Y$-column residual with bias included |
+| ADJUSTED\_R2 | + | | Adjusted $R^2$ of $Y$-column residual w. bias included |
+| PLAIN\_R2\_NOBIAS | + | | Plain $R^2$ of $Y$-column residual, bias subtracted |
+| ADJUSTED\_R2\_NOBIAS | + | | Adjusted $R^2$ of $Y$-column residual, bias subtracted |
+
+* * *
+
+
+### Details
+
+The output matrix $M$ of predicted means (or probabilities) is computed
+by matrix-multiplying $X$ with the first column of $B$ or with the
+whole $B$ in the multinomial case, adding the intercept if available
+(conceptually, appending an extra column of ones to $X$); then applying
+the inverse of the model’s link function. The difference between “means”
+and “probabilities” in the categorical case becomes significant when
+there are ${\geq}\,2$ observations per record (with the multi-label
+records) or when the labels such as $-1$ and $1$ are viewed and averaged
+as numerical response values (with the single-label records). To avoid
+any or information loss, we separately return the predicted probability
+of each category label for each record.
+
+When the “actual” response values $Y$ are available, the summary
+statistics are computed and written out as described in
+[**Table 13**](algorithms-regression.html#table13). Below we discuss each of these statistics
+in detail. Note that in the categorical case (binomial and multinomial)
+$Y$ is internally represented as the matrix of observation counts for
+each label in each record, rather than just the label ID for each
+record. The input $Y$ may already be a matrix of counts, in which case
+it is used as-is. But if $Y$ is given as a vector of response labels,
+each response label is converted into an indicator vector
+$(0,\ldots,0,1_l,0,\ldots,0)$ where $l$ is the label ID for this record.
+All negative (e.g. $-1$) or zero label IDs are converted to the
+$1 +$ maximum label ID. The largest label ID is viewed as the
+“baseline” as explained in the section on Multinomial Logistic
+Regression. We assume that there are $k\geq 1$ non-baseline categories
+and one (last) baseline category.
+
+We also estimate residual variances for each response value, although we
+do not output them, but use them only inside the summary statistics,
+scaled and unscaled by the input dispersion parameter `disp`,
+as described below.
+
+`LOGLHOOD_Z` and `LOGLHOOD_Z_PVAL` statistics measure how far the
+log-likelihood of $Y$ deviates from its expected value according to the
+model. The script implements them only for the binomial and the
+multinomial distributions, returning `NaN` for all other distributions.
+Pearson’s $X^2$ and deviance $G^2$ often perform poorly with bi- and
+multinomial distributions due to low cell counts, hence we need this
+extra goodness-of-fit measure. To compute these statistics, we use:
+
+ * the $n\times (k\,{+}\,1)$-matrix $Y$ of multi-label response counts, in
+which $y_{i,j}$ is the number of times label $j$ was observed in
+record $i$
+ * the model-estimated probability matrix $P$ of the same dimensions that
+satisfies $\sum_{j=1}^{k+1} p_{i,j} = 1$ for all $i=1,\ldots,n$ and
+where $p_{i,j}$ is the model probability of observing label $j$ in
+record $i$
+ * the $n\,{\times}\,1$-vector $N$ where $N_i$ is the aggregated count of
+observations in record $i$ (all $N_i = 1$ if each record has only one
+response label)
+
+We start by computing the multinomial log-likelihood of $Y$ given $P$
+and $N$, as well as the expected log-likelihood given a random $Y$ and
+the variance of this log-likelihood if $Y$ indeed follows the proposed
+distribution:
+
+$$
+\begin{aligned}
+\ell (Y) \,\,&=\,\, \log Prob[Y \,|\, P, N] \,\,=\,\, \sum_{i=1}^{n} \,\sum_{j=1}^{k+1} \,y_{i,j}\log p_{i,j} \\
+E_Y \ell (Y) \,\,&=\,\, \sum_{i=1}^{n}\, \sum_{j=1}^{k+1} \,\mu_{i,j} \log p_{i,j}
+ \,\,=\,\, \sum_{i=1}^{n}\, N_i \,\sum_{j=1}^{k+1} \,p_{i,j} \log p_{i,j} \\
+Var_Y \ell (Y) \,&=\, \sum_{i=1}^{n} \,N_i \left(\sum_{j=1}^{k+1} \,p_{i,j} \big(\log p_{i,j}\big)^2
+ - \Bigg( \sum_{j=1}^{k+1} \,p_{i,j} \log p_{i,j}\Bigg) ^ {\!\!2\,} \right)
+\end{aligned}
+$$
+
+
+Then we compute the $Z$-score as the difference between the actual and
+the expected log-likelihood $\ell(Y)$ divided by its expected standard
+deviation, and its two-sided p-value in the Normal distribution
+assumption ($\ell(Y)$ should approach normality due to the Central Limit
+Theorem):
+
+$$
+Z \,=\, \frac {\ell(Y) - E_Y \ell(Y)}{\sqrt{Var_Y \ell(Y)}};\quad
+\mathop{\textrm{p-value}}(Z) \,=\, Prob\Big[\,\big|\mathop{\textrm{Normal}}(0,1)\big| \, > \, |Z|\,\Big]
+$$
+
+A low p-value would indicate “underfitting” if $Z\ll 0$ or “overfitting”
+if $Z\gg 0$. Here “overfitting” means that higher-probability labels
+occur more often than their probabilities suggest.
+
+We also apply the dispersion input (`disp`) to compute the
+“scaled” version of the $Z$-score and its p-value. Since $\ell(Y)$ is a
+linear function of $Y$, multiplying the GLM-predicted variance of $Y$ by
+`disp` results in multiplying
+$Var_Y \ell(Y)$ by the same
+`disp`. This, in turn, translates into dividing the $Z$-score
+by the square root of the dispersion:
+
+$$Z_{\texttt{disp}} \,=\, \big(\ell(Y) \,-\, E_Y \ell(Y)\big) \,\big/\, \sqrt{\texttt{disp}\cdot Var_Y \ell(Y)}
+\,=\, Z / \sqrt{\texttt{disp}}$$
+
+Finally, we recalculate the p-value
+with this new $Z$-score.
+
+`PEARSON_X2`, `PEARSON_X2_BY_DF`, and `PEARSON_X2_PVAL`:
+Pearson’s residual $X^2$-statistic is a commonly used goodness-of-fit
+measure for linear models [[McCullagh1989]](algorithms-bibliography.html).
+The idea is to measure how
+well the model-predicted means and variances match the actual behavior
+of response values. For each record $i$, we estimate the mean $\mu_i$
+and the variance $v_i$ (or `disp` $\cdot v_i$) and use them to
+normalize the residual: $r_i = (y_i - \mu_i) / \sqrt{v_i}$. These
+normalized residuals are then squared, aggregated by summation, and
+tested against an appropriate $\chi^2$ distribution. The computation
+of $X^2$ is slightly different for categorical data (bi- and
+multinomial) than it is for numerical data, since $y_i$ has multiple
+correlated dimensions [[McCullagh1989]](algorithms-bibliography.html):
+
+$$X^2\,\textrm{(numer.)} \,=\, \sum_{i=1}^{n}\, \frac{(y_i - \mu_i)^2}{v_i};\quad
+X^2\,\textrm{(categ.)} \,=\, \sum_{i=1}^{n}\, \sum_{j=1}^{k+1} \,\frac{(y_{i,j} - N_i
+\hspace{0.5pt} p_{i,j})^2}{N_i \hspace{0.5pt} p_{i,j}}$$
+
+The number of
+degrees of freedom \#d.f. for the $\chi^2$ distribution is $n - m$ for
+numerical data and $(n - m)k$ for categorical data, where
+$k = \mathop{\texttt{ncol}}(Y) - 1$. Given the dispersion parameter
+`disp` the $X^2$ statistic is scaled by division: $X^2_{\texttt{disp}} = X^2 / \texttt{disp}$. If the
+dispersion is accurate, $X^2 / \texttt{disp}$ should be close to \#d.f.
+In fact, $X^2 / \textrm{\#d.f.}$ over the *training* data is the
+dispersion estimator used in our `GLM.dml` script,
+see (6). Here we provide $X^2 / \textrm{\#d.f.}$ and
+$X^2_{\texttt{disp}} / \textrm{\#d.f.}$ as
+`PEARSON_X2_BY_DF` to enable dispersion comparison between
+the training data and the test data.
+
+NOTE: For categorical data, both Pearson’s $X^2$ and the deviance $G^2$
+are unreliable (i.e. do not approach the $\chi^2$ distribution) unless
+the predicted means of multi-label counts
+$\mu_{i,j} = N_i \hspace{0.5pt} p_{i,j}$ are fairly large: all
+${\geq}\,1$ and 80% are at least $5$ [[Cochran1954]](algorithms-bibliography.html). They should not
+be used for “one label per record” categoricals.
+
+`DEVIANCE_G2`, `DEVIANCE_G2_BY_DF`, and
+`DEVIANCE_G2_PVAL`: Deviance $G^2$ is the log of the
+likelihood ratio between the “saturated” model and the linear model
+being tested for the given dataset, multiplied by two:
+
+$$
+\begin{equation}
+G^2 \,=\, 2 \,\log \frac{Prob[Y \mid \textrm{saturated model}\hspace{0.5pt}]}{Prob[Y \mid \textrm{tested linear model}\hspace{0.5pt}]}
+\end{equation}
+$$
+
+The “saturated” model sets the mean
+$\mu_i^{\mathrm{sat}}$ to equal $y_i$ for every record (for categorical
+data, $p_{i,j}^{sat} = y_{i,j} / N_i$), which represents the
+“perfect fit.” For records with $y_{i,j} \in \{0, N_i\}$ or otherwise at
+a boundary, by continuity we set $0 \log 0 = 0$. The GLM likelihood
+functions defined in (5) become simplified in
+ratio (7) due to canceling out the term $c(y, a)$
+since it is the same in both models.
+
+The log of a likelihood ratio between two nested models, times two, is
+known to approach a $\chi^2$ distribution as $n\to\infty$ if both models
+have fixed parameter spaces. But this is not the case for the
+“saturated” model: it adds more parameters with each record. In
+practice, however, $\chi^2$ distributions are used to compute the
+p-value of $G^2$ [[McCullagh1989]](algorithms-bibliography.html). The number of degrees of
+freedom \#d.f. and the treatment of dispersion are the same as for
+Pearson’s $X^2$, see above.
+
+
+### Column-Wise Statistics
+
+The rest of the statistics are computed separately for each column
+of $Y$. As explained above, $Y$ has two or more columns in bi- and
+multinomial case, either at input or after conversion. Moreover, each
+$y_{i,j}$ in record $i$ with $N_i \geq 2$ is counted as $N_i$ separate
+observations $y_{i,j,l}$ of 0 or 1 (where $l=1,\ldots,N_i$) with
+$y_{i,j}$ ones and $N_i-y_{i,j}$ zeros. For power distributions,
+including linear regression, $Y$ has only one column and all $N_i = 1$,
+so the statistics are computed for all $Y$ with each record counted
+once. Below we denote $N = \sum_{i=1}^n N_i \,\geq n$. Here is the total
+average and the residual average (residual bias) of $y_{i,j,l}$ for each
+$Y$-column:
+
+$$\texttt{AVG_TOT_Y}_j \,=\, \frac{1}{N} \sum_{i=1}^n y_{i,j}; \quad
+\texttt{AVG_RES_Y}_j \,=\, \frac{1}{N} \sum_{i=1}^n \, (y_{i,j} - \mu_{i,j})$$
+
+Dividing by $N$ (rather than $n$) gives the averages for $y_{i,j,l}$
+(rather than $y_{i,j}$). The total variance, and the standard deviation,
+for individual observations $y_{i,j,l}$ is estimated from the total
+variance for response values $y_{i,j}$ using independence assumption:
+$Var \,y_{i,j} = Var \sum_{l=1}^{N_i} y_{i,j,l} = \sum_{l=1}^{N_i} Var y_{i,j,l}$.
+This allows us to estimate the sum of squares for $y_{i,j,l}$ via the
+sum of squares for $y_{i,j}$:
+
+$$\texttt{STDEV_TOT_Y}_j \,=\,
+\Bigg[\frac{1}{N-1} \sum_{i=1}^n \Big( y_{i,j} - \frac{N_i}{N} \sum_{i'=1}^n y_{i'\!,j}\Big)^2\Bigg]^{1/2}$$
+
+Analogously, we estimate the standard deviation of the residual
+$y_{i,j,l} - \mu_{i,j,l}$:
+
+$$\texttt{STDEV_RES_Y}_j \,=\,
+\Bigg[\frac{1}{N-m'} \,\sum_{i=1}^n \Big( y_{i,j} - \mu_{i,j} - \frac{N_i}{N} \sum_{i'=1}^n (y_{i'\!,j} - \mu_{i'\!,j})\Big)^2\Bigg]^{1/2}$$
+
+Here $m'=m$ if $m$ includes the intercept as a feature and $m'=m+1$ if
+it does not. The estimated standard deviations can be compared to the
+model-predicted residual standard deviation computed from the predicted
+means by the GLM variance formula and scaled by the dispersion:
+
+$$\texttt{PRED_STDEV_RES}_j \,=\, \Big[\frac{\texttt{disp}}{N} \, \sum_{i=1}^n \, v(\mu_{i,j})\Big]^{1/2}$$
+
+We also compute the $R^2$ statistics for each column of $Y$, see
+[**Table 14**](algorithms-regression.html#table14) and [**Table 15**](algorithms-regression.html#table15) for details. We compute two versions
+of $R^2$: in one version the residual sum-of-squares (RSS) includes any
+bias in the residual that might be present (due to the lack of, or
+inaccuracy in, the intercept); in the other version of RSS the bias is
+subtracted by “centering” the residual. In both cases we subtract the
+bias in the total sum-of-squares (in the denominator), and $m'$ equals
+$m$ with the intercept or $m+1$ without the intercept.
+
+
+* * *
+
+<a name="table14" />
+**Table 14**: $R^2$ where the residual sum-of-squares includes the bias contribution.
+
+| Statistic | Formula |
+| --------------------- | ------------- |
+| $\texttt{PLAIN_R2}_j$ | $ \displaystyle 1 - \frac{\sum\limits_{i=1}^n \,(y_{i,j} - \mu_{i,j})^2}{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n y_{i',j} \Big)^{2}} $
+| $\texttt{ADJUSTED_R2}_j$ | $ \displaystyle 1 - {\textstyle\frac{N_{\mathstrut} - 1}{N^{\mathstrut} - m}} \, \frac{\sum\limits_{i=1}^n \,(y_{i,j} - \mu_{i,j})^2}{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n y_{i',j} \Big)^{2}} $
+
+
+* * *
+
+<a name="table15" />
+**Table 15**: $R^2$ where the residual sum-of-squares is centered so that the bias is subtracted.
+
+| Statistic | Formula |
+| --------------------- | ------------- |
+| $\texttt{PLAIN_R2_NOBIAS}_j$ | $ \displaystyle 1 - \frac{\sum\limits_{i=1}^n \Big(y_{i,j} \,{-}\, \mu_{i,j} \,{-}\, \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n (y_{i',j} \,{-}\, \mu_{i',j}) \Big)^{2}}{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n y_{i',j} \Big)^{2}} $
+| $\texttt{ADJUSTED_R2_NOBIAS}_j$ | $ \displaystyle 1 - {\textstyle\frac{N_{\mathstrut} - 1}{N^{\mathstrut} - m'}} \, \frac{\sum\limits_{i=1}^n \Big(y_{i,j} \,{-}\, \mu_{i,j} \,{-}\, \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n (y_{i',j} \,{-}\, \mu_{i',j}) \Big)^{2}}{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n y_{i',j} \Big)^{2}} $
+
+
+* * *
+
+
+### Returns
+
+The matrix of predicted means (if the response is numerical) or
+probabilities (if the response is categorical), see “Description”
+subsection above for more information. Given Y, we return
+some statistics in CSV format as described in
+[**Table 13**](algorithms-regression.html#table13) and in the above text.
+
+
+* * *
+
+
+[^1]: Smaller likelihood difference between two models suggests less
+ statistical evidence to pick one model over the other.
+
+[^2]: $Prob[y\mid \mu_i]$
+ is given by a density function if $y$ is continuous.
+
+[^3]: We use $k+1$ because there are $k$ non-baseline categories and one
+ baseline category, with regression parameters $B$ having
+ $k$ columns.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/css/bootstrap-responsive.css
----------------------------------------------------------------------
diff --git a/css/bootstrap-responsive.css b/css/bootstrap-responsive.css
new file mode 100644
index 0000000..daafa91
--- /dev/null
+++ b/css/bootstrap-responsive.css
@@ -0,0 +1,1040 @@
+/*!
+ * Bootstrap Responsive v2.1.0
+ *
+ * Copyright 2012 Twitter, Inc
+ * Licensed under the Apache License v2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Designed and built with all the love in the world @twitter by @mdo and @fat.
+ */
+
+.clearfix {
+ *zoom: 1;
+}
+
+.clearfix:before,
+.clearfix:after {
+ display: table;
+ line-height: 0;
+ content: "";
+}
+
+.clearfix:after {
+ clear: both;
+}
+
+.hide-text {
+ font: 0/0 a;
+ color: transparent;
+ text-shadow: none;
+ background-color: transparent;
+ border: 0;
+}
+
+.input-block-level {
+ display: block;
+ width: 100%;
+ min-height: 30px;
+ -webkit-box-sizing: border-box;
+ -moz-box-sizing: border-box;
+ box-sizing: border-box;
+}
+
+.hidden {
+ display: none;
+ visibility: hidden;
+}
+
+.visible-phone {
+ display: none !important;
+}
+
+.visible-tablet {
+ display: none !important;
+}
+
+.hidden-desktop {
+ display: none !important;
+}
+
+.visible-desktop {
+ display: inherit !important;
+}
+
+@media (min-width: 768px) and (max-width: 979px) {
+ .hidden-desktop {
+ display: inherit !important;
+ }
+ .visible-desktop {
+ display: none !important ;
+ }
+ .visible-tablet {
+ display: inherit !important;
+ }
+ .hidden-tablet {
+ display: none !important;
+ }
+}
+
+@media (max-width: 767px) {
+ .hidden-desktop {
+ display: inherit !important;
+ }
+ .visible-desktop {
+ display: none !important;
+ }
+ .visible-phone {
+ display: inherit !important;
+ }
+ .hidden-phone {
+ display: none !important;
+ }
+}
+
+@media (min-width: 1200px) {
+ .row {
+ margin-left: -30px;
+ *zoom: 1;
+ }
+ .row:before,
+ .row:after {
+ display: table;
+ line-height: 0;
+ content: "";
+ }
+ .row:after {
+ clear: both;
+ }
+ [class*="span"] {
+ float: left;
+ margin-left: 30px;
+ }
+ .container,
+ .navbar-static-top .container,
+ .navbar-fixed-top .container,
+ .navbar-fixed-bottom .container {
+ width: 1170px;
+ }
+ .span12 {
+ width: 1170px;
+ }
+ .span11 {
+ width: 1070px;
+ }
+ .span10 {
+ width: 970px;
+ }
+ .span9 {
+ width: 870px;
+ }
+ .span8 {
+ width: 770px;
+ }
+ .span7 {
+ width: 670px;
+ }
+ .span6 {
+ width: 570px;
+ }
+ .span5 {
+ width: 470px;
+ }
+ .span4 {
+ width: 370px;
+ }
+ .span3 {
+ width: 270px;
+ }
+ .span2 {
+ width: 170px;
+ }
+ .span1 {
+ width: 70px;
+ }
+ .offset12 {
+ margin-left: 1230px;
+ }
+ .offset11 {
+ margin-left: 1130px;
+ }
+ .offset10 {
+ margin-left: 1030px;
+ }
+ .offset9 {
+ margin-left: 930px;
+ }
+ .offset8 {
+ margin-left: 830px;
+ }
+ .offset7 {
+ margin-left: 730px;
+ }
+ .offset6 {
+ margin-left: 630px;
+ }
+ .offset5 {
+ margin-left: 530px;
+ }
+ .offset4 {
+ margin-left: 430px;
+ }
+ .offset3 {
+ margin-left: 330px;
+ }
+ .offset2 {
+ margin-left: 230px;
+ }
+ .offset1 {
+ margin-left: 130px;
+ }
+ .row-fluid {
+ width: 100%;
+ *zoom: 1;
+ }
+ .row-fluid:before,
+ .row-fluid:after {
+ display: table;
+ line-height: 0;
+ content: "";
+ }
+ .row-fluid:after {
+ clear: both;
+ }
+ .row-fluid [class*="span"] {
+ display: block;
+ float: left;
+ width: 100%;
+ min-height: 30px;
+ margin-left: 2.564102564102564%;
+ *margin-left: 2.5109110747408616%;
+ -webkit-box-sizing: border-box;
+ -moz-box-sizing: border-box;
+ box-sizing: border-box;
+ }
+ .row-fluid [class*="span"]:first-child {
+ margin-left: 0;
+ }
+ .row-fluid .span12 {
+ width: 100%;
+ *width: 99.94680851063829%;
+ }
+ .row-fluid .span11 {
+ width: 91.45299145299145%;
+ *width: 91.39979996362975%;
+ }
+ .row-fluid .span10 {
+ width: 82.90598290598291%;
+ *width: 82.8527914166212%;
+ }
+ .row-fluid .span9 {
+ width: 74.35897435897436%;
+ *width: 74.30578286961266%;
+ }
+ .row-fluid .span8 {
+ width: 65.81196581196582%;
+ *width: 65.75877432260411%;
+ }
+ .row-fluid .span7 {
+ width: 57.26495726495726%;
+ *width: 57.21176577559556%;
+ }
+ .row-fluid .span6 {
+ width: 48.717948717948715%;
+ *width: 48.664757228587014%;
+ }
+ .row-fluid .span5 {
+ width: 40.17094017094017%;
+ *width: 40.11774868157847%;
+ }
+ .row-fluid .span4 {
+ width: 31.623931623931625%;
+ *width: 31.570740134569924%;
+ }
+ .row-fluid .span3 {
+ width: 23.076923076923077%;
+ *width: 23.023731587561375%;
+ }
+ .row-fluid .span2 {
+ width: 14.52991452991453%;
+ *width: 14.476723040552828%;
+ }
+ .row-fluid .span1 {
+ width: 5.982905982905983%;
+ *width: 5.929714493544281%;
+ }
+ .row-fluid .offset12 {
+ margin-left: 105.12820512820512%;
+ *margin-left: 105.02182214948171%;
+ }
+ .row-fluid .offset12:first-child {
+ margin-left: 102.56410256410257%;
+ *margin-left: 102.45771958537915%;
+ }
+ .row-fluid .offset11 {
+ margin-left: 96.58119658119658%;
+ *margin-left: 96.47481360247316%;
+ }
+ .row-fluid .offset11:first-child {
+ margin-left: 94.01709401709402%;
+ *margin-left: 93.91071103837061%;
+ }
+ .row-fluid .offset10 {
+ margin-left: 88.03418803418803%;
+ *margin-left: 87.92780505546462%;
+ }
+ .row-fluid .offset10:first-child {
+ margin-left: 85.47008547008548%;
+ *margin-left: 85.36370249136206%;
+ }
+ .row-fluid .offset9 {
+ margin-left: 79.48717948717949%;
+ *margin-left: 79.38079650845607%;
+ }
+ .row-fluid .offset9:first-child {
+ margin-left: 76.92307692307693%;
+ *margin-left: 76.81669394435352%;
+ }
+ .row-fluid .offset8 {
+ margin-left: 70.94017094017094%;
+ *margin-left: 70.83378796144753%;
+ }
+ .row-fluid .offset8:first-child {
+ margin-left: 68.37606837606839%;
+ *margin-left: 68.26968539734497%;
+ }
+ .row-fluid .offset7 {
+ margin-left: 62.393162393162385%;
+ *margin-left: 62.28677941443899%;
+ }
+ .row-fluid .offset7:first-child {
+ margin-left: 59.82905982905982%;
+ *margin-left: 59.72267685033642%;
+ }
+ .row-fluid .offset6 {
+ margin-left: 53.84615384615384%;
+ *margin-left: 53.739770867430444%;
+ }
+ .row-fluid .offset6:first-child {
+ margin-left: 51.28205128205128%;
+ *margin-left: 51.175668303327875%;
+ }
+ .row-fluid .offset5 {
+ margin-left: 45.299145299145295%;
+ *margin-left: 45.1927623204219%;
+ }
+ .row-fluid .offset5:first-child {
+ margin-left: 42.73504273504273%;
+ *margin-left: 42.62865975631933%;
+ }
+ .row-fluid .offset4 {
+ margin-left: 36.75213675213675%;
+ *margin-left: 36.645753773413354%;
+ }
+ .row-fluid .offset4:first-child {
+ margin-left: 34.18803418803419%;
+ *margin-left: 34.081651209310785%;
+ }
+ .row-fluid .offset3 {
+ margin-left: 28.205128205128204%;
+ *margin-left: 28.0987452264048%;
+ }
+ .row-fluid .offset3:first-child {
+ margin-left: 25.641025641025642%;
+ *margin-left: 25.53464266230224%;
+ }
+ .row-fluid .offset2 {
+ margin-left: 19.65811965811966%;
+ *margin-left: 19.551736679396257%;
+ }
+ .row-fluid .offset2:first-child {
+ margin-left: 17.094017094017094%;
+ *margin-left: 16.98763411529369%;
+ }
+ .row-fluid .offset1 {
+ margin-left: 11.11111111111111%;
+ *margin-left: 11.004728132387708%;
+ }
+ .row-fluid .offset1:first-child {
+ margin-left: 8.547008547008547%;
+ *margin-left: 8.440625568285142%;
+ }
+ input,
+ textarea,
+ .uneditable-input {
+ margin-left: 0;
+ }
+ .controls-row [class*="span"] + [class*="span"] {
+ margin-left: 30px;
+ }
+ input.span12,
+ textarea.span12,
+ .uneditable-input.span12 {
+ width: 1156px;
+ }
+ input.span11,
+ textarea.span11,
+ .uneditable-input.span11 {
+ width: 1056px;
+ }
+ input.span10,
+ textarea.span10,
+ .uneditable-input.span10 {
+ width: 956px;
+ }
+ input.span9,
+ textarea.span9,
+ .uneditable-input.span9 {
+ width: 856px;
+ }
+ input.span8,
+ textarea.span8,
+ .uneditable-input.span8 {
+ width: 756px;
+ }
+ input.span7,
+ textarea.span7,
+ .uneditable-input.span7 {
+ width: 656px;
+ }
+ input.span6,
+ textarea.span6,
+ .uneditable-input.span6 {
+ width: 556px;
+ }
+ input.span5,
+ textarea.span5,
+ .uneditable-input.span5 {
+ width: 456px;
+ }
+ input.span4,
+ textarea.span4,
+ .uneditable-input.span4 {
+ width: 356px;
+ }
+ input.span3,
+ textarea.span3,
+ .uneditable-input.span3 {
+ width: 256px;
+ }
+ input.span2,
+ textarea.span2,
+ .uneditable-input.span2 {
+ width: 156px;
+ }
+ input.span1,
+ textarea.span1,
+ .uneditable-input.span1 {
+ width: 56px;
+ }
+ .thumbnails {
+ margin-left: -30px;
+ }
+ .thumbnails > li {
+ margin-left: 30px;
+ }
+ .row-fluid .thumbnails {
+ margin-left: 0;
+ }
+}
+
+@media (min-width: 768px) and (max-width: 979px) {
+ .row {
+ margin-left: -20px;
+ *zoom: 1;
+ }
+ .row:before,
+ .row:after {
+ display: table;
+ line-height: 0;
+ content: "";
+ }
+ .row:after {
+ clear: both;
+ }
+ [class*="span"] {
+ float: left;
+ margin-left: 20px;
+ }
+ .container,
+ .navbar-static-top .container,
+ .navbar-fixed-top .container,
+ .navbar-fixed-bottom .container {
+ width: 724px;
+ }
+ .span12 {
+ width: 724px;
+ }
+ .span11 {
+ width: 662px;
+ }
+ .span10 {
+ width: 600px;
+ }
+ .span9 {
+ width: 538px;
+ }
+ .span8 {
+ width: 476px;
+ }
+ .span7 {
+ width: 414px;
+ }
+ .span6 {
+ width: 352px;
+ }
+ .span5 {
+ width: 290px;
+ }
+ .span4 {
+ width: 228px;
+ }
+ .span3 {
+ width: 166px;
+ }
+ .span2 {
+ width: 104px;
+ }
+ .span1 {
+ width: 42px;
+ }
+ .offset12 {
+ margin-left: 764px;
+ }
+ .offset11 {
+ margin-left: 702px;
+ }
+ .offset10 {
+ margin-left: 640px;
+ }
+ .offset9 {
+ margin-left: 578px;
+ }
+ .offset8 {
+ margin-left: 516px;
+ }
+ .offset7 {
+ margin-left: 454px;
+ }
+ .offset6 {
+ margin-left: 392px;
+ }
+ .offset5 {
+ margin-left: 330px;
+ }
+ .offset4 {
+ margin-left: 268px;
+ }
+ .offset3 {
+ margin-left: 206px;
+ }
+ .offset2 {
+ margin-left: 144px;
+ }
+ .offset1 {
+ margin-left: 82px;
+ }
+ .row-fluid {
+ width: 100%;
+ *zoom: 1;
+ }
+ .row-fluid:before,
+ .row-fluid:after {
+ display: table;
+ line-height: 0;
+ content: "";
+ }
+ .row-fluid:after {
+ clear: both;
+ }
+ .row-fluid [class*="span"] {
+ display: block;
+ float: left;
+ width: 100%;
+ min-height: 30px;
+ margin-left: 2.7624309392265194%;
+ *margin-left: 2.709239449864817%;
+ -webkit-box-sizing: border-box;
+ -moz-box-sizing: border-box;
+ box-sizing: border-box;
+ }
+ .row-fluid [class*="span"]:first-child {
+ margin-left: 0;
+ }
+ .row-fluid .span12 {
+ width: 100%;
+ *width: 99.94680851063829%;
+ }
+ .row-fluid .span11 {
+ width: 91.43646408839778%;
+ *width: 91.38327259903608%;
+ }
+ .row-fluid .span10 {
+ width: 82.87292817679558%;
+ *width: 82.81973668743387%;
+ }
+ .row-fluid .span9 {
+ width: 74.30939226519337%;
+ *width: 74.25620077583166%;
+ }
+ .row-fluid .span8 {
+ width: 65.74585635359117%;
+ *width: 65.69266486422946%;
+ }
+ .row-fluid .span7 {
+ width: 57.18232044198895%;
+ *width: 57.12912895262725%;
+ }
+ .row-fluid .span6 {
+ width: 48.61878453038674%;
+ *width: 48.56559304102504%;
+ }
+ .row-fluid .span5 {
+ width: 40.05524861878453%;
+ *width: 40.00205712942283%;
+ }
+ .row-fluid .span4 {
+ width: 31.491712707182323%;
+ *width: 31.43852121782062%;
+ }
+ .row-fluid .span3 {
+ width: 22.92817679558011%;
+ *width: 22.87498530621841%;
+ }
+ .row-fluid .span2 {
+ width: 14.3646408839779%;
+ *width: 14.311449394616199%;
+ }
+ .row-fluid .span1 {
+ width: 5.801104972375691%;
+ *width: 5.747913483013988%;
+ }
+ .row-fluid .offset12 {
+ margin-left: 105.52486187845304%;
+ *margin-left: 105.41847889972962%;
+ }
+ .row-fluid .offset12:first-child {
+ margin-left: 102.76243093922652%;
+ *margin-left: 102.6560479605031%;
+ }
+ .row-fluid .offset11 {
+ margin-left: 96.96132596685082%;
+ *margin-left: 96.8549429881274%;
+ }
+ .row-fluid .offset11:first-child {
+ margin-left: 94.1988950276243%;
+ *margin-left: 94.09251204890089%;
+ }
+ .row-fluid .offset10 {
+ margin-left: 88.39779005524862%;
+ *margin-left: 88.2914070765252%;
+ }
+ .row-fluid .offset10:first-child {
+ margin-left: 85.6353591160221%;
+ *margin-left: 85.52897613729868%;
+ }
+ .row-fluid .offset9 {
+ margin-left: 79.8342541436464%;
+ *margin-left: 79.72787116492299%;
+ }
+ .row-fluid .offset9:first-child {
+ margin-left: 77.07182320441989%;
+ *margin-left: 76.96544022569647%;
+ }
+ .row-fluid .offset8 {
+ margin-left: 71.2707182320442%;
+ *margin-left: 71.16433525332079%;
+ }
+ .row-fluid .offset8:first-child {
+ margin-left: 68.50828729281768%;
+ *margin-left: 68.40190431409427%;
+ }
+ .row-fluid .offset7 {
+ margin-left: 62.70718232044199%;
+ *margin-left: 62.600799341718584%;
+ }
+ .row-fluid .offset7:first-child {
+ margin-left: 59.94475138121547%;
+ *margin-left: 59.838368402492065%;
+ }
+ .row-fluid .offset6 {
+ margin-left: 54.14364640883978%;
+ *margin-left: 54.037263430116376%;
+ }
+ .row-fluid .offset6:first-child {
+ margin-left: 51.38121546961326%;
+ *margin-left: 51.27483249088986%;
+ }
+ .row-fluid .offset5 {
+ margin-left: 45.58011049723757%;
+ *margin-left: 45.47372751851417%;
+ }
+ .row-fluid .offset5:first-child {
+ margin-left: 42.81767955801105%;
+ *margin-left: 42.71129657928765%;
+ }
+ .row-fluid .offset4 {
+ margin-left: 37.01657458563536%;
+ *margin-left: 36.91019160691196%;
+ }
+ .row-fluid .offset4:first-child {
+ margin-left: 34.25414364640884%;
+ *margin-left: 34.14776066768544%;
+ }
+ .row-fluid .offset3 {
+ margin-left: 28.45303867403315%;
+ *margin-left: 28.346655695309746%;
+ }
+ .row-fluid .offset3:first-child {
+ margin-left: 25.69060773480663%;
+ *margin-left: 25.584224756083227%;
+ }
+ .row-fluid .offset2 {
+ margin-left: 19.88950276243094%;
+ *margin-left: 19.783119783707537%;
+ }
+ .row-fluid .offset2:first-child {
+ margin-left: 17.12707182320442%;
+ *margin-left: 17.02068884448102%;
+ }
+ .row-fluid .offset1 {
+ margin-left: 11.32596685082873%;
+ *margin-left: 11.219583872105325%;
+ }
+ .row-fluid .offset1:first-child {
+ margin-left: 8.56353591160221%;
+ *margin-left: 8.457152932878806%;
+ }
+ input,
+ textarea,
+ .uneditable-input {
+ margin-left: 0;
+ }
+ .controls-row [class*="span"] + [class*="span"] {
+ margin-left: 20px;
+ }
+ input.span12,
+ textarea.span12,
+ .uneditable-input.span12 {
+ width: 710px;
+ }
+ input.span11,
+ textarea.span11,
+ .uneditable-input.span11 {
+ width: 648px;
+ }
+ input.span10,
+ textarea.span10,
+ .uneditable-input.span10 {
+ width: 586px;
+ }
+ input.span9,
+ textarea.span9,
+ .uneditable-input.span9 {
+ width: 524px;
+ }
+ input.span8,
+ textarea.span8,
+ .uneditable-input.span8 {
+ width: 462px;
+ }
+ input.span7,
+ textarea.span7,
+ .uneditable-input.span7 {
+ width: 400px;
+ }
+ input.span6,
+ textarea.span6,
+ .uneditable-input.span6 {
+ width: 338px;
+ }
+ input.span5,
+ textarea.span5,
+ .uneditable-input.span5 {
+ width: 276px;
+ }
+ input.span4,
+ textarea.span4,
+ .uneditable-input.span4 {
+ width: 214px;
+ }
+ input.span3,
+ textarea.span3,
+ .uneditable-input.span3 {
+ width: 152px;
+ }
+ input.span2,
+ textarea.span2,
+ .uneditable-input.span2 {
+ width: 90px;
+ }
+ input.span1,
+ textarea.span1,
+ .uneditable-input.span1 {
+ width: 28px;
+ }
+}
+
+@media (max-width: 767px) {
+ body {
+ padding-right: 20px;
+ padding-left: 20px;
+ }
+ .navbar-fixed-top,
+ .navbar-fixed-bottom {
+ margin-right: -20px;
+ margin-left: -20px;
+ }
+ .container-fluid {
+ padding: 0;
+ }
+ .dl-horizontal dt {
+ float: none;
+ width: auto;
+ clear: none;
+ text-align: left;
+ }
+ .dl-horizontal dd {
+ margin-left: 0;
+ }
+ .container {
+ width: auto;
+ }
+ .row-fluid {
+ width: 100%;
+ }
+ .row,
+ .thumbnails {
+ margin-left: 0;
+ }
+ .thumbnails > li {
+ float: none;
+ margin-left: 0;
+ }
+ [class*="span"],
+ .row-fluid [class*="span"] {
+ display: block;
+ float: none;
+ width: auto;
+ margin-left: 0;
+ }
+ .span12,
+ .row-fluid .span12 {
+ width: 100%;
+ -webkit-box-sizing: border-box;
+ -moz-box-sizing: border-box;
+ box-sizing: border-box;
+ }
+ .input-large,
+ .input-xlarge,
+ .input-xxlarge,
+ input[class*="span"],
+ select[class*="span"],
+ textarea[class*="span"],
+ .uneditable-input {
+ display: block;
+ width: 100%;
+ min-height: 30px;
+ -webkit-box-sizing: border-box;
+ -moz-box-sizing: border-box;
+ box-sizing: border-box;
+ }
+ .input-prepend input,
+ .input-append input,
+ .input-prepend input[class*="span"],
+ .input-append input[class*="span"] {
+ display: inline-block;
+ width: auto;
+ }
+ .modal {
+ position: fixed;
+ top: 20px;
+ right: 20px;
+ left: 20px;
+ width: auto;
+ margin: 0;
+ }
+ .modal.fade.in {
+ top: auto;
+ }
+}
+
+@media (max-width: 480px) {
+ .nav-collapse {
+ -webkit-transform: translate3d(0, 0, 0);
+ }
+ .page-header h1 small {
+ display: block;
+ line-height: 20px;
+ }
+ input[type="checkbox"],
+ input[type="radio"] {
+ border: 1px solid #ccc;
+ }
+ .form-horizontal .control-group > label {
+ float: none;
+ width: auto;
+ padding-top: 0;
+ text-align: left;
+ }
+ .form-horizontal .controls {
+ margin-left: 0;
+ }
+ .form-horizontal .control-list {
+ padding-top: 0;
+ }
+ .form-horizontal .form-actions {
+ padding-right: 10px;
+ padding-left: 10px;
+ }
+ .modal {
+ top: 10px;
+ right: 10px;
+ left: 10px;
+ }
+ .modal-header .close {
+ padding: 10px;
+ margin: -10px;
+ }
+ .carousel-caption {
+ position: static;
+ }
+}
+
+@media (max-width: 979px) {
+ body {
+ padding-top: 0;
+ }
+ .navbar-fixed-top,
+ .navbar-fixed-bottom {
+ position: static;
+ }
+ .navbar-fixed-top {
+ margin-bottom: 20px;
+ }
+ .navbar-fixed-bottom {
+ margin-top: 20px;
+ }
+ .navbar-fixed-top .navbar-inner,
+ .navbar-fixed-bottom .navbar-inner {
+ padding: 5px;
+ }
+ .navbar .container {
+ width: auto;
+ padding: 0;
+ }
+ .navbar .brand {
+ padding-right: 10px;
+ padding-left: 10px;
+ margin: 0 0 0 -5px;
+ }
+ .nav-collapse {
+ clear: both;
+ }
+ .nav-collapse .nav {
+ float: none;
+ margin: 0 0 10px;
+ }
+ .nav-collapse .nav > li {
+ float: none;
+ }
+ .nav-collapse .nav > li > a {
+ margin-bottom: 2px;
+ }
+ .nav-collapse .nav > .divider-vertical {
+ display: none;
+ }
+ .nav-collapse .nav .nav-header {
+ color: #555555;
+ text-shadow: none;
+ }
+ .nav-collapse .nav > li > a,
+ .nav-collapse .dropdown-menu a {
+ padding: 9px 15px;
+ font-weight: bold;
+ color: #555555;
+ -webkit-border-radius: 3px;
+ -moz-border-radius: 3px;
+ border-radius: 3px;
+ }
+ .nav-collapse .btn {
+ padding: 4px 10px 4px;
+ font-weight: normal;
+ -webkit-border-radius: 4px;
+ -moz-border-radius: 4px;
+ border-radius: 4px;
+ }
+ .nav-collapse .dropdown-menu li + li a {
+ margin-bottom: 2px;
+ }
+ .nav-collapse .nav > li > a:hover,
+ .nav-collapse .dropdown-menu a:hover {
+ background-color: #f2f2f2;
+ }
+ .navbar-inverse .nav-collapse .nav > li > a:hover,
+ .navbar-inverse .nav-collapse .dropdown-menu a:hover {
+ background-color: #111111;
+ }
+ .nav-collapse.in .btn-group {
+ padding: 0;
+ margin-top: 5px;
+ }
+ .nav-collapse .dropdown-menu {
+ position: static;
+ top: auto;
+ left: auto;
+ display: block;
+ float: none;
+ max-width: none;
+ padding: 0;
+ margin: 0 15px;
+ background-color: transparent;
+ border: none;
+ -webkit-border-radius: 0;
+ -moz-border-radius: 0;
+ border-radius: 0;
+ -webkit-box-shadow: none;
+ -moz-box-shadow: none;
+ box-shadow: none;
+ }
+ .nav-collapse .dropdown-menu:before,
+ .nav-collapse .dropdown-menu:after {
+ display: none;
+ }
+ .nav-collapse .dropdown-menu .divider {
+ display: none;
+ }
+ .nav-collapse .navbar-form,
+ .nav-collapse .navbar-search {
+ float: none;
+ padding: 10px 15px;
+ margin: 10px 0;
+ border-top: 1px solid #f2f2f2;
+ border-bottom: 1px solid #f2f2f2;
+ -webkit-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1), 0 1px 0 rgba(255, 255, 255, 0.1);
+ -moz-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1), 0 1px 0 rgba(255, 255, 255, 0.1);
+ box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1), 0 1px 0 rgba(255, 255, 255, 0.1);
+ }
+ .navbar .nav-collapse .nav.pull-right {
+ float: none;
+ margin-left: 0;
+ }
+ .nav-collapse,
+ .nav-collapse.collapse {
+ height: 0;
+ overflow: hidden;
+ }
+ .navbar .btn-navbar {
+ display: block;
+ }
+ .navbar-static .navbar-inner {
+ padding-right: 10px;
+ padding-left: 10px;
+ }
+}
+
+@media (min-width: 980px) {
+ .nav-collapse.collapse {
+ height: auto !important;
+ overflow: visible !important;
+ }
+}
[18/47] incubator-systemml git commit: initial conversion of dml
language ref to md
Posted by du...@apache.org.
initial conversion of dml language ref to md
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/f0a60f05
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/f0a60f05
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/f0a60f05
Branch: refs/heads/gh-pages
Commit: f0a60f0506a1cddf5830abb6411084aa20d838ba
Parents: d47e76d
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Fri Aug 21 17:32:53 2015 -0700
Committer: Deron Eriksson <de...@us.ibm.com>
Committed: Fri Aug 21 17:32:53 2015 -0700
----------------------------------------------------------------------
_layouts/global.html | 2 +-
dml-language-reference.md | 1256 ++++++++++++++++++++++++++++++++++++++++
index.md | 5 +-
3 files changed, 1260 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/f0a60f05/_layouts/global.html
----------------------------------------------------------------------
diff --git a/_layouts/global.html b/_layouts/global.html
index 73551a4..6499efb 100644
--- a/_layouts/global.html
+++ b/_layouts/global.html
@@ -36,7 +36,7 @@
<li><a href="http://www.github.com/SparkTC/systemml">SystemML GitHub README</a></li>
<li><a href="algorithms-reference.html">Algorithms Reference</a></li>
- <!-- <li><a href="dml-language-reference.html">DML Language Reference</a></li> -->
+ <li><a href="dml-language-reference.html">DML Language Reference</a></li>
<li class="divider"></li>
<li> More Coming Soon...</li>
</ul>
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/f0a60f05/dml-language-reference.md
----------------------------------------------------------------------
diff --git a/dml-language-reference.md b/dml-language-reference.md
new file mode 100644
index 0000000..2824f97
--- /dev/null
+++ b/dml-language-reference.md
@@ -0,0 +1,1256 @@
+---
+layout: global
+title: Declarative Machine Learning (DML) Language Reference
+description: Declarative Machine Learning (DML) Language Reference
+---
+
+
+### Table of Contents
+
+ * [Introduction](#introduction)
+ * [Variables](#variables)
+ * [Identifier Names](#identifier-names)
+ * [Data Types](#data-types)
+ * [Comments](#comments)
+ * [Expressions](#expressions)
+ * [Operators](#operators)
+ * [Matrix-Vector Operations](#matrix-vector-operations)
+ * [Matrix Indexing](#matrix-indexing)
+ * [Statements](#statements)
+ * [Assignment Statement](#assignment-statement)
+ * [Control Statements](#control-statements)
+ * [While Statement](#while-statement)
+ * [If Statement](#if-statement)
+ * [For Statement](#for-statement)
+ * [ParFor Statement](#parfor-statement)
+ * [User-Defined Function (UDF)](#user-defined-function-udf)
+ * [Variable Scoping](#variable-scoping)
+ * [Command-Line Arguments](#command-line-arguments)
+ * [Built-in Functions](#built-in-functions)
+ * [Matrix Construction, Manipulation, and Aggregation Built-In Functions](#matrix-construction-manipulation-and-aggregation-built-in-functions)
+ * [Matrix and/or Scalar Comparison Built-In Functions](#matrix-andor-scalar-comparison-built-in-functions)
+ * [Casting Built-In Functions](#casting-built-in-functions)
+ * [Statistical Built-In Functions](#statistical-built-in-functions)
+ * [Mathematical and Trigonometric Built-In Functions](#mathematical-and-trigonometric-built-in-functions)
+ * [Linear Algebra Built-In Functions](#linear-algebra-built-in-functions)
+ * [Read/Write Built-In Functions](#readwrite-built-in-functions)
+ * [Data Pre-Processing Built-In Functions](#data-pre-processing-built-in-functions)
+ * [Other Built-In Functions](#other-built-in-functions)
+ * [Modules](#modules)
+ * [Reserved Keywords](#reserved-keywords)
+ * [Invocation of SystemML](#invocation-of-systemml)
+ * [MLContext API](#mlcontext-api)
+
+## Introduction
+
+SystemML compiles scripts written in Declarative Machine Learning (or DML for short) into MapReduce jobs. DML’s syntax closely follows R, thereby minimizing the learning curve to use SystemML. Before getting into detail, let’s start with a simple Hello World program in DML. Assuming that Hadoop is installed on your machine or cluster, place SystemML.jar and SystemML-config.xml into your directory. Now, create a text file “hello.dml” containing following code:
+
+ print(“Hello World”);
+
+
+To run this program on your machine, use following command:
+
+ hadoop jar SystemML.jar –f hello.dml
+
+The option ```-f``` in the above command refers to the path to the DML script. The detailed list of the options is given in the section “Invocation of SystemML”. Note: to install Hadoop, please follow the instructions given at [http://ibm.co/1yHtrdb](http://ibm.co/1yHtrdb).
+
+
+## Variables
+
+### Identifier Names
+
+Identifiers are case-sensitive (e.g., ```var1```, ```Var1```, and ```VAR1``` are different identifier names), must start with either an upper-case or lower-case letter, and may contain any alphanumeric character including underscore after the first letter. The reserved keywords described later cannot be used as identifier names. Though it is allowed, but not recommended to use built-in functions as an identifier. The only exceptions to this rule are five built-in functions: 'as.scalar', 'as.matrix', 'as.double', 'as.integer' and 'as.logical'.
+
+
+#### Examples
+
+ A # valid variable name
+ _A # invalid variable name -- starts with underscore
+ 1_A # invalid variable name -- starts with number
+ A_1 # valid variable name
+ min = 10 # valid but deprecated
+
+Before, proceeding ahead let’s run the Hello World program using variable:
+
+ helloStr = “Hello World”
+ print(helloStr)
+
+As seen in above example, there is no formal declaration of a variable. A variable is created when first assigned a value, and its type is inferred.
+
+### Data Types
+
+Three data types (frame, matrix and scalar) and four value types (double, integer, string, and boolean) are supported. Matrices are 2-dimensional, and support the double value type (i.e., the cells in a matrix are of type double). The frame data type denotes the tabular data, potentially containing columns of value type numeric, string, and boolean, This data type currently supports a single operation, transform(), which transforms the given tabular data with arbitrary value types into a matrix of doubles. SystemML supports type polymorphism for both data type (primarily, matrix and scalar types) and value type during evaluation. For example:
+
+ # Spoiler alert: matrix() is a built-in function to
+ # create matrix, which will be discussed later
+ A = matrix(0, rows=10, cols=10)
+ B = 10
+ C = B + sum(A)
+ print( "B:" + B + ", C:" + C + ", A[1,1]:" + as.scalar(A[1,1]))
+
+In the above script, we create three variables: ```A```, ```B``` and ```C``` of type ```matrix```, ```scalar integer``` and ```scalar double``` respectively. Since ```A``` is a ```matrix```, it has to be converted to scalar using a built-in function ```as.scalar```. In the above script the operator ```+``` used inside ```print()``` function, performs string concatenation. Hence, the output of above script is as follows:
+
+ B:10, C:10.0, A[1,1]:0.0
+
+If instead of ```as.scalar(A[1,1])``` we would have used ```A[1,1]```, then we will get an compilation error ```print statement can only print scalars```.
+
+### Comments
+
+Two forms of commenting are supported: line and block comments. A line comment is indicated using a hash (```#```), and everything to the right of the hash is commented out. A block comment is indicated using "```/*```" to start the comment block and "```*/```" to end it.
+
+#### Examples
+
+ # this is an example of a line comment
+ /* this is an example of a
+ multi-line block comment
+ */
+
+* * *
+
+## Expressions
+
+Now that we have familiarized ourselves with variables and data type, let’s understand how to use them in expressions.
+
+### Operators
+
+SystemML follows same associativity and precedence order as R as described in below table. The dimensions of the input matrices need to match the operator semantics, otherwise an exception will be raised at compile time. When one of the operands is a matrix and the other operand is a scalar value, the operation is performed cell-wise on the matrix using the scalar operand.
+
+Table 1. Operators
+
+| Operator | Input | Output | Details
+| :------: | ----- | ------ | --------
+| ^ | Matrix or Scalar | Matrix or Scalar<sup>1, 2</sup> | Exponentiation (right associativity) – Highest precedence
+| - + | Matrix or Scalar | Matrix or Scalar<sup>1</sup> | Unary plus, minus
+| %*% | Matrix | Matrix | Matrix multiplication
+| %/% %% | Matrix or Scalar | Matrix or Scalar<sup>1, 2</sup> | Integer division and Modulus operator
+| / * | Matrix or Scalar | Matrix or Scalar<sup>1, 2</sup> | Multiplication and Division
+| + - | Matrix or Scalar | Matrix or Scalar<sup>1, 2</sup> | Addition (or string concatenation) and Subtraction
+| < > == != <= >= | Matrix or Scalar (any value type) | Scalar<sup>2</sup> (boolean type) | Relational operators
+| & \| ! | Scalar | Scalar | Boolean operators (Note: operators && and \|\| are not supported)
+| = | - | - | Assignment (Lowest precendence). Note: associativity of assignment “a = b = 3” is not supported
+
+1 If one of the operands is a matrix, output is matrix; otherwise it is scalar.
+
+2 Support for Matrix-vector operations
+
+
+#### Example
+
+ A = matrix(1, rows=2,cols=2)
+ B = matrix(3, rows=2,cols=2)
+ C = 10
+ D = A %*% B + C * 2.1
+ print( "D[1,1]:" + as.scalar(D[1,1]))
+
+Since matrix multiplication has higher precedence than scalar multiplication, which in turns has higher precedence than addition, the first cell of matrix ```D``` is evaluated as ```((1*3)+(1*3))+(10*2.1) = 27.0```.
+
+
+### Matrix-Vector Operations
+
+Arithmetic and relational operations described in above table support matrix-vector operations. This allows efficient cell-wise operations with either row or a column vector.
+
+#### Syntax
+
+ Input_Matrix operation Input_Vector
+
+#### Example
+
+ M + V or M > V, where M is a matrix and V is either row matrix or a column matrix.
+
+
+Matrix-Vector operation avoids need for creating replicated matrix for certain subset of operations. For example: to compute class conditional probabilities in Naïve-Bayes, without support for matrix-vector operations, one might write below given inefficient script that creates unnecessary and possibly huge replicatedClassSums.
+
+ ones = matrix(1, rows=1, cols=numFeatures)
+ repClassSums = classSums %*% ones
+ class_conditionals = (classFeatureCounts + laplace_correction) / repClassSums
+
+With support of matrix-vector operations, the above script becomes much more efficient as well as concise:
+
+ class_conditionals = (classFeatureCounts + laplace_correction) / classSums
+
+
+### Matrix Indexing
+
+Each matrix has a specified number of rows and columns. A 1x1 matrix is not equivalent to a scalar double. The first index for both row and columns in a matrix is 1. For example, a matrix with 10 rows and 10 columns would have rows numbered 1 to 10, and columns numbered 1 to 10.
+
+The elements of the matrix can be accessed by matrix indexing, with both row and column indices required. The indices must either be an expression evaluating to a positive numeric (integer or double) scalar value, or blank. To select the entire row or column of a matrix, leave the appropriate index blank. If a double value is used for indexing, the index value is implicitly cast to an integer with floor (value+eps) in order to account for double inaccuracy (see IEEE754, double precision, eps=pow(2,-53)).
+
+#### Examples
+
+ X[1,4] # access cell in row 1, column 4 of matrix X
+ X[i,j] # access cell in row i, column j of X.
+ X[1,] # access the 1st row of X
+ X[,2] # access the 2nd column of X
+ X[,] # access all rows and columns of X
+
+Range indexing is supported to access a contiguous block of rows and columns in the matrix. The grammar for range-based indexing is below. The constraint is that lower-row < upper-row, and lower-column < upper-column.
+
+ [Matrix name][lower-row : upper-row],[lower-column : upper-column]
+
+#### Examples
+
+ X[1:4, 1:4] # access the 4 x 4 submatrix comprising columns 1 – 4 of rows 1 – 4 of X
+ X[1:4, ] # select the first 4 rows of X
+ X[1:, ] # incorrect format
+
+
+* * *
+
+## Statements
+
+A script is a sequence of statements with the default computation semantics being sequential evaluation of the individual statements. The use of a semi-colon at the end of a statement is optional. The types of statements supported are
+
+ * assignment,
+ * control structures (while, if, for), and
+ * user-defined function declaration.
+
+### Assignment Statement
+
+An assignment statement consists of an expression, the result of which is assigned to a variable. The variable gets the appropriate data type (matrix or scalar) and value type (double, int, string, boolean) depending on the type of the variable output by the expression.
+
+#### Examples
+
+ # max_iteration is of type integer
+ max_iteration = 3;
+ # V has data type matrix and value type double.
+ V = W %*% H;
+
+
+### Control Statements
+
+#### While Statement
+
+The syntax for a while statement is as follows:
+
+ while (predicate){
+ statement1
+ statement2
+ ...
+ }
+
+The statements in the while statement body are evaluated repeatedly until the predicate evaluates to true. The while statement body must be surrounded by braces, even if the body only has a single statement.
+The predicate in the while statement consist of operations on scalar variables and literals. The body of a while statement may contain any sequence of statements.
+
+##### Example
+
+ while( (i < 20) & (!converge) ) {
+ H = H * (t(W) %*% V) / ( t(W) %*% W %*% H);
+ W = W * (V %*% t(H) / (W %*% H %*% t(H));
+ i = i + 1;
+ }
+
+#### If Statement
+
+The syntax for an if statement is as follows:
+
+ if (predicate) {
+ statement1
+ statement2
+ ...
+ } [ else {
+ statement1
+ statement2
+ ...
+ } ]
+
+The If statement has two bodies: the if body (evaluated if the predicate evaluates to true) and the optional else body (evaluated otherwise). Both, the statements in the if body and else body must be surrounded by braces, even if the body only has a single statement. The if body and else body may contain any sequence of statements.
+
+##### Examples
+
+ # example of if statement
+ if( i < 20 ) {
+ converge = false;
+ } else {
+ converge = true;
+ }
+ # example of nested control structures
+ while( !converge ) {
+ H = H * (t(W) %*% V) / ( t(W) %*% W %*% H);
+ W = W * (V %*% t(H) / (W %*% H %*% t(H));
+ i = i + 1;
+ zerror = sum(z - W %*% H);
+ if (zerror < maxError) {
+ converge = true;
+ } else {
+ converge = false;
+ }
+ }
+
+
+#### For Statement
+
+The syntax for a for statement is as follows.
+
+ for (var in <for_predicate> ) {
+ <statement>*
+ }
+ <for_predicate> ::= [lower]:[upper] | seq ([lower], [upper], [increment])
+
+var is an integer scalar variable. lower, upper, and increment are integer expressions.
+
+[lower]:[upper] defines a sequence of numbers with increment 1: {lower, lower + 1, lower + 2, …, upper – 1, upper}.
+
+Similarly, seq([lower],[upper],[increment]) defines a sequence of numbers: {lower, lower + increment, lower + 2(increment), … }. For each element in the sequence, var is assigned the value, and statements in the for loop body are executed.
+
+The for loop body may contain any sequence of statements. The statements in the for statement body must be surrounded by braces, even if the body only has a single statement.
+
+##### Example
+
+ # example for statement
+ A = 5;
+ for (i in 1:20) {
+ A = A + 1;
+ }
+
+#### ParFor Statement
+
+The syntax and semantics of a parfor statement are equivalent to a for statement except for the different keyword and a list of optional parameters.
+
+ parfor (var in <for_predicate> <parfor_paramslist> ) {
+ <statement>*
+ }
+
+ <parfor_paramslist> ::= <,<parfor_parameter>>*
+ <parfor_parameter> ::= check = <dependency_analysis>
+ ||= par = <degree_of_parallelism>
+ ||= mode = <execution_mode>
+ ||= taskpartitioner = <task_partitioning_algorithm>
+ ||= tasksize = <task_size>
+ ||= datapartitioner = <data_partitioning_mode>
+ ||= resultmerge = <result_merge_mode>
+ ||= opt = <optimization_mode>
+
+ <dependency_analysis> is one of the following tokens: 0 1
+ <degree_of_parallelism> is an arbitrary integer number
+ <execution_mode> is one of the following tokens: LOCAL REMOTE_MR
+ <task_partitioning_algorithm> is one of the following tokens: FIXED NAIVE STATIC FACTORING FACTORING_CMIN FACTORING_CMAX
+ <task_size> is an arbitrary integer number
+ <data_partitioning_mode> is one of the following tokens: NONE LOCAL REMOTE_MR
+ <result_merge_mode> is one of the following tokens: LOCAL_MEM LOCAL_FILE LOCAL_AUTOMATIC REMOTE_MR
+ <optimization_mode> is one of the following tokens: NONE RULEBASED HEURISTIC GREEDY FULL_DP
+
+If any of these parameters is not specified, the following respective defaults are used: check = 1, par = [number of virtual processors on master node], mode = LOCAL, taskpartitioner = FIXED, tasksize =1, datapartitioner = NONE, resultmerge = LOCAL_AUTOMATIC, opt = RULEBASED.
+
+### User-Defined Function (UDF)
+
+The UDF function declaration statement provides the function signature, which defines the formal parameters used to call the function and return values for the function. The function definition specifies the function implementation, and can either be a sequence of statements or external packages / libraries. If the UDF is implemented in a SystemML script, then UDF declaration and definition occur together.
+
+The syntax for the UDF function declaration is given as follows. The function definition is stored as a list of statements in the function body. The explanation of the parameters is given below. Any statement can be placed inside a UDF definition except UDF function declaration statements. The variables specified in the return clause will be returned, and no explicit return statement within the function body is required.
+
+ functionName = function([ <DataType>? <ValueType> <var>, ]* )
+ return ([ <DataType>? <ValueType> <var>,]*) {
+ # function body definition in DML
+ statement1
+ statement2
+ ...
+ }
+
+The syntax for the UDF function declaration for functions defined in external packages/ ibraries is given as follows. The parameters are explained below. The main difference is that a user must specify the appropriate collection of userParam=value pairs for the given external package. Also, one of the userParam should be ’classname’.
+
+ functionName = externalFunction(
+ [<DataType>? <ValueType> <var>, ]* )
+ return ([<DataType>? <ValueType> <var>,]*)
+ implemented in ([userParam=value]*)
+
+
+Table 2: Parameters for UDF Function Definition Statements
+
+Parameter Name | Description | Optional | Permissible Values
+-------------- | ----------- | -------- | ------------------
+functionName | Name of the function. | No | Any non-keyword string
+DataType | The data type of the identifier for a formal parameter or return value. | If the value value is scalar or object, then DataType is optional | matrix, scalar, object (capitalization does not matter)
+ValueType | The value type of the identifier for a formal parameter or return value. | No. The value type object can only use used with data type object. | double, integer, string, boolean, object
+Var | The identifier for a formal parameter or return value. | No | Any non-keyword sting
+userParam=value | User-defined parameter to invoke the package. | Yes | Any non-keyword string
+
+
+#### Examples
+
+ # example of a UDF defined in DML
+ mean = function (matrix[double] A) return (double m) {
+ m = sum(A)/nrow(A)
+ }
+ # example of a UDF defined in DML with multiple return values
+ minMax = function( matrix[double] M) return (double minVal, double maxVal) {
+ minVal = min(M);
+ maxVal = max(M);
+ }
+ # example of an external UDF
+ eigen = externalFunction(matrix[double] A)
+ return (matrix[double] evec, matrix[double] eval)
+ implemented in (classname="com.ibm.bi.dml.packagesupport.JLapackEigenWrapper")
+
+A UDF invocation specifies the function identifier, variable identifiers for calling parameters, and the variables to be populated by the returned values from the function. The syntax for function calls is as follows.
+
+ returnVal = functionName( param1, param2, ….)
+ [returnVal1, returnVal2, ...] = functionName(param1, param2, ….)
+
+
+#### Examples
+
+ # DML script with a function call
+ B = matrix(0, rows = 10,cols = 10);
+ C = matrix(0, rows = 100, cols = 100);
+ D = addEach(1, C);
+ index = 0;
+ while (index < 5) {
+ [minD, maxD] = minMax(D);
+ index = index + 1
+ }
+
+
+## Variable Scoping
+
+DML supports following two types of scoping:
+ 1. Default: All the variables are bound to global unbounded scope.
+ 2. Function scope: Only the variables specified in the function declaration can be accessed inside function.
+
+Note: The command-line parameters are treated as constants which are introduced during parse-time.
+
+### Example of Default Scope
+
+ if(1!=0) {
+ A = 1;
+ }
+ print("A:" + A);
+
+This will result in parser warning, but the program will run to completion. If the expression in the “if” predicate would have evaluated to false, it would have resulted in runtime error. Also, functions need not be defined prior to its call. That is: following code will work without parser warning:
+
+ A = 2;
+ C = foo(1, A)
+ print("C:" + C);
+ foo = function(double A, double B) return (double C) {
+ C = A + B;
+ }
+
+### Example of Function Scope
+
+ A = 2;
+ D = 1;
+ foo = function(double A, double B) return (double C) {
+ A = 3.0; # value of global A won’t change since it is pass by value
+
+ C = A + B # Note: C = A + D will result in compilation error
+ }
+ C = foo(A, 1)
+ print("C:" + C + " A:" + A);
+
+The above code will output: ```C:4.0 A:2```
+
+
+## Command-Line Arguments
+
+Since most algorithms require arguments to be passed from command line, DML supports command-line arguments. The command line parameters are treated as constants (similar to arguments passed to main function of a java program). The command line parameters can be passed in two ways:
+
+1. As named arguments (recommended):
+
+ ```-nvargs param1=7 param2=”abc” param3=3.14```
+
+2. As positional arguments (deprecated):
+
+ ```-args 7 “abc” 3.14```
+
+The named arguments can be accessed by adding “\\$” before the parameter name, i.e. \\$param1. On the other hand, the positional parameter are accessible by adding “\\$” before their positions (starting from index 1), i.e. \\$1. A string parameter can be passed without quote. For example, ```param2=abc``` is valid argument, but it is not recommend.
+
+Sometimes the user would want to support default values in case user does not explicitly pass the corresponding command line parameter (in below example: ```$nbrRows```). To do so, we use the ```ifdef``` function which assigns either command line parameter or the default value to the local parameter.
+
+ local_variable = ifdef(command line variable, default value)
+
+### Example: Script in file test.dml
+
+ localVar_nbrRows=ifdef($nbrRows , 10)
+ M = rand (rows = localVar_nbrRows, cols = $nbrCols)
+ write (M, $fname, format="csv")
+ print("Done creating and writing random matrix in " + $fname)
+
+In above script, ```ifdef(\$nbrRows, 10)``` function is a short-hand for “```ifdef(\$nbrRows) then \$nbrRows else 10```”.
+
+Let’s assume that the above script is invoked using following the command line values:
+
+ hadoop jar SystemML.jar -f test.dml -nvargs fname=test.mtx nbrRows=5 nbrCols=5
+
+In this case, the script will create a random matrix M with 5 rows and 5 columns and write it to the file “text.mtx” in csv format. After that it will print the message “Done creating and writing random matrix in test.mtx” on the standard output.
+
+If however, the above script is invoked from the command line using named arguments:
+
+ hadoop jar SystemML.jar -f test.dml -nvargs fname=test.mtx nbrCols=5
+
+Then, the script will instead create a random matrix M with 10 rows (i.e. default value provided in the script) and 5 columns.
+
+It is important to note that the placeholder variables should be treated like constants that are initialized once, either via command line-arguments or via default values at the beginning of the script.
+
+Each argValue passed from the command-line has a scalar data type, and the value type for argValue is inferred using the following logic:
+
+ if (argValue can be cast as Integer)
+ Assign argValue integer value type
+ else if (argValue can be cast as Double)
+ Assign argValue double value type
+ else if (argValue can be cast as Boolean)
+ Assign argValue boolean value type
+ else
+ Assign argValue string value type
+
+In above example, the placeholder variable ```\$nbrCols``` will be treated as integer in the script. If however, the command line arguments were “```nbrCols=5.0```”, then it would be treated as a double.
+
+NOTE: argName must be a valid identifier.
+NOTE: If argValue contains spaces, it must be enclosed in double-quotes.
+NOTE: The values passed from the command-line are passed as literal values which replace the placeholders in the DML script, and are not interpreted as DML.
+
+
+## Built-In Functions
+
+Built-in functions are categorized in:
+
+ * Matrix Construction, Manipulation, and Aggregation Built-In Functions
+ * Matrix and/or Scalar Comparison Built-In Functions
+ * Casting Built-In Functions
+ * Statistical Built-In Functions
+ * Mathematical and Trigonometric Built-In Functions
+ * Linear Algebra Built-In Functions
+ * Other Built-In Functions
+
+The tables below list the supported built-in functions.
+For example, consider the following expressions:
+
+ s = sum(A);
+ B = rowSums(A);
+ C = colSums(A);
+ D = rowSums(C);
+ diff = s – as.scalar(D);
+
+The builtin function ```sum``` operates on a matrix (say A of dimensionality (m x n)) and returns a scalar value corresponding to the sum of all values in the matrix. The built-in functions ```rowSums``` and ```colSums```, on the other hand, aggregate values on a per-row and per-column basis respectively. They output matrices of dimensionality (m x 1) and 1xn, respectively. Therefore, B is a m x 1 matrix and C is a 1 x n matrix. Applying ```rowSums``` on matrix C, we obtain matrix D as a 1 x 1 matrix. A 1 x 1 matrix is different from a scalar; to treat D as a scalar, an explicit ```as.scalar``` operation is invoked in the final statement. The difference between s and ```as.scalar(D)``` should be 0.
+
+### Matrix Construction, Manipulation, and Aggregation Built-In Functions
+
+Table 3. Matrix Construction, Manipulation, and Aggregation Built-In Functions
+
+Function | Description | Parameters | Example
+-------- | ----------- | ---------- | -------
+append() | Adds the second argument as additional columns to the first argument (note that the first argument is not over-written). Append is meant to be used in situations where one cannot use left-indexing. | Input : (X <matrix>, Y <matrix>) <br/>Output : <matrix> <br/> X and Y are matrices (with possibly multiple columns), where the number of rows in X and Y must be the same. Output is a matrix with exactly the same number of rows as X and Y. Let n1 and n2 denote the number of columns of matrix X and Y, respectively. The returned matrix has n1+n2 columns, where the first n1 columns contain X and the last n2 columns contain Y. | A = matrix(1, rows=2,cols=5) <br/> B = matrix(1, rows=2,cols=3) <br/> C = append(A,B) <br/> print( "Dimensions of C:" + nrow(C) + " X " + ncol(C)) <br/> The output of above example is: <br/> Dimensions of C:2 X 8
+matrix() | Matrix constructor (assigning all the cells to numeric literals). | Input: (<init>, rows=<value>, cols=<value>) <br/> init: numeric literal; <br/> rows/cols: number of rows/cols (expression) <br/> Output: matrix | # 10x10 matrix initialized to 0 <br/> A = matrix (0, rows=10, cols=10)
+ | Matrix constructor (reshaping an existing matrix). | Input: (<existing matrix>, rows=<value>, cols=<value>, byrow=TRUE) <br/> Output: matrix | A = matrix (0, rows=10, cols=10) <br/> B = matrix (A, rows=100, cols=1)
+ | Matrix constructor (initializing using string). | Input: (<initialization string>, rows=<value>, cols=<value>) <br/> Output: matrix | A = matrix("4 3 2 5 7 8", rows=3, cols=2) <br/> Creates a matrix: [ [4, 3], [2, 5], [7, 8] ]
+min() <br/> max() | Return the minimum/maximum cell value in matrix | Input: matrix <br/> Output: scalar | min(X) <br/> max(Y)
+min() <br/> max() | Return the minimum/maximum cell values of two matrices, matrix and scalar, or scalar value of two scalars. | Input: matrices or scalars <br/> Output: matrix or scalar | With x,y, z as scalars, and X, Y, Z as matrices: <br/> Z = min (X, Y) <br/> Z = min (X, y) <br/> z = min(x,y)
+nrow(), <br/> ncol(), <br/> length() | Return the number of rows, number of columns, or number of cells in matrix respectively. | Input: matrix <br/> Output: scalar | nrow(X)
+prod() | Return the product of all cells in matrix | Input: matrix <br/> Output: scalarj | prod(X)
+rand() | Generates a random matrix | Input: (rows=<value>, cols=<value>, min=<value>, max=<value>, sparsity=<value>, pdf=<string>) <br/> rows/cols: Number of rows/cols (expression) <br/> min/max: Min/max value for cells (either constant value, or variable that evaluates to constant value) <br/> sparsity: fraction of non-zero cells (constant value) <br/> pdf: “uniform” (min, max) distribution, or “normal” (0,1) distribution; or “poisson” (lambda=1) distribution. string; default value is “uniform”. Note that, for the Poisson distribution, users can provide the mean/lambda parameter as follows: <br/> rand(rows=1000,cols=1000, pdf=”poisson”, lambda=2.5). <br/> The default value for lambda is 1. <br/> Output: matrix | X = rand(rows=10, cols=20, min=0, max=1, pdf="uniform", sparsity=0.2) <br/> The example generates a 10 x 20 matrix, with cell values uniformly chosen at random between 0 and 1, and approximately 20% of cells will have
non-zero values.
+removeEmpty() | Removes all empty rows or columns from the input matrix target X according to the specified margin. | Input : (target= X <matrix>, margin="...") <br/> Output : <matrix> <br/> Valid values for margin are "rows" or "cols". | A = removeEmpty(target=X, margin="rows")
+replace() | Creates a copy of input matrix X, where all values that are equal to the scalar pattern s1 are replaced with the scalar replacement s2. | Input : (target= X <matrix>, pattern=<scalar>, replacement=<scalar>) <br/> Output : <matrix> <br/> If s1 is NaN, then all NaN values of X are treated as equal and hence replaced with s2. Positive and negative infinity are treated as different values. | A = replace(target=X, pattern=s1, replacement=s2)
+seq() | Creates a single column vector with values starting from <from>, to <to>, in increments of <increment> | Input: (<from>, <to>, <increment>) <br/> Output: <matrix> | S = seq (10, 200, 10)
+sum() | Sum of all cells in matrix | Input: matrix <br/> Output: scalar | sum(X)
+
+### Matrix and/or Scalar Comparison Built-In Functions
+
+Table 4. Matrix and/or Scalar Comparison Built-In Functions
+
+Function | Description | Parameters | Example
+-------- | ----------- | ---------- | -------
+pmin() <br/> pmax() | "parallel min/max".<br/> Return cell-wise minimum/maximum. If the second input is a scalar then it is compared against all cells in the first input. | Input: (<matrix>, <matrix>), or (<matrix>, <scalar>) <br/> Output: matrix | pmin(X,Y) <br/> pmax(X,y)
+rowIndexMax() | Row-wise computation -- for each row, find the max value, and return its column index. | Input: (matrix) <br/> Output: (n x 1) matrix | rowIndexMax(X)
+rowIndexMin() | Row-wise computation -- for each row, find the minimum value, and return its column index. | Input: (matrix) <br/> Output: (n x 1) matrix | rowIndexMin(X)
+ppred() | "parallel predicate".<br/> The relational operator specified in the third argument is cell-wise applied to input matrices. If the second argument is a scalar, then it is used against all cells in the first argument. | Input: (<matrix>, <matrix>, <string with relational operator>), or <br/> (<matrix>, <scalar>, <string with relational operator>) <br/> Output: matrix | ppred(X,Y,"<") <br/> ppred(X,y,"<")
+
+### Casting Built-In Functions
+
+Table 5. Casting Built-In Functions
+
+Function | Description | Parameters | Example
+-------- | ----------- | ---------- | -------
+as.scalar(), <br/> as.matrix() | A 1x1 matrix is cast as scalar (value type preserving), and a scalar is cast as 1x1 matrix with value type double | Input: (<matrix>), or (<scalar>) <br/> Output: <scalar>, or <matrix> | as.scalar(X) <br/> as.matrix(x)
+as.double(), <br/> as.integer(), <br/> as.logical() | A variable is cast as the respective value type, data type preserving. as.integer() performs a safe cast. For numerical inputs, as.logical() returns false if the input value is 0 or 0.0, and true otherwise. | Input: (<scalar>) <br/> Output: <scalar> | as.double(X) <br/> as.integer(x) <br/> as.logical(y)
+
+### Statistical Built-In Functions
+
+Table 6. Statistical Built-In Functions
+
+Function | Description | Parameters | Example
+-------- | ----------- | ---------- | -------
+mean() <br/> avg() | Return the mean value of all cells in matrix | Input: matrix <br/> Output: scalar | mean(X)
+moment() | Returns the kth central moment of values in a column matrix V, where k = 2, 3, or 4. It can be used to compute statistical measures like Variance, Kurtosis, and Skewness. This function also takes an optional weights parameter W. | Input: (X <(n x 1) matrix>, [W <(n x 1) matrix>),] k <scalar>) <br/> Output: <scalar> | A = rand(rows=100000,cols=1, pdf="normal") <br/> print("Variance from our (standard normal) random generator is approximately " + moment(A,2))
+colSums() <br/> colMeans() <br/> colMaxs() <br/> colMins() | Column-wise computations -- for each column, compute the sum/mean/max/min of cell values | Input: matrix <br/> Output: (1 x n) matrix | colSums(X) <br/> colMeans(X) <br/> colMaxs(X) <br/>colMins(X)
+cov() | Returns the covariance between two 1-dimensional column matrices X and Y. The function takes an optional weights parameter W. All column matrices X, Y, and W (when specified) must have the exact same dimension. | Input: (X <(n x 1) matrix>, Y <(n x 1) matrix> [, W <(n x 1) matrix>)]) <br/> Output: <scalar> | cov(X,Y) <br/> cov(X,Y,W)
+table() | Returns the contingency table of two vectors A and B. The resulting table F consists of max(A) rows and max(B) columns. <br/> More precisely, F[i,j] = \\|{ k \\| A[k] = i and B[k] = j, 1 ≤ k ≤ n }\\|, where A and B are two n-dimensional vectors. <br/> This function supports multiple other variants, which can be found below, at the end of this Table 6. | Input: (<(n x 1) matrix>, <(n x 1) matrix>), [<(n x 1) matrix>]) <br/> Output: <matrix> | F = table(A, B) <br/> F = table(A, B, C) <br/> And, several other forms (see below Table 6.)
+cdf()<br/> pnorm()<br/> pexp()<br/> pchisq()<br/> pf()<br/> pt()<br/> icdf()<br/> qnorm()<br/> qexp()<br/> qchisq()<br/> qf()<br/> qt() | p=cdf(target=q, ...) returns the cumulative probability P[X <= q]. <br/> q=icdf(target=p, ...) returns the inverse cumulative probability i.e., it returns q such that the given target p = P[X<=q]. <br/> For more details, please see the section “Probability Distribution Functions” below Table 6. | Input: (target=<scalar>, dist="...", ...) <br/> Output: <scalar> | p = cdf(target=q, dist="normal", mean=1.5, sd=2); is same as p=pnorm(target=q, mean=1.5, sd=2); <br/> q=icdf(target=p, dist=”normal”) is same as q=qnorm(target=p, mean=0,sd=1) <br/> More examples can be found in the section “Probability Distribution Functions” below Table 6.
+aggregate() | Splits/groups the values from X according to the corresponding values from G, and then applies the function fn on each group. <br/> The result F is a column matrix, in which each row contains the value computed from a distinct group in G. More specifically, F[k,1] = fn( {X[i,1] \\| 1<=i<=n and G[i,1] = k} ), where n = nrow(X) = nrow(G). <br/> Note that the distinct values in G are used as row indexes in the result matrix F. Therefore, nrow(F) = max(G). It is thus recommended that the values in G are consecutive and start from 1. <br/> This function supports multiple other variants, which can be found below, at the end of this Table 6. | Input: (target= X <(n x 1) matrix>, groups = G <(n x 1) matrix>, fn="..." [,weights= W<(n x 1) matrix>]) <br/> Output: F <matrix> <br/> The parameter fn takes one of the following functions: "count", "sum", "mean", "variance", "centralmoment”. In the case of central moment, one must also provide the ord
er of the moment that need to be computed (see example). | F = aggregate(target=X, groups=G, fn="..." [,weights = W]) <br/> F = aggregate(target=X, groups=G1, fn="sum"); <br/> F = aggregate(target=Y, groups=G2, fn="mean", weights=W); <br/> F = aggregate(target=Z, groups=G3, fn="centralmoment", order="2"); <br/> And, several other forms (see below Table 6.)
+interQuartileMean() | Returns the mean of all x in X such that x>quantile(X, 0.25) and x<=quantile(X, 0.75). X, W are column matrices (vectors) of the same size. W contains the weights for data in X. | Input: (X <(n x 1) matrix> [, W <(n x 1) matrix>)]) <br/> Output: <scalar> | interQuartileMean(X) <br/> interQuartileMean(X, W)
+quantile () | The p-quantile for a random variable X is the value x such that Pr[X<x] <= p and Pr[X<= x] >= p <br/> let n=nrow(X), i=ceiling(p*n), quantile() will return X[i]. p is a scalar (0<p<1) that specifies the quantile to be computed. Optionally, a weight vector may be provided for X. | Input: (X <(n x 1) matrix>, [W <(n x 1) matrix>),] p <scalar>) <br/> Output: <scalar> | quantile(X, p) <br/> quantile(X, W, p)
+quantile () | Returns a column matrix with list of all quantiles requested in P. | Input: (X <(n x 1) matrix>, [W <(n x 1) matrix>),] P <(q x 1) matrix>) <br/> Output: matrix | quantile(X, P) <br/> quantile(X, W, P)
+median() | Computes the median in a given column matrix of values | Input: (X <(n x 1) matrix>, [W <(n x 1) matrix>),]) <br/> Output: <scalar> | median(X) <br/> median(X,W)
+rowSums() <br/> rowMeans() <br/> rowMaxs() <br/> rowMins() | Row-wise computations -- for each row, compute the sum/mean/max/min of cell value | Input: matrix <br/> Output: (n x 1) matrix | rowSums(X) <br/>rowMeans(X) <br/> rowMaxs(X) <br/> rowMins(X)
+cumsum() | Column prefix-sum (For row-prefix sum, use cumsum(t(X)) | Input: matrix <br/> Output: matrix of the same dimensions | A = matrix("1 2 3 4 5 6", rows=3, cols=2) <br/> B = cumsum(A) <br/> The output matrix B = [[1, 2], [4, 6], [9, 12]]
+cumprod() | Column prefix-prod (For row-prefix prod, use cumprod(t(X)) | Input: matrix <br/> Output: matrix of the same dimensions | A = matrix("1 2 3 4 5 6", rows=3, cols=2) <br/> B = cumprod(A) <br/> The output matrix B = [[1, 2], [3, 8], [15, 48]]
+cummin() | Column prefix-min (For row-prefix min, use cummin(t(X)) | Input: matrix <br/> Output: matrix of the same dimensions | A = matrix("3 4 1 6 5 2", rows=3, cols=2) <br/> B = cummin(A) <br/> The output matrix B = [[3, 4], [1, 4], [1, 2]]
+cummax() | Column prefix-max (For row-prefix min, use cummax(t(X)) | Input: matrix <br/> Output: matrix of the same dimensions | A = matrix("3 4 1 6 5 2", rows=3, cols=2) <br/> B = cummax(A) <br/> The output matrix B = [[3, 4], [3, 6], [5, 6]]
+sample(range, size, replacement, seed) | Sample returns a column vector of length size, containing uniform random numbers from [1, range] | Input: <br/> range: integer <br/> size: integer <br/> replacement: boolean (Optional, default: FALSE) <br/> seed: integer (Optional) <br/> Output: Matrix dimensions are size x 1 | sample(100, 5) <br/> sample(100, 5, TRUE) <br/> sample(100, 120, TRUE) <br/> sample(100, 5, 1234) # 1234 is the seed <br/> sample(100, 5, TRUE, 1234)
+outer(vector1, vector2, “op”) | Applies element wise binary operation “op” (for example: “<”, “==”, “>=”, “*”, “min”) on the all combination of vector. <br/> Note: Using “*”, we get outer product of two vectors. | Input: vectors of same size d, string <br/> Output: matrix of size d X d | A = matrix("1 4", rows = 2, cols = 1) <br/> B = matrix("3 6", rows = 1, cols = 2) <br/> C = outer(A, B, "<") <br/> D = outer(A, B, "*") <br/> The output matrix C = [[1, 1], [0, 1]] <br/> The output matrix D = [[3, 6], [12, 24]]<br/>
+
+#### Alternative forms of table()
+
+The built-in function table() supports different types of input parameters. These variations are described below:
+
+ * Basic form: ```F=table(A,B)```
+ As described above in Table 6.
+ * Weighted form: ```F=table(A,B,W)```
+ Users can provide an optional third parameter C with the same dimensions as of A and B. In this case, the output F[i,j] = ∑kC[k], where A[k] = i and B[k] = j (1 ≤ k ≤ n).
+ * Scalar form
+ In basic and weighted forms, both B and W are one dimensional matrices with same number of rows/columns as in A. Instead, one can also pass-in scalar values in the place of B and W. For example, F=table(A,1) is same as the basic form where B is a matrix with all 1’s. Similarly, ```F=table(A,B,3)``` is identical to the following two DML statements. <br/>
+ ```m3 = matrix(3,rows=nrow(A),cols=1); ``` <br/>
+ ```F = table(A,B,m3);```
+ * Specified Output Size
+ In the above forms, the dimensions of the matrix produced this function is known only after its execution is complete. Users can precisely control the size of the output matrix via two additional arguments, odim1 and odim2, as shown below: <br/>
+ ```F = table(A,B,odim1,odim2);``` <br/>
+ The output F will have exactly ```odim1``` rows and ```odim2``` columns. F may be a truncated or padded (with zeros) version of the output produced by ```table(A,B)``` -- depending on the values of ```max(A)``` and ```max(B)```. For example, if ```max(A) < odim1``` then the last (```odim1-max(A)```) rows will have zeros.
+
+#### Alternative forms of aggregate()
+
+The built-in function aggregate() supports different types of input parameters. These variations are described below:
+
+ * Basic form: ```F=aggregate(target=X, groups=G, fn=”sum”)```
+ As described above in Table 6.
+ * Weighted form: ```F=aggregate(target=X, groups=G, weights=W, fn=”sum”) ```
+ Users can provide an optional parameter W with the same dimensions as of A and B. In this case, fn computes the weighted statistics over values from X, which are grouped by values from G.
+ * Specified Output Size
+As noted in Table 6, the number of rows in the output matrix F is equal to the maximum value in the grouping matrix G. Therefore, the dimensions of F are known only after its execution is complete. When needed, users can precisely control the size of the output matrix via an additional argument, ```ngroups```, as shown below: <br/>
+ ```F = aggregate(target=X, groups=G, fn=”sum”, ngroups=10);``` <br/>
+The output F will have exactly 10 rows and 1 column. F may be a truncated or padded (with zeros) version of the output produced by ```aggregate(target=X, groups=G, fn=”sum”)``` – depending on the values of ```ngroups``` and ```max(G)```. For example, if ```max(G) < ngroups``` then the last (```ngroups-max(G)```) rows will have zeros.
+
+#### Probability Distribution Functions
+
+##### ```p = cdf(target=q, dist=fn, ..., lower.tail=TRUE)```
+
+This computes the cumulative probability at the given quantile i.e., P[X<=q], where X is random variable whose distribution is specified via string argument fn.
+
+ * ```target```: input quantile at which cumulative probability P[X<=q] is computed, where X is random variable whose distribution is specified via string argument fn. This is a mandatory argument.
+ * ```dist```: name of the distribution specified as a string. Valid values are “normal” (for Normal or Gaussian distribution), “f” (for F distribution), “t” (for Student t-distribution), “chisq” (for Chi Squared distribution), and “exp” (for Exponential distribution). This is a mandatory argument.
+ * ```...```: parameters of the distribution
+ * For ```dist=”normal”```, valid parameters are mean and sd that specify the mean and standard deviation of the normal distribution. The default values for mean and sd are 0.0 and 1.0, respectively.
+ * For ```dist=”f”```, valid parameters are df1 and df2 that specify two degrees of freedom. Both these parameters are mandatory.
+ * For ```dist=”t”```, and dist=”chisq”, valid parameter is df that specifies the degrees of freedom. This parameter is mandatory.
+ * For ```dist=”exp”```, valid parameter is rate that specifies the rate at which events occur. Note that the mean of exponential distribution is 1.0/rate. The default value is 1.0.
+ * ```Lower.tail```: a Boolean value with default set to TRUE. cdf() computes P[X<=q] when lower.tail=TRUE and it computes P[X>q] when lower.tail=FALSE. In other words, a complement of the cumulative distribution is computed when lower.tail=FALSE.
+
+##### ```q = icdf(target=p, dist=fn, ...)```
+
+This computes the inverse cumulative probability i.e., it computes a quantile q such that the given probability p = P[X<=q], where X is random variable whose distribution is specified via string argument fn.
+
+ * ```target```: a mandatory argument that specifies the input probability.
+ * ```dist```: name of the distribution specified as a string. Same as that in cdf().
+ * ```...```: parameters of the distribution. Same as those in cdf().
+
+Alternative to ```cdf()``` and ```icdf()```, users can also use distribution-specific functions. The functions ```pnorm()```, ```pf()```, ```pt()```, ```pchisq()```, and ```pexp()``` computes the cumulative probabilities for Normal, F, t, Chi Squared, and Exponential distributions, respectively. Appropriate distribution parameters must be provided for each function. Similarly, ```qnorm()```, ```qf()```, ```qt()```, ```qchisq()```, and ```qexp()``` compute the inverse cumulative probabilities for Normal, F, t, Chi Squared, and Exponential distributions.
+
+Following pairs of DML statements are equivalent.
+
+```p = cdf(target=q, dist="normal", mean=1.5, sd=2);```
+is same as
+```p=pnorm(target=q, mean=1.5, sd=2);```
+
+```p = cdf(target=q, dist="exp", rate=5);```
+is same as
+```pexp(target=q,rate=5);```
+
+```p = cdf(target=q, dist="chisq", df=100);```
+is same as
+```pchisq(target=q, df=100)```
+
+```p = cdf(target=q, dist="f", df1=100, df2=200);```
+is same as
+```pf(target=q, df1=100, df2=200);```
+
+```p = cdf(target=q, dist="t", df=100);```
+is same as
+```pt(target=q, df=100)```
+
+```p = cdf(target=q, dist="normal", lower.tail=FALSE);```
+is same as
+```p=pnorm(target=q, lower.tail=FALSE);```
+is same as
+```p=pnorm(target=q, mean=0, sd=1.0, lower.tail=FALSE);```
+is same as
+```p=pnorm(target=q, sd=1.0, lower.tail=FALSE);```
+
+Examples of icdf():
+
+```q=icdf(target=p, dist=”normal”);```
+is same as
+```q=qnorm(target=p, mean=0,sd=1);```
+
+```q=icdf(target=p, dist=”exp”);```
+is same as
+```q=qexp(target=p, rate=1);```
+
+```q=icdf(target=p, dist=”chisq”, df=50);```
+is same as
+```qchisq(target=p, df=50);```
+
+```q=icdf(target=p, dist=”f”, df1=50, df2=25);```
+is same as
+```qf(target=p, , df1=50, df2=25);```
+
+```q=icdf(target=p, dist=”t”, df=50);```
+is same as
+```qt(target=p, df=50);```
+
+### Mathematical and Trigonometric Built-In Functions
+
+Table 7. Mathematical and Trigonometric Built-In Functions
+
+Function | Description | Parameters | Example
+-------- | ----------- | ---------- | -------
+exp(), log(), abs(), sqrt(), round(), floor(), ceil() | Apply mathematical function on input (cell wise if input is matrix) | Input: (<matrix>), or (<scalar>) <br/> Output: <matrix>, or <scalar> | sqrt(X) <br/> log(X,y) <br/> round(X) <br/> floor(X) <br/> ceil(X)
+sin(), cos(), tan(), asin(), acos(), atan() | Apply trigonometric function on input (cell wise if input is matrix) | Input: (<matrix>), or (<scalar>) <br/> Output: <matrix>, or <scalar> | sin(X)
+
+### Linear Algebra Built-In Functions
+
+Table 8. Linear Algebra Built-In Functions
+
+Function | Description | Parameters | Example
+-------- | ----------- | ---------- | -------
+diag() | Create diagonal matrix from (n x 1) or (1 x n) matrix, or take diagonal from square matrix | Input: (n x 1) or (1 x n) matrix, or (n x n) matrix <br/> Output: (n x n) matrix, or (n x 1) matrix | diag(X)
+eigen() | Computes Eigen decomposition of input matrix A. The Eigen decomposition consists of two matrices V and w such that A = V %\*% diag(w) %\*% t(V). The columns of V are the eigenvectors of the original matrix A. And, the eigen values are given by w. <br/> It is important to note that this function can operate only on small-to-medium sized input matrix that can fit in the main memory. For larger matrices, an out-of-memory exception is raised. | Input : (A <matrix>) <br/> Output : [w <(m x 1) matrix>, V <matrix>] <br/> A is a square symmetric matrix with dimensions (m x m). This function returns two matrices w and V, where w is (m x 1) and V is of size (m x m). | [w, V] = eigen(A)
+lu() | Computes Pivoted LU decomposition of input matrix A. The LU decomposition consists of three matrices P, L, and U such that P %\*% A = L %\*% U, where P is a permutation matrix that is used to rearrange the rows in A before the decomposition can be computed. L is a lower-triangular matrix whereas U is an upper-triangular matrix. <br/> It is important to note that this function can operate only on small-to-medium sized input matrix that can fit in the main memory. For larger matrices, an out-of-memory exception is raised. | Input : (A <matrix>) <br/> Output : [<matrix>, <matrix>, <matrix>] <br/> A is a square matrix with dimensions m x m. This function returns three matrices P, L, and U, all of which are of size m x m. | [P, L, U] = lu(A)
+qr() | Computes QR decomposition of input matrix A using Householder reflectors. The QR decomposition of A consists of two matrices Q and R such that A = Q%\*%R where Q is an orthogonal matrix (i.e., Q%\*%t(Q) = t(Q)%\*%Q = I, identity matrix) and R is an upper triangular matrix. For efficiency purposes, this function returns the matrix of Householder reflector vectors H instead of Q (which is a large m x m potentially dense matrix). The Q matrix can be explicitly computed from H, if needed. In most applications of QR, one is interested in calculating Q %\*% B or t(Q) %\*% B – and, both can be computed directly using H instead of explicitly constructing the large Q matrix. <br/> It is important to note that this function can operate only on small-to-medium sized input matrix that can fit in the main memory. For larger matrices, an out-of-memory exception is raised. | Input : (A <matrix>) <br/> Output : [<matrix>, <matrix>] <br/> A is a (m x n) matrix, which can e
ither be a square matrix (m=n) or a rectangular matrix (m != n). This function returns two matrices H and R of size (m x n) i.e., same size as of the input matrix A. | [H, R] = qr(A)
+solve() | Computes the least squares solution for system of linear equations A %\*% x = b i.e., it finds x such that \|\|A%*%x – b\|\| is minimized. The solution vector x is computed using a QR decomposition of A. <br/> It is important to note that this function can operate only on small-to-medium sized input matrix that can fit in the main memory. For larger matrices, an out-of-memory exception is raised. | Input : (A <(m x n) matrix>, b <(m x 1) matrix>) <br/> Output : <matrix> <br/> A is a matrix of size (m x n) and b is a 1D matrix of size m x 1. This function returns a 1D matrix x of size n x 1. | x = solve(A,b)
+t() | Transpose matrix | Input: matrix <br/> Output: matrix | t(X)
+trace() | Return the sum of the cells of the main diagonal square matrix | Input: matrix <br/> Output: scalar | trace(X)
+
+### Read/Write Built-In Functions
+
+Read/Write supports reading/writing of all data types (matrices, scalars) from/to HDFS files. Associated with each data file is a optional metadata file (MTD) which stores metadata information about the content of the data file. For data files generated by SystemML, a MTD file will automatically be generated. The name of the MTD file associated with the data file <filename> must be <filename>.mtd. A user can provide an MTD file for their own data as well.
+
+#### File formats and MTD files
+
+SystemML supports 3 external file formats:
+
+ * (i,j,v)-format
+ * MatrixMarket format
+ * Delimited or CSV format
+
+The (i,j,v)-format is a sparse format in which the cell values of a matrix are serialized in space separated triplets of rowId, columnId, and cellValue with rowId and columnId indices being 1-based. The MatrixMarket format (see open source R documentation) only supports headers of "%%MatrixMarket matrix coordinate real general". The (i,j,v) formats can be stored in text form, or binary. The binary format can only be read and written by SystemML.
+
+As an example, see the content of a matrix file X.mtx in text format with 4 non-zero cells. The matrix has 10 rows and 8 columns.
+
+ 1 1 4.0
+ 3 4 10.0
+ 5 6 7.0
+ 9 8 20.0
+
+The content of the MTD file X.mtx.mtd associated with data X.mtx is:
+
+ { "data_type": "matrix",
+ "value_type": "double",
+ "rows": 10,
+ "cols": 8,
+ "nnz": 4,
+ "format": "text",
+ "description": { "author": "SystemML" } }
+
+The content of the MTD file scalar_i.mtd associated with a scalar data file scalar_i (which contains the integer scalar value 2)
+
+ { "data_type": "scalar",
+ "value_type": "double",
+ "format": "text",
+ "description": { "author": "SystemML" } }
+
+The MTD file contains a single JSON object with the attributes described below.
+
+Table 9. MTD attributes
+
+Parameter Name | Description | Optional | Permissible values | Data type valid for
+-------------- | ----------- | -------- | ------------------ | -------------------
+data_type | Indicates the data type of the data | Yes. Default value is matrix if not specified. | matrix, scalar. | matrix, scalar.
+value_type | Indicates the value type of data | Yes. Default value is double if not specified | double, int, string, boolean (Not case-sensitive). Must be double when data_type matrix | matrix, scalar.
+rows | Number of rows in matrix | Yes – only when format=”csv”. | Any integer > 0 | matrix
+cols | Number of columns in matrix | Yes – only when format=”csv” | Any integer > 0 | matrix
+rows_in_block, cols_in_block | Valid only for binary blocked matrix. Indicate dimensions of blocks | No. Only valid if matrix in binary blocked format. | Any integer > 0 | matrix in blocked binary format. Valid only when format "binary"
+nnz | Number of non-zero values | Yes | Any integer > 0 | matrix
+format | data file format | Yes. Default value is text | binary, text, mm, csv | matrix, scalar. Formats mm and csv are applicable only for matrices.
+Description | description of the data. | Yes | Any valid JSON string | matrix, scalar
+
+In addition, when reading/writing delimited or CSV files, the metadata file (mtd file) may contain one or more of the following four attributes.
+
+Table 10. Additional MTD attributes when reading/writing delimited or CSV files
+
+Parameter Name | Description | Optional | Permissible values | Data type valid for
+-------------- | ----------- | -------- | ------------------ | -------------------
+header | Specifies whether the input data file has a header. Header, if exists, must be a single line and it must also be the first line in the file. | Yes, default value is False. | true or false | Matrix
+sep | Specifies the separator or the delimiter used in the input data file. Note that using a delimiter composed of just numeric values or a dot (decimal point) can be ambiguous and may lead to unexpected results. | Yes, default value is ‘,’ (comma) | A quoted string | matrix
+fill | Only valid when reading CSV files. It specifies whether or not to fill the empty fields in the input file. Empty fields are denoted by consecutive separators/delimiters. If fill=true then every empty field is filled with the value specified by “default” attribute. An exception is raised if fill=false and and the input file has one or more empty fields. | Yes, default is true. | true or false | matrix
+default | Only valid when reading CSV files and fill=true. It specifies the special value with which all empty values are filled while reading the input matrix. | Yes, default value is 0 (zero) | Any double value | matrix
+sparse | Only valid when writing CSV files. It specifies whether or not to explicitly output zero (0) values. Zero values are written out only when sparse=false. | Yes, default value is true. | true or false | matrix
+
+Furthermore, the following additional notes apply when reading and writing CSV files.
+
+ * Every line in the input file must have the same number of fields or values.
+ * Input file can only contain numeric values separated by the delimitor (as specifed by sep).
+ * While writing CSV files, if header=true is specified then the header line is formed as a concatenated string of column names separated by delimiters. Columns are of the form "C<column_id>". For a matrix with 5 columns, the header line would look like: "C1,C2,C3,C4,C5" (assuming sep=",").
+
+#### Read Built-In Function
+
+The syntax of a read statement is as follows:
+
+ read(“inputfile”, [list of parameters])
+
+where inputfile is the path to the data file in HDFS. The list of parameters is the same as provided in MTD files. For "inputfile" parameter, the user can use constant string concatenation to give the full path of the file, where "+" is used as the concatenation operator. However, the file path must evaluate to a constant string at compile time. For example, “/my/dir”+“filename.mtx” is valid parameter but “/my/dir”+“filename”+i+“.mtx” is not (where i is a variable).
+
+The user has the option of specifying each parameter value either in the MTD file, the read function invocation, or in both locations. **However, parameter values specified in both the read invocation and MTD file must have the same value. Also, if a scalar value is being read, then format cannot be specified.** The read invocation in SystemML is parameterized as follows during compilation.
+
+ 1. Default values are assigned to parameters with default values.
+ 2. Parameters provided in read() either fill in values or override defaults.
+ 3. SystemML will look for the MTD file at compile time in the specified location (same path as data file, filename for MTD file is same name as data file with ".mtd" appended).
+ 4. If all non-optional parameters are specified or conflicting values are detected, then an exception is thrown.
+
+##### Examples
+
+ # read a matrix from HDFS with path "in/v".
+ # Defaults for data_type and value_type are used.
+ V = read ("in/v", rows=10, cols=8, format="text");
+ # read a matrix from HDFS with path "in/v".
+ # The user specifies "in" as the directory and “v” as
+ # the file name and uses constant string concatenation.
+ dir = "in";
+ file = "v";
+ V = read (dir+file, rows=10, cols=8, format="text");
+ # read a matrix data file from HDFS with an MTD file available
+ # (MTD file path: in/data.mtd)
+ V = read("in/data");
+ # read a scalar integer value from HDFS file "in/scalar_i"
+ V = read(“in/scalar_i”,data_type=”scalar", value_type="int");
+
+Additionally, readMM() and read.csv() are supported and can be used instead of specifying format="mm", or format="csv" in the read() function.
+
+#### Write Built-In Function
+
+The write() method is used to persist scalar and matrix data to files in HDFS. The syntax of write() is below. The parameters are described in the table. Note, that the set of supported parameters for write() is NOT the same as for read(). SystemML writes a MTD file for the written data.
+
+ write(identifier, “outputfile”, format = “fmt”)
+
+The user can use constant string concatenation in “outputfile” parameter to give the full path of the file, where "+" is used as the concatenation operator.
+
+Table 11. Parameters for ```write()``` method
+
+Parameter Name | Description | Optional | Permissible Values
+-------------- | ----------- | -------- | ------------------
+identifier | variable whose data is to be written to a file. Can be any data type: Matrix, Scalar. | No | Any non-keyword string
+outputfile | | No | Any valid filename
+format | Valid for all three datatypes. <br/> Indicates the output file format. | Yes | text, binary, mm, csv <br/> Constraints: when datatype is matrix, can be text or binary. When datatype is scalar, can only be text
+
+
+##### Examples
+
+ # write V to HDFS file “out/file”, in text format.
+ # Create MTD file out/file.mtd
+ write(V, "out/file");
+
+out/file.mtd:
+
+ { "data_type": "matrix",
+ "value_type": "double",
+ "rows": 10,
+ "cols": 8,
+ "nnz": 4,
+ "format": "text”,
+ "description": { "author": "SystemML" } }
+
+Write V to HDFS file”out/file” in binary blocked format:
+
+ write(V, “out/file”, format=”binary”);
+
+out/file.mtd:
+
+ { "data_type": "matrix",
+ "value_type": "double",
+ "rows": 10,
+ "cols": 8,
+ "nnz": 4,
+ "rows_in_block": 1000,
+ "cols_in_block": 1000,
+ "format": "binary",
+ "description": { "author": "SystemML" } }
+
+Write a scalar integer value to HDFS file “out/scalar_i”
+
+ write(x, "out/scalar_i");
+
+out/scalar_i.mtd:
+
+ {"data_type": "scalar",
+ "value_type": "double",
+ "format": "text",
+ "description": { "author": "SystemML" } }
+
+Unlike read(), write() function does not need a constant string expression, so following example will work:
+
+ A = rand(rows=10, cols=2)
+ dir = "tmp/"
+ i = 1
+ file = "A" + i + ".mtx"
+ write(A, dir+file, format="csv")
+
+### Data Pre-Processing Built-In Functions
+
+Data pre-processing built-in ```transform()``` is used to transform a given tabular input data set (with data type ```frame```) in CSV format into a ```matrix```. The ```transform()``` function supports the following five column-level data transformations:
+
+ * *Missing Value Imputation*: This replaces missing data in individual columns with valid values, depending on the specific imputation method. There are three supported imputation methods -- ```global_mean``` that replaces a missing value in a *numeric/scale* column with the mean of all non-missing entries in the column; ```global_mode``` that replaces a missing value in a *categorical* column with the mode of all non-missing entries in the column; and ```constant``` that replaces missing values in a *scale/categorical* column with the specified constant.
+ * *Recoding*: This is applicable for *categorical* columns. It maps all distinct categories (potentially, strings and booleans) in the column into consecutive numbers, starting from 1. For example, a ```direction``` column with four distinct values (east, west, north, south) into a column with four numeric values 1.0, 2.0, 3.0, and 4.0.
+ * *Binning*: This procedure is used to group a number of continuous values (i.e., discretize) into a small number of *bins*. For example, a column with ```age``` values can be discretized into a small number of age intervals. The only method that is currently supported is ```equi-width``` binning.
+ * *Dummycoding*: This procedure transforms a categorical column into multiple columns of zeros and ones, which collectively capture the full information about the categorical variable. The number of resulting columns is equal to the number of distinct values in the input column. In the example of ```direction``` variable mentioned above, this procedure replaces the original column with four new columns with zeros and ones – ```direction_east```, ```direction_west```, ```direction_north```, and ```direction_south```.
+ * *Scaling*: This centers and/or scales the values in a given numeric/continuous column. The two supported methods are ```mean-subtraction``` that centers each value by subtracting the mean, and ```z-score``` that scales mean subtracted values by dividing them with the respective column-wise standard deviation.
+
+The transformations are specified to operate on individual columns. The set of all required transformations must be provided via a *specification* file in JSON format. Furthermore, the notation indicating missing values must be specified in the mtd file associated with the input CSV data, along with other properties such as header and delimiter. As an example, consider the following sample of *homes* data set.
+
+```data.csv```
+
+Note that the missing values are denoted either by an empty value (as in the 6<sup>th</sup> row) or as a string “NA”. This information must be captured via ```na.strings``` property in the mtd file associated with the input data. Assuming that this data is stored in CSV format with “,” as the delimiter, the mtd file ```data.csv.mtd``` would look as follows:
+
+```data.csv.mtd```
+
+ {
+ "data_type": "frame",
+ "format": "csv",
+ "delimiter": ","
+ "header": true,
+ "na.strings": ["NA", ""]
+ }
+
+An example of the transformation *specification* file is given below:
+
+ {
+ "impute":
+ [ { "name": "zipcode" , "method": "global_mode" }
+ ,{ "name": "district" , "method": "constant", "value": "south" }
+ ,{ "name": "numbedrooms" , "method": "constant", "value": 2 }
+ ,{ "name": "numbathrooms", "method": "constant", "value": 1 }
+ ,{ "name": "floors" , "method": "constant", "value": 1 }
+ ,{ "name": "view" , "method": "global_mode" }
+ ,{ "name": "askingprice" , "method": "global_mean" }
+ ]
+
+ ,"recode":
+ [ "zipcode", "district", "numbedrooms", "numbathrooms", "floors", "view" ]
+
+ ,"bin":
+ [ { "name": "saleprice" , "method": "equi-width", "numbins": 3 }
+ ,{ "name": "sqft" , "method": "equi-width", "numbins": 4 }
+ ]
+
+ ,"dummycode":
+ [ "district", "numbathrooms", "floors", "view", "saleprice", "sqft" ]
+
+ ,"scale":
+ [ { "name": "sqft", "method": "mean-subtraction" }
+ ,{ "name": "saleprice", "method": "z-score" }
+ ,{ "name": "askingprice", "method": "z-score" }
+ ]
+ }
+
+Evidently, the *specification* file must provide the complete information about *all* required transformations across *all* the columns in the input data. Given such a *specification* file, the input data set ```data.csv``` is transformed using the built-in function ```transform()```:
+
+ output = transform(target = input,
+ transformPath = ”...”,
+ transformSpec = ”...”,
+ applyTransformPath = ”...”)
+
+Following sample DML script shows its usage.
+
+ D = read("data.csv");
+ tfD = transform(target=D,
+ transformSpec="/path/to/transformation/specification/",
+ transformPath="/path/to/transformation/metadata");
+ s = sum(tfD);
+ print(“Sum = “ + s);
+ write(tfD, "/path/to/transformed/data", format="binary");
+
+The ```target``` parameter points to the input tabular data that needs to be transformed, the ```transformSpec``` refers to the transformation specification JSON indicating the list of transformations that must be performed, and finally ```transformPath``` denotes the output path on HDFS at which all the metadata constructed during the transformation process is stored. Examples of such metadata includes, the number distinct values in a categorical column, the list of distinct values and associated *recoded* IDs, the bin definitions (number of bins, bin widths), etc. This metadata can subsequently be utilized to transform new incoming data, for example, the test set in a predictive modeling exercise. The function returns the actual transformed data ```tfD``` in the form of a matrix, containing only numeric values.
+
+Following code snippet shows an example scenario of transforming a training data set, and subsequently the testing data set.
+
+#### Training Phase
+
+ Train = read(“/user/ml/trainset.csv”);
+ trainD = transform(target=Train,
+ transformSpec=”/user/ml/tf.spec.json”,
+ transformPath=”/user/ml/train_tf_metadata”);
+ # Build a predictive model using trainD
+ ...
+
+#### Testing Phase
+
+ Test = read(“/user/ml/testset.csv”);
+ testD = transform(target=Test,
+ transformPath=”/user/ml/test_tf_metadata”,
+ applyTransformPath=”/user/ml/train_tf_metdata”);
+ # Test the model using testD
+ ...
+
+Note that the metadata generated during the training phase (located at ```/user/ml/train_tf_metadata``` on HDFS) is used to apply the list of transformations (that were carried out on training data set) on the test data set. The parameter ```applyTransformPath``` refers to an existing metadata, which was generated by some earlier invocation of ```transform()``` function. Therefore, in any invocation of ```transform()```, only one of ```transformSpec``` or ```applyTransformPath``` can be specified. The transformation metadata is generated when ```transformSpec``` is specified, and it is simply used and applied when ```applyTransformPath``` is specified. On the other hand, the ```transformPath``` always refers to a location on HDFS where the resulting transformation metadata is stored. Since the second invocation of ```transform()``` does not really generate any new metdata data, the given metadata (```/user/ml/train_tf_metdata```) is simply copied to the target location (```/user/ml/
test_tf_metdata```). Even though such a behavior creates redundant copies of transformation metadata, it is preferred as it allows us to associate every data set with the corresponding transformation metadata.
+
+
+### Other Built-In Functions
+
+Table 12. Other Built-In Functions
+
+Function | Description | Parameters | Example
+-------- | ----------- | ---------- | -------
+append() | Append a string to another string separated by "\n" <br/> Limitation: The string may grow up to 1 MByte. | Input: (<string>, <string>) <br/> Output: <string> | s = "iter=" + i <br/> i = i + 1 <br/> s = append(s, "iter=" + i) <br/> write(s, "s.out")
+print() | Prints the value of a scalar variable x. This built-in takes an optional string parameter. | Input: (<scalar>) | print(“hello”) <br/> print(“hello” + “world”) <br/> print("value of x is " + x )
+stop() | Halts the execution of DML program by printing the message that is passed in as the argument. <br/> Note that the use of stop() is not allowed inside a parfor loop. | Input: (<scalar>) | stop(“Inputs to DML program are invalid”) <br/> stop(“Class labels must be either -1 or +1”)
+order() | Sort a column of the matrix X in decreasing/increasing order and return either index (indexreturn=TRUE) or data (indexreturn=FALSE). | Input: (target=X, by=column, decreasing, indexreturn) | order(X, by=1, decreasing=FALSE, indexreturn=FALSE)
+
+
+* * *
+
+## Modules
+
+A module is a collection of UDF declarations. For calling a module, source(...) and setwd(...) are used to read and use a source file.
+
+### Syntax
+
+ setwd(<file-path>);
+ source(<DML-filename>) as <namespace-name>;
+
+It is important to note that:
+
+ 1. setwd(...) and source(...) do not support \$-parameters.
+ 2. Nested namespaces are not supported.
+ 3. Namespace are required for source(...).
+ 4. Only UDFs are imported, not the statements.
+ 5. Path for input/output files is not affected by setwd.
+ 6. setwd is applicable only for local filesystem not HDFS.
+ 7. Spaces are not allowed between namespace and function name during call. For example: ns1::foo(...) is correct way to call the function.
+ 8. Like R, the path of source() is relative to where the calling java program is running.
+
+### Example
+
+Assume the file a.dml contains:
+
+ #source("/home/ml/spark_test/b.dml") as ns1 # will work
+ #source("b.dml") as ns1 # will work
+ #source("./b.dml") as ns1 # will work
+ source("hdfs:/user/ml/nike/b.dml") as ns1
+ f1 = function() {
+ print("From a.dml's function()");
+ }
+ setwd("dir1")
+ source("c.dml") as ns2
+ tmp = ns2::f();
+ tmp1 = ns1::f();
+ tmp = f1();
+
+The file b.dml contains:
+
+ f = function() {
+ print("From b.dml's function()");
+ }
+
+The file c.dml contains:
+
+ f = function() {
+ print("From c.dml's function()");
+ }
+
+The output after running a.dml is as follows:
+
+ From c.dml's function()
+ From b.dml's function()
+ From a.dml's function()
+
+
+## Reserved Keywords
+
+Reserved keywords cannot be used as variable names.
+
+All reserved keywords are case-sensitive.
+
+ as
+ boolean
+ Boolean
+ double
+ Double
+ else
+ externalFunction
+ for
+ function
+ FALSE
+ if
+ ifdef
+ implemented
+ in
+ int
+ integer
+ Int
+ Integer
+ parfor
+ return
+ setwd
+ source
+ string
+ String
+ TRUE
+ while
+
+
+## Invocation of SystemML
+
+To execute a DML script, SystemML is invoked as follows:
+
+ hadoop jar SystemML.jar [-? | -help | -f] <filename> (-config=<config_filename>)? (-args | -nvargs)? <args-list>?
+
+Where
+
+```-f <filename>: will be interpreted as a path to file with DML script. <filename> prefixed with hdfs or gpfs is assumed path in DFS, otherwise <filename> treated as path on local file system --debug: (optional) run in debug mode
+-config=<config_filename>: (optional) use config file located at specified path <config_filename>. <config_filename> prefixed with hdfs or gpfs is assumed path in DFS, otherwise <config_filename> treated as path on local file system (default value for <config_filename> is ./SystemML-config.xml)```
+
+```-args <args-list>: (optional) parameterize DML script with contents of <args-list>, which is ALL args after -args flag. Each argument must be an unnamed-argument, where 1st value after -args will replace \$1 in DML script, 2nd value will replace \$2 in DML script, and so on.```
+
+```-nvargs <args-list>: (optional) parameterize DML script with contents of <args-list>, which is ALL args after -nvargs flag. Each argument must be named-argument of form name=value, where value will replace \$name in DML script.```
+
+```-?, or -help: show this help.```
+
+NOTE: Please refer to section on Command-line Arguments for more details and restrictions on usage of command-line arguments to DML script using ```–args <args-list> and –nvargs <args-list>```.
+
+
+### Examples
+
+Run a script in local file foo.dml:
+
+ hadoop jar SystemML.jar -f foo.dml
+
+An example debug session:
+
+First, you need to call SystemML using –debug flag.
+
+ hadoop jar SystemML.jar -f test.dml –debug
+
+You can see the line numbers in your DML script by “list” (or simply “l”) command:
+
+ (SystemMLdb) l
+ line 1: A = matrix("1 2 3 4 5 6", rows=3, cols=2)
+ line 2:
+ line 3: B = cumsum(A)
+ line 4: #print(B)
+ line 5: print(sum(B))
+
+The next step is usually to set a breakpoint where we need to analyze the state of our variables:
+
+ (SystemMLdb) b 5
+
+Breakpoint added at .defaultNS::main, line 5.
+
+Now, that we have set a breakpoint, we can start running our DML script:
+
+ (SystemMLdb) r
+ Breakpoint reached at .defaultNS::main instID 15: (line 5).
+ (SystemMLdb) p B
+ 1.0000 2.0000
+ 4.0000 6.0000
+ 9.0000 12.0000
+
+
+## MLContext API
+----
+
+The MLContext API allows users to pass RDDs as input/output to SystemML through Java, Scala, or Python.
+
+Typical usage for MLContext using Spark's Scala Shell is as follows:
+
+ scala> import com.ibm.bi.dml.api.MLContext
+
+Create input DataFrame from CSV file and potentially perform some feature transformation
+
+ scala> val W = sqlContext.load(...)
+ scala> val H = sc.textFile("V.csv")
+ scala> val V = sc.textFile("V.text")
+
+Create MLContext
+
+ scala> val ml = new MLContext(sc)
+
+Register input and output DataFrame/RDD
+
+Supported formats are:
+
+ 1. DataFrame
+ 2. CSV/Text (as JavaRDD<String> or JavaPairRDD<LongWritable, Text>)
+ 3. Binary blocked RDD (JavaPairRDD<MatrixIndexes,MatrixBlock>))
+
+Also overloaded to support metadata information such as format, rlen,
+clen, etc.
+
+Please note the variable names given below in quotes correspond to the
+variables in DML script.
+
+These variables need to have corresponding read/write associated in DML
+script.
+
+Currently, only matrix variables are supported through
+registerInput/registerOutput interface.
+
+To pass scalar variables, use named/positional arguments (described
+later) or wrap them into matrix variable.
+
+ scala> ml.registerInput("V", V)
+ scala> ml.registerInput("W", W, "csv")
+ scala> ml.registerInput("H", H, "text", 50, 1500)
+ scala> ml.registerOutput("H")
+ scala> ml.registerOutput("W")
+
+As DataFrame is internally converted to CSV format, one can skip
+providing dimensions.
+
+Call script with default arguments:
+
+ scala> val outputs = ml.execute("GNMF.dml")
+
+MLContext also supports calling script with positional arguments (args)
+and named arguments (nargs):
+
+ scala> val args = Array("V.mtx", "W.mtx", "H.mtx", "2000", "1500", "50", "1", "WOut.mtx", "HOut.mtx")
+ scala> val nargs = Map("maxIter"->"1")
+ scala> val outputs = ml.execute("GNMF.dml", args) # or ml.execute("GNMF.dml", nargs)
+
+We can then fetch the output RDDs in SystemML’s binary blocked format or
+as DataFrame.
+
+ scala> val HOut = outputs.getDF(sqlContext, "H")
+ scala> val WOut = outputs. getBinaryBlockedRDD(sqlContext, "W")
+
+To register new input/outputs and to re-execute the script, it is
+recommended that you first reset MLContext
+
+ scala> ml.reset()
+ scala> ml.registerInput("V", newV)
+
+Though it is possible to re-run the script using different (or even same
+arguments), but using same registered input/outputs without reset, it is
+discouraged. This is because the symbol table entries would have been
+updated since last invocation:
+
+ scala> val new_outputs = ml.execute("GNMF.dml", new_args)
+
+The Python MLContext API is similar to Scala/Java MLContext API. Here is
+an example:
+
+ >>> from pyspark.sql import SQLContext
+ >>> from SystemML import MLContext
+ >>> sqlContext = SQLContext(sc)
+ >>> H = sqlContext.jsonFile("H.json")
+ >>> V = sqlContext.jsonFile("V.json")
+ >>> W = sqlContext.jsonFile("W.json")
+ >>> ml = MLContext(sc)
+ >>> ml.registerInput("V", V)
+ >>> ml.registerInput("W", W)
+ >>> ml.registerInput("H", H)
+ >>> ml.registerOutput("H")
+ >>> ml.registerOutput("W")
+ >>> outputs = ml.execute("GNMF.dml")
+
+Note:
+
+- The current version does not allow users to create multiple
+ MLContexts and only allows one thread to execute DML script using
+ the created MLContext.
+- Even though the above example shows the usage through Scala/Python
+ Shell, it works for Spark-Submit and PySpark-Submit as well.
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/f0a60f05/index.md
----------------------------------------------------------------------
diff --git a/index.md b/index.md
index 1a8690c..4dbaf86 100644
--- a/index.md
+++ b/index.md
@@ -18,5 +18,6 @@ For more information about SystemML, please consult the following references:
* [SystemML GitHub README](http://www.github.com/SparkTC/systemml)
* Quick Start - **Coming Soon**
* [Algorithms Reference](algorithms-reference.html)
-* R-Like Declarative Machine Learning (DML) Language Reference - **Coming Soon**
-* Python-Like Declarative Machine Learning (DML) Language Reference - **Coming Soon**
+* [DML (Declarative Machine Learning) Language Reference](dml-language-reference.html)
+* PYDML (Python-Like Declarative Machine Learning) Language Reference - **Coming Soon**
+
[09/47] incubator-systemml git commit: initial port of documentation
to md, algorithms reference converted to md, template based on spark docs
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/js/vendor/jquery-1.8.0.min.js
----------------------------------------------------------------------
diff --git a/js/vendor/jquery-1.8.0.min.js b/js/vendor/jquery-1.8.0.min.js
new file mode 100644
index 0000000..066d72c
--- /dev/null
+++ b/js/vendor/jquery-1.8.0.min.js
@@ -0,0 +1,2 @@
+/*! jQuery v@1.8.0 jquery.com | jquery.org/license */
+(function(a,b){function G(a){var b=F[a]={};return p.each(a.split(s),function(a,c){b[c]=!0}),b}function J(a,c,d){if(d===b&&a.nodeType===1){var e="data-"+c.replace(I,"-$1").toLowerCase();d=a.getAttribute(e);if(typeof d=="string"){try{d=d==="true"?!0:d==="false"?!1:d==="null"?null:+d+""===d?+d:H.test(d)?p.parseJSON(d):d}catch(f){}p.data(a,c,d)}else d=b}return d}function K(a){var b;for(b in a){if(b==="data"&&p.isEmptyObject(a[b]))continue;if(b!=="toJSON")return!1}return!0}function ba(){return!1}function bb(){return!0}function bh(a){return!a||!a.parentNode||a.parentNode.nodeType===11}function bi(a,b){do a=a[b];while(a&&a.nodeType!==1);return a}function bj(a,b,c){b=b||0;if(p.isFunction(b))return p.grep(a,function(a,d){var e=!!b.call(a,d,a);return e===c});if(b.nodeType)return p.grep(a,function(a,d){return a===b===c});if(typeof b=="string"){var d=p.grep(a,function(a){return a.nodeType===1});if(be.test(b))return p.filter(b,d,!c);b=p.filter(b,d)}return p.grep(a,function(a,d){return p.inArray(
a,b)>=0===c})}function bk(a){var b=bl.split("|"),c=a.createDocumentFragment();if(c.createElement)while(b.length)c.createElement(b.pop());return c}function bC(a,b){return a.getElementsByTagName(b)[0]||a.appendChild(a.ownerDocument.createElement(b))}function bD(a,b){if(b.nodeType!==1||!p.hasData(a))return;var c,d,e,f=p._data(a),g=p._data(b,f),h=f.events;if(h){delete g.handle,g.events={};for(c in h)for(d=0,e=h[c].length;d<e;d++)p.event.add(b,c,h[c][d])}g.data&&(g.data=p.extend({},g.data))}function bE(a,b){var c;if(b.nodeType!==1)return;b.clearAttributes&&b.clearAttributes(),b.mergeAttributes&&b.mergeAttributes(a),c=b.nodeName.toLowerCase(),c==="object"?(b.parentNode&&(b.outerHTML=a.outerHTML),p.support.html5Clone&&a.innerHTML&&!p.trim(b.innerHTML)&&(b.innerHTML=a.innerHTML)):c==="input"&&bv.test(a.type)?(b.defaultChecked=b.checked=a.checked,b.value!==a.value&&(b.value=a.value)):c==="option"?b.selected=a.defaultSelected:c==="input"||c==="textarea"?b.defaultValue=a.defaultValue:c==="scri
pt"&&b.text!==a.text&&(b.text=a.text),b.removeAttribute(p.expando)}function bF(a){return typeof a.getElementsByTagName!="undefined"?a.getElementsByTagName("*"):typeof a.querySelectorAll!="undefined"?a.querySelectorAll("*"):[]}function bG(a){bv.test(a.type)&&(a.defaultChecked=a.checked)}function bX(a,b){if(b in a)return b;var c=b.charAt(0).toUpperCase()+b.slice(1),d=b,e=bV.length;while(e--){b=bV[e]+c;if(b in a)return b}return d}function bY(a,b){return a=b||a,p.css(a,"display")==="none"||!p.contains(a.ownerDocument,a)}function bZ(a,b){var c,d,e=[],f=0,g=a.length;for(;f<g;f++){c=a[f];if(!c.style)continue;e[f]=p._data(c,"olddisplay"),b?(!e[f]&&c.style.display==="none"&&(c.style.display=""),c.style.display===""&&bY(c)&&(e[f]=p._data(c,"olddisplay",cb(c.nodeName)))):(d=bH(c,"display"),!e[f]&&d!=="none"&&p._data(c,"olddisplay",d))}for(f=0;f<g;f++){c=a[f];if(!c.style)continue;if(!b||c.style.display==="none"||c.style.display==="")c.style.display=b?e[f]||"":"none"}return a}function b$(a,b,c){
var d=bO.exec(b);return d?Math.max(0,d[1]-(c||0))+(d[2]||"px"):b}function b_(a,b,c,d){var e=c===(d?"border":"content")?4:b==="width"?1:0,f=0;for(;e<4;e+=2)c==="margin"&&(f+=p.css(a,c+bU[e],!0)),d?(c==="content"&&(f-=parseFloat(bH(a,"padding"+bU[e]))||0),c!=="margin"&&(f-=parseFloat(bH(a,"border"+bU[e]+"Width"))||0)):(f+=parseFloat(bH(a,"padding"+bU[e]))||0,c!=="padding"&&(f+=parseFloat(bH(a,"border"+bU[e]+"Width"))||0));return f}function ca(a,b,c){var d=b==="width"?a.offsetWidth:a.offsetHeight,e=!0,f=p.support.boxSizing&&p.css(a,"boxSizing")==="border-box";if(d<=0){d=bH(a,b);if(d<0||d==null)d=a.style[b];if(bP.test(d))return d;e=f&&(p.support.boxSizingReliable||d===a.style[b]),d=parseFloat(d)||0}return d+b_(a,b,c||(f?"border":"content"),e)+"px"}function cb(a){if(bR[a])return bR[a];var b=p("<"+a+">").appendTo(e.body),c=b.css("display");b.remove();if(c==="none"||c===""){bI=e.body.appendChild(bI||p.extend(e.createElement("iframe"),{frameBorder:0,width:0,height:0}));if(!bJ||!bI.createEle
ment)bJ=(bI.contentWindow||bI.contentDocument).document,bJ.write("<!doctype html><html><body>"),bJ.close();b=bJ.body.appendChild(bJ.createElement(a)),c=bH(b,"display"),e.body.removeChild(bI)}return bR[a]=c,c}function ch(a,b,c,d){var e;if(p.isArray(b))p.each(b,function(b,e){c||cd.test(a)?d(a,e):ch(a+"["+(typeof e=="object"?b:"")+"]",e,c,d)});else if(!c&&p.type(b)==="object")for(e in b)ch(a+"["+e+"]",b[e],c,d);else d(a,b)}function cy(a){return function(b,c){typeof b!="string"&&(c=b,b="*");var d,e,f,g=b.toLowerCase().split(s),h=0,i=g.length;if(p.isFunction(c))for(;h<i;h++)d=g[h],f=/^\+/.test(d),f&&(d=d.substr(1)||"*"),e=a[d]=a[d]||[],e[f?"unshift":"push"](c)}}function cz(a,c,d,e,f,g){f=f||c.dataTypes[0],g=g||{},g[f]=!0;var h,i=a[f],j=0,k=i?i.length:0,l=a===cu;for(;j<k&&(l||!h);j++)h=i[j](c,d,e),typeof h=="string"&&(!l||g[h]?h=b:(c.dataTypes.unshift(h),h=cz(a,c,d,e,h,g)));return(l||!h)&&!g["*"]&&(h=cz(a,c,d,e,"*",g)),h}function cA(a,c){var d,e,f=p.ajaxSettings.flatOptions||{};for(d in c
)c[d]!==b&&((f[d]?a:e||(e={}))[d]=c[d]);e&&p.extend(!0,a,e)}function cB(a,c,d){var e,f,g,h,i=a.contents,j=a.dataTypes,k=a.responseFields;for(f in k)f in d&&(c[k[f]]=d[f]);while(j[0]==="*")j.shift(),e===b&&(e=a.mimeType||c.getResponseHeader("content-type"));if(e)for(f in i)if(i[f]&&i[f].test(e)){j.unshift(f);break}if(j[0]in d)g=j[0];else{for(f in d){if(!j[0]||a.converters[f+" "+j[0]]){g=f;break}h||(h=f)}g=g||h}if(g)return g!==j[0]&&j.unshift(g),d[g]}function cC(a,b){var c,d,e,f,g=a.dataTypes.slice(),h=g[0],i={},j=0;a.dataFilter&&(b=a.dataFilter(b,a.dataType));if(g[1])for(c in a.converters)i[c.toLowerCase()]=a.converters[c];for(;e=g[++j];)if(e!=="*"){if(h!=="*"&&h!==e){c=i[h+" "+e]||i["* "+e];if(!c)for(d in i){f=d.split(" ");if(f[1]===e){c=i[h+" "+f[0]]||i["* "+f[0]];if(c){c===!0?c=i[d]:i[d]!==!0&&(e=f[0],g.splice(j--,0,e));break}}}if(c!==!0)if(c&&a["throws"])b=c(b);else try{b=c(b)}catch(k){return{state:"parsererror",error:c?k:"No conversion from "+h+" to "+e}}}h=e}return{state:"succe
ss",data:b}}function cK(){try{return new a.XMLHttpRequest}catch(b){}}function cL(){try{return new a.ActiveXObject("Microsoft.XMLHTTP")}catch(b){}}function cT(){return setTimeout(function(){cM=b},0),cM=p.now()}function cU(a,b){p.each(b,function(b,c){var d=(cS[b]||[]).concat(cS["*"]),e=0,f=d.length;for(;e<f;e++)if(d[e].call(a,b,c))return})}function cV(a,b,c){var d,e=0,f=0,g=cR.length,h=p.Deferred().always(function(){delete i.elem}),i=function(){var b=cM||cT(),c=Math.max(0,j.startTime+j.duration-b),d=1-(c/j.duration||0),e=0,f=j.tweens.length;for(;e<f;e++)j.tweens[e].run(d);return h.notifyWith(a,[j,d,c]),d<1&&f?c:(h.resolveWith(a,[j]),!1)},j=h.promise({elem:a,props:p.extend({},b),opts:p.extend(!0,{specialEasing:{}},c),originalProperties:b,originalOptions:c,startTime:cM||cT(),duration:c.duration,tweens:[],createTween:function(b,c,d){var e=p.Tween(a,j.opts,b,c,j.opts.specialEasing[b]||j.opts.easing);return j.tweens.push(e),e},stop:function(b){var c=0,d=b?j.tweens.length:0;for(;c<d;c++)j.t
weens[c].run(1);return b?h.resolveWith(a,[j,b]):h.rejectWith(a,[j,b]),this}}),k=j.props;cW(k,j.opts.specialEasing);for(;e<g;e++){d=cR[e].call(j,a,k,j.opts);if(d)return d}return cU(j,k),p.isFunction(j.opts.start)&&j.opts.start.call(a,j),p.fx.timer(p.extend(i,{anim:j,queue:j.opts.queue,elem:a})),j.progress(j.opts.progress).done(j.opts.done,j.opts.complete).fail(j.opts.fail).always(j.opts.always)}function cW(a,b){var c,d,e,f,g;for(c in a){d=p.camelCase(c),e=b[d],f=a[c],p.isArray(f)&&(e=f[1],f=a[c]=f[0]),c!==d&&(a[d]=f,delete a[c]),g=p.cssHooks[d];if(g&&"expand"in g){f=g.expand(f),delete a[d];for(c in f)c in a||(a[c]=f[c],b[c]=e)}else b[d]=e}}function cX(a,b,c){var d,e,f,g,h,i,j,k,l=this,m=a.style,n={},o=[],q=a.nodeType&&bY(a);c.queue||(j=p._queueHooks(a,"fx"),j.unqueued==null&&(j.unqueued=0,k=j.empty.fire,j.empty.fire=function(){j.unqueued||k()}),j.unqueued++,l.always(function(){l.always(function(){j.unqueued--,p.queue(a,"fx").length||j.empty.fire()})})),a.nodeType===1&&("height"in b||
"width"in b)&&(c.overflow=[m.overflow,m.overflowX,m.overflowY],p.css(a,"display")==="inline"&&p.css(a,"float")==="none"&&(!p.support.inlineBlockNeedsLayout||cb(a.nodeName)==="inline"?m.display="inline-block":m.zoom=1)),c.overflow&&(m.overflow="hidden",p.support.shrinkWrapBlocks||l.done(function(){m.overflow=c.overflow[0],m.overflowX=c.overflow[1],m.overflowY=c.overflow[2]}));for(d in b){f=b[d];if(cO.exec(f)){delete b[d];if(f===(q?"hide":"show"))continue;o.push(d)}}g=o.length;if(g){h=p._data(a,"fxshow")||p._data(a,"fxshow",{}),q?p(a).show():l.done(function(){p(a).hide()}),l.done(function(){var b;p.removeData(a,"fxshow",!0);for(b in n)p.style(a,b,n[b])});for(d=0;d<g;d++)e=o[d],i=l.createTween(e,q?h[e]:0),n[e]=h[e]||p.style(a,e),e in h||(h[e]=i.start,q&&(i.end=i.start,i.start=e==="width"||e==="height"?1:0))}}function cY(a,b,c,d,e){return new cY.prototype.init(a,b,c,d,e)}function cZ(a,b){var c,d={height:a},e=0;for(;e<4;e+=2-b)c=bU[e],d["margin"+c]=d["padding"+c]=a;return b&&(d.opacity=d
.width=a),d}function c_(a){return p.isWindow(a)?a:a.nodeType===9?a.defaultView||a.parentWindow:!1}var c,d,e=a.document,f=a.location,g=a.navigator,h=a.jQuery,i=a.$,j=Array.prototype.push,k=Array.prototype.slice,l=Array.prototype.indexOf,m=Object.prototype.toString,n=Object.prototype.hasOwnProperty,o=String.prototype.trim,p=function(a,b){return new p.fn.init(a,b,c)},q=/[\-+]?(?:\d*\.|)\d+(?:[eE][\-+]?\d+|)/.source,r=/\S/,s=/\s+/,t=r.test(" ")?/^[\s\xA0]+|[\s\xA0]+$/g:/^\s+|\s+$/g,u=/^(?:[^#<]*(<[\w\W]+>)[^>]*$|#([\w\-]*)$)/,v=/^<(\w+)\s*\/?>(?:<\/\1>|)$/,w=/^[\],:{}\s]*$/,x=/(?:^|:|,)(?:\s*\[)+/g,y=/\\(?:["\\\/bfnrt]|u[\da-fA-F]{4})/g,z=/"[^"\\\r\n]*"|true|false|null|-?(?:\d\d*\.|)\d+(?:[eE][\-+]?\d+|)/g,A=/^-ms-/,B=/-([\da-z])/gi,C=function(a,b){return(b+"").toUpperCase()},D=function(){e.addEventListener?(e.removeEventListener("DOMContentLoaded",D,!1),p.ready()):e.readyState==="complete"&&(e.detachEvent("onreadystatechange",D),p.ready())},E={};p.fn=p.prototype={constructor:p,init:fu
nction(a,c,d){var f,g,h,i;if(!a)return this;if(a.nodeType)return this.context=this[0]=a,this.length=1,this;if(typeof a=="string"){a.charAt(0)==="<"&&a.charAt(a.length-1)===">"&&a.length>=3?f=[null,a,null]:f=u.exec(a);if(f&&(f[1]||!c)){if(f[1])return c=c instanceof p?c[0]:c,i=c&&c.nodeType?c.ownerDocument||c:e,a=p.parseHTML(f[1],i,!0),v.test(f[1])&&p.isPlainObject(c)&&this.attr.call(a,c,!0),p.merge(this,a);g=e.getElementById(f[2]);if(g&&g.parentNode){if(g.id!==f[2])return d.find(a);this.length=1,this[0]=g}return this.context=e,this.selector=a,this}return!c||c.jquery?(c||d).find(a):this.constructor(c).find(a)}return p.isFunction(a)?d.ready(a):(a.selector!==b&&(this.selector=a.selector,this.context=a.context),p.makeArray(a,this))},selector:"",jquery:"1.8.0",length:0,size:function(){return this.length},toArray:function(){return k.call(this)},get:function(a){return a==null?this.toArray():a<0?this[this.length+a]:this[a]},pushStack:function(a,b,c){var d=p.merge(this.constructor(),a);return
d.prevObject=this,d.context=this.context,b==="find"?d.selector=this.selector+(this.selector?" ":"")+c:b&&(d.selector=this.selector+"."+b+"("+c+")"),d},each:function(a,b){return p.each(this,a,b)},ready:function(a){return p.ready.promise().done(a),this},eq:function(a){return a=+a,a===-1?this.slice(a):this.slice(a,a+1)},first:function(){return this.eq(0)},last:function(){return this.eq(-1)},slice:function(){return this.pushStack(k.apply(this,arguments),"slice",k.call(arguments).join(","))},map:function(a){return this.pushStack(p.map(this,function(b,c){return a.call(b,c,b)}))},end:function(){return this.prevObject||this.constructor(null)},push:j,sort:[].sort,splice:[].splice},p.fn.init.prototype=p.fn,p.extend=p.fn.extend=function(){var a,c,d,e,f,g,h=arguments[0]||{},i=1,j=arguments.length,k=!1;typeof h=="boolean"&&(k=h,h=arguments[1]||{},i=2),typeof h!="object"&&!p.isFunction(h)&&(h={}),j===i&&(h=this,--i);for(;i<j;i++)if((a=arguments[i])!=null)for(c in a){d=h[c],e=a[c];if(h===e)contin
ue;k&&e&&(p.isPlainObject(e)||(f=p.isArray(e)))?(f?(f=!1,g=d&&p.isArray(d)?d:[]):g=d&&p.isPlainObject(d)?d:{},h[c]=p.extend(k,g,e)):e!==b&&(h[c]=e)}return h},p.extend({noConflict:function(b){return a.$===p&&(a.$=i),b&&a.jQuery===p&&(a.jQuery=h),p},isReady:!1,readyWait:1,holdReady:function(a){a?p.readyWait++:p.ready(!0)},ready:function(a){if(a===!0?--p.readyWait:p.isReady)return;if(!e.body)return setTimeout(p.ready,1);p.isReady=!0;if(a!==!0&&--p.readyWait>0)return;d.resolveWith(e,[p]),p.fn.trigger&&p(e).trigger("ready").off("ready")},isFunction:function(a){return p.type(a)==="function"},isArray:Array.isArray||function(a){return p.type(a)==="array"},isWindow:function(a){return a!=null&&a==a.window},isNumeric:function(a){return!isNaN(parseFloat(a))&&isFinite(a)},type:function(a){return a==null?String(a):E[m.call(a)]||"object"},isPlainObject:function(a){if(!a||p.type(a)!=="object"||a.nodeType||p.isWindow(a))return!1;try{if(a.constructor&&!n.call(a,"constructor")&&!n.call(a.constructor.p
rototype,"isPrototypeOf"))return!1}catch(c){return!1}var d;for(d in a);return d===b||n.call(a,d)},isEmptyObject:function(a){var b;for(b in a)return!1;return!0},error:function(a){throw new Error(a)},parseHTML:function(a,b,c){var d;return!a||typeof a!="string"?null:(typeof b=="boolean"&&(c=b,b=0),b=b||e,(d=v.exec(a))?[b.createElement(d[1])]:(d=p.buildFragment([a],b,c?null:[]),p.merge([],(d.cacheable?p.clone(d.fragment):d.fragment).childNodes)))},parseJSON:function(b){if(!b||typeof b!="string")return null;b=p.trim(b);if(a.JSON&&a.JSON.parse)return a.JSON.parse(b);if(w.test(b.replace(y,"@").replace(z,"]").replace(x,"")))return(new Function("return "+b))();p.error("Invalid JSON: "+b)},parseXML:function(c){var d,e;if(!c||typeof c!="string")return null;try{a.DOMParser?(e=new DOMParser,d=e.parseFromString(c,"text/xml")):(d=new ActiveXObject("Microsoft.XMLDOM"),d.async="false",d.loadXML(c))}catch(f){d=b}return(!d||!d.documentElement||d.getElementsByTagName("parsererror").length)&&p.error("In
valid XML: "+c),d},noop:function(){},globalEval:function(b){b&&r.test(b)&&(a.execScript||function(b){a.eval.call(a,b)})(b)},camelCase:function(a){return a.replace(A,"ms-").replace(B,C)},nodeName:function(a,b){return a.nodeName&&a.nodeName.toUpperCase()===b.toUpperCase()},each:function(a,c,d){var e,f=0,g=a.length,h=g===b||p.isFunction(a);if(d){if(h){for(e in a)if(c.apply(a[e],d)===!1)break}else for(;f<g;)if(c.apply(a[f++],d)===!1)break}else if(h){for(e in a)if(c.call(a[e],e,a[e])===!1)break}else for(;f<g;)if(c.call(a[f],f,a[f++])===!1)break;return a},trim:o?function(a){return a==null?"":o.call(a)}:function(a){return a==null?"":a.toString().replace(t,"")},makeArray:function(a,b){var c,d=b||[];return a!=null&&(c=p.type(a),a.length==null||c==="string"||c==="function"||c==="regexp"||p.isWindow(a)?j.call(d,a):p.merge(d,a)),d},inArray:function(a,b,c){var d;if(b){if(l)return l.call(b,a,c);d=b.length,c=c?c<0?Math.max(0,d+c):c:0;for(;c<d;c++)if(c in b&&b[c]===a)return c}return-1},merge:functi
on(a,c){var d=c.length,e=a.length,f=0;if(typeof d=="number")for(;f<d;f++)a[e++]=c[f];else while(c[f]!==b)a[e++]=c[f++];return a.length=e,a},grep:function(a,b,c){var d,e=[],f=0,g=a.length;c=!!c;for(;f<g;f++)d=!!b(a[f],f),c!==d&&e.push(a[f]);return e},map:function(a,c,d){var e,f,g=[],h=0,i=a.length,j=a instanceof p||i!==b&&typeof i=="number"&&(i>0&&a[0]&&a[i-1]||i===0||p.isArray(a));if(j)for(;h<i;h++)e=c(a[h],h,d),e!=null&&(g[g.length]=e);else for(f in a)e=c(a[f],f,d),e!=null&&(g[g.length]=e);return g.concat.apply([],g)},guid:1,proxy:function(a,c){var d,e,f;return typeof c=="string"&&(d=a[c],c=a,a=d),p.isFunction(a)?(e=k.call(arguments,2),f=function(){return a.apply(c,e.concat(k.call(arguments)))},f.guid=a.guid=a.guid||f.guid||p.guid++,f):b},access:function(a,c,d,e,f,g,h){var i,j=d==null,k=0,l=a.length;if(d&&typeof d=="object"){for(k in d)p.access(a,c,k,d[k],1,g,e);f=1}else if(e!==b){i=h===b&&p.isFunction(e),j&&(i?(i=c,c=function(a,b,c){return i.call(p(a),c)}):(c.call(a,e),c=null));if
(c)for(;k<l;k++)c(a[k],d,i?e.call(a[k],k,c(a[k],d)):e,h);f=1}return f?a:j?c.call(a):l?c(a[0],d):g},now:function(){return(new Date).getTime()}}),p.ready.promise=function(b){if(!d){d=p.Deferred();if(e.readyState==="complete"||e.readyState!=="loading"&&e.addEventListener)setTimeout(p.ready,1);else if(e.addEventListener)e.addEventListener("DOMContentLoaded",D,!1),a.addEventListener("load",p.ready,!1);else{e.attachEvent("onreadystatechange",D),a.attachEvent("onload",p.ready);var c=!1;try{c=a.frameElement==null&&e.documentElement}catch(f){}c&&c.doScroll&&function g(){if(!p.isReady){try{c.doScroll("left")}catch(a){return setTimeout(g,50)}p.ready()}}()}}return d.promise(b)},p.each("Boolean Number String Function Array Date RegExp Object".split(" "),function(a,b){E["[object "+b+"]"]=b.toLowerCase()}),c=p(e);var F={};p.Callbacks=function(a){a=typeof a=="string"?F[a]||G(a):p.extend({},a);var c,d,e,f,g,h,i=[],j=!a.once&&[],k=function(b){c=a.memory&&b,d=!0,h=f||0,f=0,g=i.length,e=!0;for(;i&&h<g;
h++)if(i[h].apply(b[0],b[1])===!1&&a.stopOnFalse){c=!1;break}e=!1,i&&(j?j.length&&k(j.shift()):c?i=[]:l.disable())},l={add:function(){if(i){var b=i.length;(function d(b){p.each(b,function(b,c){p.isFunction(c)&&(!a.unique||!l.has(c))?i.push(c):c&&c.length&&d(c)})})(arguments),e?g=i.length:c&&(f=b,k(c))}return this},remove:function(){return i&&p.each(arguments,function(a,b){var c;while((c=p.inArray(b,i,c))>-1)i.splice(c,1),e&&(c<=g&&g--,c<=h&&h--)}),this},has:function(a){return p.inArray(a,i)>-1},empty:function(){return i=[],this},disable:function(){return i=j=c=b,this},disabled:function(){return!i},lock:function(){return j=b,c||l.disable(),this},locked:function(){return!j},fireWith:function(a,b){return b=b||[],b=[a,b.slice?b.slice():b],i&&(!d||j)&&(e?j.push(b):k(b)),this},fire:function(){return l.fireWith(this,arguments),this},fired:function(){return!!d}};return l},p.extend({Deferred:function(a){var b=[["resolve","done",p.Callbacks("once memory"),"resolved"],["reject","fail",p.Callba
cks("once memory"),"rejected"],["notify","progress",p.Callbacks("memory")]],c="pending",d={state:function(){return c},always:function(){return e.done(arguments).fail(arguments),this},then:function(){var a=arguments;return p.Deferred(function(c){p.each(b,function(b,d){var f=d[0],g=a[b];e[d[1]](p.isFunction(g)?function(){var a=g.apply(this,arguments);a&&p.isFunction(a.promise)?a.promise().done(c.resolve).fail(c.reject).progress(c.notify):c[f+"With"](this===e?c:this,[a])}:c[f])}),a=null}).promise()},promise:function(a){return typeof a=="object"?p.extend(a,d):d}},e={};return d.pipe=d.then,p.each(b,function(a,f){var g=f[2],h=f[3];d[f[1]]=g.add,h&&g.add(function(){c=h},b[a^1][2].disable,b[2][2].lock),e[f[0]]=g.fire,e[f[0]+"With"]=g.fireWith}),d.promise(e),a&&a.call(e,e),e},when:function(a){var b=0,c=k.call(arguments),d=c.length,e=d!==1||a&&p.isFunction(a.promise)?d:0,f=e===1?a:p.Deferred(),g=function(a,b,c){return function(d){b[a]=this,c[a]=arguments.length>1?k.call(arguments):d,c===h?f.n
otifyWith(b,c):--e||f.resolveWith(b,c)}},h,i,j;if(d>1){h=new Array(d),i=new Array(d),j=new Array(d);for(;b<d;b++)c[b]&&p.isFunction(c[b].promise)?c[b].promise().done(g(b,j,c)).fail(f.reject).progress(g(b,i,h)):--e}return e||f.resolveWith(j,c),f.promise()}}),p.support=function(){var b,c,d,f,g,h,i,j,k,l,m,n=e.createElement("div");n.setAttribute("className","t"),n.innerHTML=" <link/><table></table><a href='/a'>a</a><input type='checkbox'/>",c=n.getElementsByTagName("*"),d=n.getElementsByTagName("a")[0],d.style.cssText="top:1px;float:left;opacity:.5";if(!c||!c.length||!d)return{};f=e.createElement("select"),g=f.appendChild(e.createElement("option")),h=n.getElementsByTagName("input")[0],b={leadingWhitespace:n.firstChild.nodeType===3,tbody:!n.getElementsByTagName("tbody").length,htmlSerialize:!!n.getElementsByTagName("link").length,style:/top/.test(d.getAttribute("style")),hrefNormalized:d.getAttribute("href")==="/a",opacity:/^0.5/.test(d.style.opacity),cssFloat:!!d.style.cssFloat,checkO
n:h.value==="on",optSelected:g.selected,getSetAttribute:n.className!=="t",enctype:!!e.createElement("form").enctype,html5Clone:e.createElement("nav").cloneNode(!0).outerHTML!=="<:nav></:nav>",boxModel:e.compatMode==="CSS1Compat",submitBubbles:!0,changeBubbles:!0,focusinBubbles:!1,deleteExpando:!0,noCloneEvent:!0,inlineBlockNeedsLayout:!1,shrinkWrapBlocks:!1,reliableMarginRight:!0,boxSizingReliable:!0,pixelPosition:!1},h.checked=!0,b.noCloneChecked=h.cloneNode(!0).checked,f.disabled=!0,b.optDisabled=!g.disabled;try{delete n.test}catch(o){b.deleteExpando=!1}!n.addEventListener&&n.attachEvent&&n.fireEvent&&(n.attachEvent("onclick",m=function(){b.noCloneEvent=!1}),n.cloneNode(!0).fireEvent("onclick"),n.detachEvent("onclick",m)),h=e.createElement("input"),h.value="t",h.setAttribute("type","radio"),b.radioValue=h.value==="t",h.setAttribute("checked","checked"),h.setAttribute("name","t"),n.appendChild(h),i=e.createDocumentFragment(),i.appendChild(n.lastChild),b.checkClone=i.cloneNode(!0).c
loneNode(!0).lastChild.checked,b.appendChecked=h.checked,i.removeChild(h),i.appendChild(n);if(n.attachEvent)for(k in{submit:!0,change:!0,focusin:!0})j="on"+k,l=j in n,l||(n.setAttribute(j,"return;"),l=typeof n[j]=="function"),b[k+"Bubbles"]=l;return p(function(){var c,d,f,g,h="padding:0;margin:0;border:0;display:block;overflow:hidden;",i=e.getElementsByTagName("body")[0];if(!i)return;c=e.createElement("div"),c.style.cssText="visibility:hidden;border:0;width:0;height:0;position:static;top:0;margin-top:1px",i.insertBefore(c,i.firstChild),d=e.createElement("div"),c.appendChild(d),d.innerHTML="<table><tr><td></td><td>t</td></tr></table>",f=d.getElementsByTagName("td"),f[0].style.cssText="padding:0;margin:0;border:0;display:none",l=f[0].offsetHeight===0,f[0].style.display="",f[1].style.display="none",b.reliableHiddenOffsets=l&&f[0].offsetHeight===0,d.innerHTML="",d.style.cssText="box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box;padding:1px;border:1px;display
:block;width:4px;margin-top:1%;position:absolute;top:1%;",b.boxSizing=d.offsetWidth===4,b.doesNotIncludeMarginInBodyOffset=i.offsetTop!==1,a.getComputedStyle&&(b.pixelPosition=(a.getComputedStyle(d,null)||{}).top!=="1%",b.boxSizingReliable=(a.getComputedStyle(d,null)||{width:"4px"}).width==="4px",g=e.createElement("div"),g.style.cssText=d.style.cssText=h,g.style.marginRight=g.style.width="0",d.style.width="1px",d.appendChild(g),b.reliableMarginRight=!parseFloat((a.getComputedStyle(g,null)||{}).marginRight)),typeof d.style.zoom!="undefined"&&(d.innerHTML="",d.style.cssText=h+"width:1px;padding:1px;display:inline;zoom:1",b.inlineBlockNeedsLayout=d.offsetWidth===3,d.style.display="block",d.style.overflow="visible",d.innerHTML="<div></div>",d.firstChild.style.width="5px",b.shrinkWrapBlocks=d.offsetWidth!==3,c.style.zoom=1),i.removeChild(c),c=d=f=g=null}),i.removeChild(n),c=d=f=g=h=i=n=null,b}();var H=/^(?:\{.*\}|\[.*\])$/,I=/([A-Z])/g;p.extend({cache:{},deletedIds:[],uuid:0,expando:"jQu
ery"+(p.fn.jquery+Math.random()).replace(/\D/g,""),noData:{embed:!0,object:"clsid:D27CDB6E-AE6D-11cf-96B8-444553540000",applet:!0},hasData:function(a){return a=a.nodeType?p.cache[a[p.expando]]:a[p.expando],!!a&&!K(a)},data:function(a,c,d,e){if(!p.acceptData(a))return;var f,g,h=p.expando,i=typeof c=="string",j=a.nodeType,k=j?p.cache:a,l=j?a[h]:a[h]&&h;if((!l||!k[l]||!e&&!k[l].data)&&i&&d===b)return;l||(j?a[h]=l=p.deletedIds.pop()||++p.uuid:l=h),k[l]||(k[l]={},j||(k[l].toJSON=p.noop));if(typeof c=="object"||typeof c=="function")e?k[l]=p.extend(k[l],c):k[l].data=p.extend(k[l].data,c);return f=k[l],e||(f.data||(f.data={}),f=f.data),d!==b&&(f[p.camelCase(c)]=d),i?(g=f[c],g==null&&(g=f[p.camelCase(c)])):g=f,g},removeData:function(a,b,c){if(!p.acceptData(a))return;var d,e,f,g=a.nodeType,h=g?p.cache:a,i=g?a[p.expando]:p.expando;if(!h[i])return;if(b){d=c?h[i]:h[i].data;if(d){p.isArray(b)||(b in d?b=[b]:(b=p.camelCase(b),b in d?b=[b]:b=b.split(" ")));for(e=0,f=b.length;e<f;e++)delete d[b[e]];
if(!(c?K:p.isEmptyObject)(d))return}}if(!c){delete h[i].data;if(!K(h[i]))return}g?p.cleanData([a],!0):p.support.deleteExpando||h!=h.window?delete h[i]:h[i]=null},_data:function(a,b,c){return p.data(a,b,c,!0)},acceptData:function(a){var b=a.nodeName&&p.noData[a.nodeName.toLowerCase()];return!b||b!==!0&&a.getAttribute("classid")===b}}),p.fn.extend({data:function(a,c){var d,e,f,g,h,i=this[0],j=0,k=null;if(a===b){if(this.length){k=p.data(i);if(i.nodeType===1&&!p._data(i,"parsedAttrs")){f=i.attributes;for(h=f.length;j<h;j++)g=f[j].name,g.indexOf("data-")===0&&(g=p.camelCase(g.substring(5)),J(i,g,k[g]));p._data(i,"parsedAttrs",!0)}}return k}return typeof a=="object"?this.each(function(){p.data(this,a)}):(d=a.split(".",2),d[1]=d[1]?"."+d[1]:"",e=d[1]+"!",p.access(this,function(c){if(c===b)return k=this.triggerHandler("getData"+e,[d[0]]),k===b&&i&&(k=p.data(i,a),k=J(i,a,k)),k===b&&d[1]?this.data(d[0]):k;d[1]=c,this.each(function(){var b=p(this);b.triggerHandler("setData"+e,d),p.data(this,a,
c),b.triggerHandler("changeData"+e,d)})},null,c,arguments.length>1,null,!1))},removeData:function(a){return this.each(function(){p.removeData(this,a)})}}),p.extend({queue:function(a,b,c){var d;if(a)return b=(b||"fx")+"queue",d=p._data(a,b),c&&(!d||p.isArray(c)?d=p._data(a,b,p.makeArray(c)):d.push(c)),d||[]},dequeue:function(a,b){b=b||"fx";var c=p.queue(a,b),d=c.shift(),e=p._queueHooks(a,b),f=function(){p.dequeue(a,b)};d==="inprogress"&&(d=c.shift()),d&&(b==="fx"&&c.unshift("inprogress"),delete e.stop,d.call(a,f,e)),!c.length&&e&&e.empty.fire()},_queueHooks:function(a,b){var c=b+"queueHooks";return p._data(a,c)||p._data(a,c,{empty:p.Callbacks("once memory").add(function(){p.removeData(a,b+"queue",!0),p.removeData(a,c,!0)})})}}),p.fn.extend({queue:function(a,c){var d=2;return typeof a!="string"&&(c=a,a="fx",d--),arguments.length<d?p.queue(this[0],a):c===b?this:this.each(function(){var b=p.queue(this,a,c);p._queueHooks(this,a),a==="fx"&&b[0]!=="inprogress"&&p.dequeue(this,a)})},dequeue
:function(a){return this.each(function(){p.dequeue(this,a)})},delay:function(a,b){return a=p.fx?p.fx.speeds[a]||a:a,b=b||"fx",this.queue(b,function(b,c){var d=setTimeout(b,a);c.stop=function(){clearTimeout(d)}})},clearQueue:function(a){return this.queue(a||"fx",[])},promise:function(a,c){var d,e=1,f=p.Deferred(),g=this,h=this.length,i=function(){--e||f.resolveWith(g,[g])};typeof a!="string"&&(c=a,a=b),a=a||"fx";while(h--)(d=p._data(g[h],a+"queueHooks"))&&d.empty&&(e++,d.empty.add(i));return i(),f.promise(c)}});var L,M,N,O=/[\t\r\n]/g,P=/\r/g,Q=/^(?:button|input)$/i,R=/^(?:button|input|object|select|textarea)$/i,S=/^a(?:rea|)$/i,T=/^(?:autofocus|autoplay|async|checked|controls|defer|disabled|hidden|loop|multiple|open|readonly|required|scoped|selected)$/i,U=p.support.getSetAttribute;p.fn.extend({attr:function(a,b){return p.access(this,p.attr,a,b,arguments.length>1)},removeAttr:function(a){return this.each(function(){p.removeAttr(this,a)})},prop:function(a,b){return p.access(this,p.pro
p,a,b,arguments.length>1)},removeProp:function(a){return a=p.propFix[a]||a,this.each(function(){try{this[a]=b,delete this[a]}catch(c){}})},addClass:function(a){var b,c,d,e,f,g,h;if(p.isFunction(a))return this.each(function(b){p(this).addClass(a.call(this,b,this.className))});if(a&&typeof a=="string"){b=a.split(s);for(c=0,d=this.length;c<d;c++){e=this[c];if(e.nodeType===1)if(!e.className&&b.length===1)e.className=a;else{f=" "+e.className+" ";for(g=0,h=b.length;g<h;g++)~f.indexOf(" "+b[g]+" ")||(f+=b[g]+" ");e.className=p.trim(f)}}}return this},removeClass:function(a){var c,d,e,f,g,h,i;if(p.isFunction(a))return this.each(function(b){p(this).removeClass(a.call(this,b,this.className))});if(a&&typeof a=="string"||a===b){c=(a||"").split(s);for(h=0,i=this.length;h<i;h++){e=this[h];if(e.nodeType===1&&e.className){d=(" "+e.className+" ").replace(O," ");for(f=0,g=c.length;f<g;f++)while(d.indexOf(" "+c[f]+" ")>-1)d=d.replace(" "+c[f]+" "," ");e.className=a?p.trim(d):""}}}return this},toggleCla
ss:function(a,b){var c=typeof a,d=typeof b=="boolean";return p.isFunction(a)?this.each(function(c){p(this).toggleClass(a.call(this,c,this.className,b),b)}):this.each(function(){if(c==="string"){var e,f=0,g=p(this),h=b,i=a.split(s);while(e=i[f++])h=d?h:!g.hasClass(e),g[h?"addClass":"removeClass"](e)}else if(c==="undefined"||c==="boolean")this.className&&p._data(this,"__className__",this.className),this.className=this.className||a===!1?"":p._data(this,"__className__")||""})},hasClass:function(a){var b=" "+a+" ",c=0,d=this.length;for(;c<d;c++)if(this[c].nodeType===1&&(" "+this[c].className+" ").replace(O," ").indexOf(b)>-1)return!0;return!1},val:function(a){var c,d,e,f=this[0];if(!arguments.length){if(f)return c=p.valHooks[f.type]||p.valHooks[f.nodeName.toLowerCase()],c&&"get"in c&&(d=c.get(f,"value"))!==b?d:(d=f.value,typeof d=="string"?d.replace(P,""):d==null?"":d);return}return e=p.isFunction(a),this.each(function(d){var f,g=p(this);if(this.nodeType!==1)return;e?f=a.call(this,d,g.va
l()):f=a,f==null?f="":typeof f=="number"?f+="":p.isArray(f)&&(f=p.map(f,function(a){return a==null?"":a+""})),c=p.valHooks[this.type]||p.valHooks[this.nodeName.toLowerCase()];if(!c||!("set"in c)||c.set(this,f,"value")===b)this.value=f})}}),p.extend({valHooks:{option:{get:function(a){var b=a.attributes.value;return!b||b.specified?a.value:a.text}},select:{get:function(a){var b,c,d,e,f=a.selectedIndex,g=[],h=a.options,i=a.type==="select-one";if(f<0)return null;c=i?f:0,d=i?f+1:h.length;for(;c<d;c++){e=h[c];if(e.selected&&(p.support.optDisabled?!e.disabled:e.getAttribute("disabled")===null)&&(!e.parentNode.disabled||!p.nodeName(e.parentNode,"optgroup"))){b=p(e).val();if(i)return b;g.push(b)}}return i&&!g.length&&h.length?p(h[f]).val():g},set:function(a,b){var c=p.makeArray(b);return p(a).find("option").each(function(){this.selected=p.inArray(p(this).val(),c)>=0}),c.length||(a.selectedIndex=-1),c}}},attrFn:{},attr:function(a,c,d,e){var f,g,h,i=a.nodeType;if(!a||i===3||i===8||i===2)return;
if(e&&p.isFunction(p.fn[c]))return p(a)[c](d);if(typeof a.getAttribute=="undefined")return p.prop(a,c,d);h=i!==1||!p.isXMLDoc(a),h&&(c=c.toLowerCase(),g=p.attrHooks[c]||(T.test(c)?M:L));if(d!==b){if(d===null){p.removeAttr(a,c);return}return g&&"set"in g&&h&&(f=g.set(a,d,c))!==b?f:(a.setAttribute(c,""+d),d)}return g&&"get"in g&&h&&(f=g.get(a,c))!==null?f:(f=a.getAttribute(c),f===null?b:f)},removeAttr:function(a,b){var c,d,e,f,g=0;if(b&&a.nodeType===1){d=b.split(s);for(;g<d.length;g++)e=d[g],e&&(c=p.propFix[e]||e,f=T.test(e),f||p.attr(a,e,""),a.removeAttribute(U?e:c),f&&c in a&&(a[c]=!1))}},attrHooks:{type:{set:function(a,b){if(Q.test(a.nodeName)&&a.parentNode)p.error("type property can't be changed");else if(!p.support.radioValue&&b==="radio"&&p.nodeName(a,"input")){var c=a.value;return a.setAttribute("type",b),c&&(a.value=c),b}}},value:{get:function(a,b){return L&&p.nodeName(a,"button")?L.get(a,b):b in a?a.value:null},set:function(a,b,c){if(L&&p.nodeName(a,"button"))return L.set(a,b
,c);a.value=b}}},propFix:{tabindex:"tabIndex",readonly:"readOnly","for":"htmlFor","class":"className",maxlength:"maxLength",cellspacing:"cellSpacing",cellpadding:"cellPadding",rowspan:"rowSpan",colspan:"colSpan",usemap:"useMap",frameborder:"frameBorder",contenteditable:"contentEditable"},prop:function(a,c,d){var e,f,g,h=a.nodeType;if(!a||h===3||h===8||h===2)return;return g=h!==1||!p.isXMLDoc(a),g&&(c=p.propFix[c]||c,f=p.propHooks[c]),d!==b?f&&"set"in f&&(e=f.set(a,d,c))!==b?e:a[c]=d:f&&"get"in f&&(e=f.get(a,c))!==null?e:a[c]},propHooks:{tabIndex:{get:function(a){var c=a.getAttributeNode("tabindex");return c&&c.specified?parseInt(c.value,10):R.test(a.nodeName)||S.test(a.nodeName)&&a.href?0:b}}}}),M={get:function(a,c){var d,e=p.prop(a,c);return e===!0||typeof e!="boolean"&&(d=a.getAttributeNode(c))&&d.nodeValue!==!1?c.toLowerCase():b},set:function(a,b,c){var d;return b===!1?p.removeAttr(a,c):(d=p.propFix[c]||c,d in a&&(a[d]=!0),a.setAttribute(c,c.toLowerCase())),c}},U||(N={name:!0,id:
!0,coords:!0},L=p.valHooks.button={get:function(a,c){var d;return d=a.getAttributeNode(c),d&&(N[c]?d.value!=="":d.specified)?d.value:b},set:function(a,b,c){var d=a.getAttributeNode(c);return d||(d=e.createAttribute(c),a.setAttributeNode(d)),d.value=b+""}},p.each(["width","height"],function(a,b){p.attrHooks[b]=p.extend(p.attrHooks[b],{set:function(a,c){if(c==="")return a.setAttribute(b,"auto"),c}})}),p.attrHooks.contenteditable={get:L.get,set:function(a,b,c){b===""&&(b="false"),L.set(a,b,c)}}),p.support.hrefNormalized||p.each(["href","src","width","height"],function(a,c){p.attrHooks[c]=p.extend(p.attrHooks[c],{get:function(a){var d=a.getAttribute(c,2);return d===null?b:d}})}),p.support.style||(p.attrHooks.style={get:function(a){return a.style.cssText.toLowerCase()||b},set:function(a,b){return a.style.cssText=""+b}}),p.support.optSelected||(p.propHooks.selected=p.extend(p.propHooks.selected,{get:function(a){var b=a.parentNode;return b&&(b.selectedIndex,b.parentNode&&b.parentNode.selec
tedIndex),null}})),p.support.enctype||(p.propFix.enctype="encoding"),p.support.checkOn||p.each(["radio","checkbox"],function(){p.valHooks[this]={get:function(a){return a.getAttribute("value")===null?"on":a.value}}}),p.each(["radio","checkbox"],function(){p.valHooks[this]=p.extend(p.valHooks[this],{set:function(a,b){if(p.isArray(b))return a.checked=p.inArray(p(a).val(),b)>=0}})});var V=/^(?:textarea|input|select)$/i,W=/^([^\.]*|)(?:\.(.+)|)$/,X=/(?:^|\s)hover(\.\S+|)\b/,Y=/^key/,Z=/^(?:mouse|contextmenu)|click/,$=/^(?:focusinfocus|focusoutblur)$/,_=function(a){return p.event.special.hover?a:a.replace(X,"mouseenter$1 mouseleave$1")};p.event={add:function(a,c,d,e,f){var g,h,i,j,k,l,m,n,o,q,r;if(a.nodeType===3||a.nodeType===8||!c||!d||!(g=p._data(a)))return;d.handler&&(o=d,d=o.handler,f=o.selector),d.guid||(d.guid=p.guid++),i=g.events,i||(g.events=i={}),h=g.handle,h||(g.handle=h=function(a){return typeof p!="undefined"&&(!a||p.event.triggered!==a.type)?p.event.dispatch.apply(h.elem,argu
ments):b},h.elem=a),c=p.trim(_(c)).split(" ");for(j=0;j<c.length;j++){k=W.exec(c[j])||[],l=k[1],m=(k[2]||"").split(".").sort(),r=p.event.special[l]||{},l=(f?r.delegateType:r.bindType)||l,r=p.event.special[l]||{},n=p.extend({type:l,origType:k[1],data:e,handler:d,guid:d.guid,selector:f,namespace:m.join(".")},o),q=i[l];if(!q){q=i[l]=[],q.delegateCount=0;if(!r.setup||r.setup.call(a,e,m,h)===!1)a.addEventListener?a.addEventListener(l,h,!1):a.attachEvent&&a.attachEvent("on"+l,h)}r.add&&(r.add.call(a,n),n.handler.guid||(n.handler.guid=d.guid)),f?q.splice(q.delegateCount++,0,n):q.push(n),p.event.global[l]=!0}a=null},global:{},remove:function(a,b,c,d,e){var f,g,h,i,j,k,l,m,n,o,q,r=p.hasData(a)&&p._data(a);if(!r||!(m=r.events))return;b=p.trim(_(b||"")).split(" ");for(f=0;f<b.length;f++){g=W.exec(b[f])||[],h=i=g[1],j=g[2];if(!h){for(h in m)p.event.remove(a,h+b[f],c,d,!0);continue}n=p.event.special[h]||{},h=(d?n.delegateType:n.bindType)||h,o=m[h]||[],k=o.length,j=j?new RegExp("(^|\\.)"+j.split(
".").sort().join("\\.(?:.*\\.|)")+"(\\.|$)"):null;for(l=0;l<o.length;l++)q=o[l],(e||i===q.origType)&&(!c||c.guid===q.guid)&&(!j||j.test(q.namespace))&&(!d||d===q.selector||d==="**"&&q.selector)&&(o.splice(l--,1),q.selector&&o.delegateCount--,n.remove&&n.remove.call(a,q));o.length===0&&k!==o.length&&((!n.teardown||n.teardown.call(a,j,r.handle)===!1)&&p.removeEvent(a,h,r.handle),delete m[h])}p.isEmptyObject(m)&&(delete r.handle,p.removeData(a,"events",!0))},customEvent:{getData:!0,setData:!0,changeData:!0},trigger:function(c,d,f,g){if(!f||f.nodeType!==3&&f.nodeType!==8){var h,i,j,k,l,m,n,o,q,r,s=c.type||c,t=[];if($.test(s+p.event.triggered))return;s.indexOf("!")>=0&&(s=s.slice(0,-1),i=!0),s.indexOf(".")>=0&&(t=s.split("."),s=t.shift(),t.sort());if((!f||p.event.customEvent[s])&&!p.event.global[s])return;c=typeof c=="object"?c[p.expando]?c:new p.Event(s,c):new p.Event(s),c.type=s,c.isTrigger=!0,c.exclusive=i,c.namespace=t.join("."),c.namespace_re=c.namespace?new RegExp("(^|\\.)"+t.join(
"\\.(?:.*\\.|)")+"(\\.|$)"):null,m=s.indexOf(":")<0?"on"+s:"";if(!f){h=p.cache;for(j in h)h[j].events&&h[j].events[s]&&p.event.trigger(c,d,h[j].handle.elem,!0);return}c.result=b,c.target||(c.target=f),d=d!=null?p.makeArray(d):[],d.unshift(c),n=p.event.special[s]||{};if(n.trigger&&n.trigger.apply(f,d)===!1)return;q=[[f,n.bindType||s]];if(!g&&!n.noBubble&&!p.isWindow(f)){r=n.delegateType||s,k=$.test(r+s)?f:f.parentNode;for(l=f;k;k=k.parentNode)q.push([k,r]),l=k;l===(f.ownerDocument||e)&&q.push([l.defaultView||l.parentWindow||a,r])}for(j=0;j<q.length&&!c.isPropagationStopped();j++)k=q[j][0],c.type=q[j][1],o=(p._data(k,"events")||{})[c.type]&&p._data(k,"handle"),o&&o.apply(k,d),o=m&&k[m],o&&p.acceptData(k)&&o.apply(k,d)===!1&&c.preventDefault();return c.type=s,!g&&!c.isDefaultPrevented()&&(!n._default||n._default.apply(f.ownerDocument,d)===!1)&&(s!=="click"||!p.nodeName(f,"a"))&&p.acceptData(f)&&m&&f[s]&&(s!=="focus"&&s!=="blur"||c.target.offsetWidth!==0)&&!p.isWindow(f)&&(l=f[m],l&&(f[
m]=null),p.event.triggered=s,f[s](),p.event.triggered=b,l&&(f[m]=l)),c.result}return},dispatch:function(c){c=p.event.fix(c||a.event);var d,e,f,g,h,i,j,k,l,m,n,o=(p._data(this,"events")||{})[c.type]||[],q=o.delegateCount,r=[].slice.call(arguments),s=!c.exclusive&&!c.namespace,t=p.event.special[c.type]||{},u=[];r[0]=c,c.delegateTarget=this;if(t.preDispatch&&t.preDispatch.call(this,c)===!1)return;if(q&&(!c.button||c.type!=="click")){g=p(this),g.context=this;for(f=c.target;f!=this;f=f.parentNode||this)if(f.disabled!==!0||c.type!=="click"){i={},k=[],g[0]=f;for(d=0;d<q;d++)l=o[d],m=l.selector,i[m]===b&&(i[m]=g.is(m)),i[m]&&k.push(l);k.length&&u.push({elem:f,matches:k})}}o.length>q&&u.push({elem:this,matches:o.slice(q)});for(d=0;d<u.length&&!c.isPropagationStopped();d++){j=u[d],c.currentTarget=j.elem;for(e=0;e<j.matches.length&&!c.isImmediatePropagationStopped();e++){l=j.matches[e];if(s||!c.namespace&&!l.namespace||c.namespace_re&&c.namespace_re.test(l.namespace))c.data=l.data,c.handleObj=
l,h=((p.event.special[l.origType]||{}).handle||l.handler).apply(j.elem,r),h!==b&&(c.result=h,h===!1&&(c.preventDefault(),c.stopPropagation()))}}return t.postDispatch&&t.postDispatch.call(this,c),c.result},props:"attrChange attrName relatedNode srcElement altKey bubbles cancelable ctrlKey currentTarget eventPhase metaKey relatedTarget shiftKey target timeStamp view which".split(" "),fixHooks:{},keyHooks:{props:"char charCode key keyCode".split(" "),filter:function(a,b){return a.which==null&&(a.which=b.charCode!=null?b.charCode:b.keyCode),a}},mouseHooks:{props:"button buttons clientX clientY fromElement offsetX offsetY pageX pageY screenX screenY toElement".split(" "),filter:function(a,c){var d,f,g,h=c.button,i=c.fromElement;return a.pageX==null&&c.clientX!=null&&(d=a.target.ownerDocument||e,f=d.documentElement,g=d.body,a.pageX=c.clientX+(f&&f.scrollLeft||g&&g.scrollLeft||0)-(f&&f.clientLeft||g&&g.clientLeft||0),a.pageY=c.clientY+(f&&f.scrollTop||g&&g.scrollTop||0)-(f&&f.clientTop||g&
&g.clientTop||0)),!a.relatedTarget&&i&&(a.relatedTarget=i===a.target?c.toElement:i),!a.which&&h!==b&&(a.which=h&1?1:h&2?3:h&4?2:0),a}},fix:function(a){if(a[p.expando])return a;var b,c,d=a,f=p.event.fixHooks[a.type]||{},g=f.props?this.props.concat(f.props):this.props;a=p.Event(d);for(b=g.length;b;)c=g[--b],a[c]=d[c];return a.target||(a.target=d.srcElement||e),a.target.nodeType===3&&(a.target=a.target.parentNode),a.metaKey=!!a.metaKey,f.filter?f.filter(a,d):a},special:{ready:{setup:p.bindReady},load:{noBubble:!0},focus:{delegateType:"focusin"},blur:{delegateType:"focusout"},beforeunload:{setup:function(a,b,c){p.isWindow(this)&&(this.onbeforeunload=c)},teardown:function(a,b){this.onbeforeunload===b&&(this.onbeforeunload=null)}}},simulate:function(a,b,c,d){var e=p.extend(new p.Event,c,{type:a,isSimulated:!0,originalEvent:{}});d?p.event.trigger(e,null,b):p.event.dispatch.call(b,e),e.isDefaultPrevented()&&c.preventDefault()}},p.event.handle=p.event.dispatch,p.removeEvent=e.removeEventList
ener?function(a,b,c){a.removeEventListener&&a.removeEventListener(b,c,!1)}:function(a,b,c){var d="on"+b;a.detachEvent&&(typeof a[d]=="undefined"&&(a[d]=null),a.detachEvent(d,c))},p.Event=function(a,b){if(this instanceof p.Event)a&&a.type?(this.originalEvent=a,this.type=a.type,this.isDefaultPrevented=a.defaultPrevented||a.returnValue===!1||a.getPreventDefault&&a.getPreventDefault()?bb:ba):this.type=a,b&&p.extend(this,b),this.timeStamp=a&&a.timeStamp||p.now(),this[p.expando]=!0;else return new p.Event(a,b)},p.Event.prototype={preventDefault:function(){this.isDefaultPrevented=bb;var a=this.originalEvent;if(!a)return;a.preventDefault?a.preventDefault():a.returnValue=!1},stopPropagation:function(){this.isPropagationStopped=bb;var a=this.originalEvent;if(!a)return;a.stopPropagation&&a.stopPropagation(),a.cancelBubble=!0},stopImmediatePropagation:function(){this.isImmediatePropagationStopped=bb,this.stopPropagation()},isDefaultPrevented:ba,isPropagationStopped:ba,isImmediatePropagationStop
ped:ba},p.each({mouseenter:"mouseover",mouseleave:"mouseout"},function(a,b){p.event.special[a]={delegateType:b,bindType:b,handle:function(a){var c,d=this,e=a.relatedTarget,f=a.handleObj,g=f.selector;if(!e||e!==d&&!p.contains(d,e))a.type=f.origType,c=f.handler.apply(this,arguments),a.type=b;return c}}}),p.support.submitBubbles||(p.event.special.submit={setup:function(){if(p.nodeName(this,"form"))return!1;p.event.add(this,"click._submit keypress._submit",function(a){var c=a.target,d=p.nodeName(c,"input")||p.nodeName(c,"button")?c.form:b;d&&!p._data(d,"_submit_attached")&&(p.event.add(d,"submit._submit",function(a){a._submit_bubble=!0}),p._data(d,"_submit_attached",!0))})},postDispatch:function(a){a._submit_bubble&&(delete a._submit_bubble,this.parentNode&&!a.isTrigger&&p.event.simulate("submit",this.parentNode,a,!0))},teardown:function(){if(p.nodeName(this,"form"))return!1;p.event.remove(this,"._submit")}}),p.support.changeBubbles||(p.event.special.change={setup:function(){if(V.test(t
his.nodeName)){if(this.type==="checkbox"||this.type==="radio")p.event.add(this,"propertychange._change",function(a){a.originalEvent.propertyName==="checked"&&(this._just_changed=!0)}),p.event.add(this,"click._change",function(a){this._just_changed&&!a.isTrigger&&(this._just_changed=!1),p.event.simulate("change",this,a,!0)});return!1}p.event.add(this,"beforeactivate._change",function(a){var b=a.target;V.test(b.nodeName)&&!p._data(b,"_change_attached")&&(p.event.add(b,"change._change",function(a){this.parentNode&&!a.isSimulated&&!a.isTrigger&&p.event.simulate("change",this.parentNode,a,!0)}),p._data(b,"_change_attached",!0))})},handle:function(a){var b=a.target;if(this!==b||a.isSimulated||a.isTrigger||b.type!=="radio"&&b.type!=="checkbox")return a.handleObj.handler.apply(this,arguments)},teardown:function(){return p.event.remove(this,"._change"),V.test(this.nodeName)}}),p.support.focusinBubbles||p.each({focus:"focusin",blur:"focusout"},function(a,b){var c=0,d=function(a){p.event.simul
ate(b,a.target,p.event.fix(a),!0)};p.event.special[b]={setup:function(){c++===0&&e.addEventListener(a,d,!0)},teardown:function(){--c===0&&e.removeEventListener(a,d,!0)}}}),p.fn.extend({on:function(a,c,d,e,f){var g,h;if(typeof a=="object"){typeof c!="string"&&(d=d||c,c=b);for(h in a)this.on(h,c,d,a[h],f);return this}d==null&&e==null?(e=c,d=c=b):e==null&&(typeof c=="string"?(e=d,d=b):(e=d,d=c,c=b));if(e===!1)e=ba;else if(!e)return this;return f===1&&(g=e,e=function(a){return p().off(a),g.apply(this,arguments)},e.guid=g.guid||(g.guid=p.guid++)),this.each(function(){p.event.add(this,a,e,d,c)})},one:function(a,b,c,d){return this.on(a,b,c,d,1)},off:function(a,c,d){var e,f;if(a&&a.preventDefault&&a.handleObj)return e=a.handleObj,p(a.delegateTarget).off(e.namespace?e.origType+"."+e.namespace:e.origType,e.selector,e.handler),this;if(typeof a=="object"){for(f in a)this.off(f,c,a[f]);return this}if(c===!1||typeof c=="function")d=c,c=b;return d===!1&&(d=ba),this.each(function(){p.event.remove(t
his,a,d,c)})},bind:function(a,b,c){return this.on(a,null,b,c)},unbind:function(a,b){return this.off(a,null,b)},live:function(a,b,c){return p(this.context).on(a,this.selector,b,c),this},die:function(a,b){return p(this.context).off(a,this.selector||"**",b),this},delegate:function(a,b,c,d){return this.on(b,a,c,d)},undelegate:function(a,b,c){return arguments.length==1?this.off(a,"**"):this.off(b,a||"**",c)},trigger:function(a,b){return this.each(function(){p.event.trigger(a,b,this)})},triggerHandler:function(a,b){if(this[0])return p.event.trigger(a,b,this[0],!0)},toggle:function(a){var b=arguments,c=a.guid||p.guid++,d=0,e=function(c){var e=(p._data(this,"lastToggle"+a.guid)||0)%d;return p._data(this,"lastToggle"+a.guid,e+1),c.preventDefault(),b[e].apply(this,arguments)||!1};e.guid=c;while(d<b.length)b[d++].guid=c;return this.click(e)},hover:function(a,b){return this.mouseenter(a).mouseleave(b||a)}}),p.each("blur focus focusin focusout load resize scroll unload click dblclick mousedown m
ouseup mousemove mouseover mouseout mouseenter mouseleave change select submit keydown keypress keyup error contextmenu".split(" "),function(a,b){p.fn[b]=function(a,c){return c==null&&(c=a,a=null),arguments.length>0?this.on(b,null,a,c):this.trigger(b)},Y.test(b)&&(p.event.fixHooks[b]=p.event.keyHooks),Z.test(b)&&(p.event.fixHooks[b]=p.event.mouseHooks)}),function(a,b){function bd(a,b,c,d){var e=0,f=b.length;for(;e<f;e++)Z(a,b[e],c,d)}function be(a,b,c,d,e,f){var g,h=$.setFilters[b.toLowerCase()];return h||Z.error(b),(a||!(g=e))&&bd(a||"*",d,g=[],e),g.length>0?h(g,c,f):[]}function bf(a,c,d,e,f){var g,h,i,j,k,l,m,n,p=0,q=f.length,s=L.POS,t=new RegExp("^"+s.source+"(?!"+r+")","i"),u=function(){var a=1,c=arguments.length-2;for(;a<c;a++)arguments[a]===b&&(g[a]=b)};for(;p<q;p++){s.exec(""),a=f[p],j=[],i=0,k=e;while(g=s.exec(a)){n=s.lastIndex=g.index+g[0].length;if(n>i){m=a.slice(i,g.index),i=n,l=[c],B.test(m)&&(k&&(l=k),k=e);if(h=H.test(m))m=m.slice(0,-5).replace(B,"$&*");g.length>1&&g[0]
.replace(t,u),k=be(m,g[1],g[2],l,k,h)}}k?(j=j.concat(k),(m=a.slice(i))&&m!==")"?B.test(m)?bd(m,j,d,e):Z(m,c,d,e?e.concat(k):k):o.apply(d,j)):Z(a,c,d,e)}return q===1?d:Z.uniqueSort(d)}function bg(a,b,c){var d,e,f,g=[],i=0,j=D.exec(a),k=!j.pop()&&!j.pop(),l=k&&a.match(C)||[""],m=$.preFilter,n=$.filter,o=!c&&b!==h;for(;(e=l[i])!=null&&k;i++){g.push(d=[]),o&&(e=" "+e);while(e){k=!1;if(j=B.exec(e))e=e.slice(j[0].length),k=d.push({part:j.pop().replace(A," "),captures:j});for(f in n)(j=L[f].exec(e))&&(!m[f]||(j=m[f](j,b,c)))&&(e=e.slice(j.shift().length),k=d.push({part:f,captures:j}));if(!k)break}}return k||Z.error(a),g}function bh(a,b,e){var f=b.dir,g=m++;return a||(a=function(a){return a===e}),b.first?function(b,c){while(b=b[f])if(b.nodeType===1)return a(b,c)&&b}:function(b,e){var h,i=g+"."+d,j=i+"."+c;while(b=b[f])if(b.nodeType===1){if((h=b[q])===j)return b.sizset;if(typeof h=="string"&&h.indexOf(i)===0){if(b.sizset)return b}else{b[q]=j;if(a(b,e))return b.sizset=!0,b;b.sizset=!1}}}}func
tion bi(a,b){return a?function(c,d){var e=b(c,d);return e&&a(e===!0?c:e,d)}:b}function bj(a,b,c){var d,e,f=0;for(;d=a[f];f++)$.relative[d.part]?e=bh(e,$.relative[d.part],b):(d.captures.push(b,c),e=bi(e,$.filter[d.part].apply(null,d.captures)));return e}function bk(a){return function(b,c){var d,e=0;for(;d=a[e];e++)if(d(b,c))return!0;return!1}}var c,d,e,f,g,h=a.document,i=h.documentElement,j="undefined",k=!1,l=!0,m=0,n=[].slice,o=[].push,q=("sizcache"+Math.random()).replace(".",""),r="[\\x20\\t\\r\\n\\f]",s="(?:\\\\.|[-\\w]|[^\\x00-\\xa0])+",t=s.replace("w","w#"),u="([*^$|!~]?=)",v="\\["+r+"*("+s+")"+r+"*(?:"+u+r+"*(?:(['\"])((?:\\\\.|[^\\\\])*?)\\3|("+t+")|)|)"+r+"*\\]",w=":("+s+")(?:\\((?:(['\"])((?:\\\\.|[^\\\\])*?)\\2|((?:[^,]|\\\\,|(?:,(?=[^\\[]*\\]))|(?:,(?=[^\\(]*\\))))*))\\)|)",x=":(nth|eq|gt|lt|first|last|even|odd)(?:\\((\\d*)\\)|)(?=[^-]|$)",y=r+"*([\\x20\\t\\r\\n\\f>+~])"+r+"*",z="(?=[^\\x20\\t\\r\\n\\f])(?:\\\\.|"+v+"|"+w.replace(2,7)+"|[^\\\\(),])+",A=new RegExp("^"+r+"+|
((?:^|[^\\\\])(?:\\\\.)*)"+r+"+$","g"),B=new RegExp("^"+y),C=new RegExp(z+"?(?="+r+"*,|$)","g"),D=new RegExp("^(?:(?!,)(?:(?:^|,)"+r+"*"+z+")*?|"+r+"*(.*?))(\\)|$)"),E=new RegExp(z.slice(19,-6)+"\\x20\\t\\r\\n\\f>+~])+|"+y,"g"),F=/^(?:#([\w\-]+)|(\w+)|\.([\w\-]+))$/,G=/[\x20\t\r\n\f]*[+~]/,H=/:not\($/,I=/h\d/i,J=/input|select|textarea|button/i,K=/\\(?!\\)/g,L={ID:new RegExp("^#("+s+")"),CLASS:new RegExp("^\\.("+s+")"),NAME:new RegExp("^\\[name=['\"]?("+s+")['\"]?\\]"),TAG:new RegExp("^("+s.replace("[-","[-\\*")+")"),ATTR:new RegExp("^"+v),PSEUDO:new RegExp("^"+w),CHILD:new RegExp("^:(only|nth|last|first)-child(?:\\("+r+"*(even|odd|(([+-]|)(\\d*)n|)"+r+"*(?:([+-]|)"+r+"*(\\d+)|))"+r+"*\\)|)","i"),POS:new RegExp(x,"ig"),needsContext:new RegExp("^"+r+"*[>+~]|"+x,"i")},M={},N=[],O={},P=[],Q=function(a){return a.sizzleFilter=!0,a},R=function(a){return function(b){return b.nodeName.toLowerCase()==="input"&&b.type===a}},S=function(a){return function(b){var c=b.nodeName.toLowerCase();return
(c==="input"||c==="button")&&b.type===a}},T=function(a){var b=!1,c=h.createElement("div");try{b=a(c)}catch(d){}return c=null,b},U=T(function(a){a.innerHTML="<select></select>";var b=typeof a.lastChild.getAttribute("multiple");return b!=="boolean"&&b!=="string"}),V=T(function(a){a.id=q+0,a.innerHTML="<a name='"+q+"'></a><div name='"+q+"'></div>",i.insertBefore(a,i.firstChild);var b=h.getElementsByName&&h.getElementsByName(q).length===2+h.getElementsByName(q+0).length;return g=!h.getElementById(q),i.removeChild(a),b}),W=T(function(a){return a.appendChild(h.createComment("")),a.getElementsByTagName("*").length===0}),X=T(function(a){return a.innerHTML="<a href='#'></a>",a.firstChild&&typeof a.firstChild.getAttribute!==j&&a.firstChild.getAttribute("href")==="#"}),Y=T(function(a){return a.innerHTML="<div class='hidden e'></div><div class='hidden'></div>",!a.getElementsByClassName||a.getElementsByClassName("e").length===0?!1:(a.lastChild.className="e",a.getElementsByClassName("e").length!=
=1)}),Z=function(a,b,c,d){c=c||[],b=b||h;var e,f,g,i,j=b.nodeType;if(j!==1&&j!==9)return[];if(!a||typeof a!="string")return c;g=ba(b);if(!g&&!d)if(e=F.exec(a))if(i=e[1]){if(j===9){f=b.getElementById(i);if(!f||!f.parentNode)return c;if(f.id===i)return c.push(f),c}else if(b.ownerDocument&&(f=b.ownerDocument.getElementById(i))&&bb(b,f)&&f.id===i)return c.push(f),c}else{if(e[2])return o.apply(c,n.call(b.getElementsByTagName(a),0)),c;if((i=e[3])&&Y&&b.getElementsByClassName)return o.apply(c,n.call(b.getElementsByClassName(i),0)),c}return bm(a,b,c,d,g)},$=Z.selectors={cacheLength:50,match:L,order:["ID","TAG"],attrHandle:{},createPseudo:Q,find:{ID:g?function(a,b,c){if(typeof b.getElementById!==j&&!c){var d=b.getElementById(a);return d&&d.parentNode?[d]:[]}}:function(a,c,d){if(typeof c.getElementById!==j&&!d){var e=c.getElementById(a);return e?e.id===a||typeof e.getAttributeNode!==j&&e.getAttributeNode("id").value===a?[e]:b:[]}},TAG:W?function(a,b){if(typeof b.getElementsByTagName!==j)retur
n b.getElementsByTagName(a)}:function(a,b){var c=b.getElementsByTagName(a);if(a==="*"){var d,e=[],f=0;for(;d=c[f];f++)d.nodeType===1&&e.push(d);return e}return c}},relative:{">":{dir:"parentNode",first:!0}," ":{dir:"parentNode"},"+":{dir:"previousSibling",first:!0},"~":{dir:"previousSibling"}},preFilter:{ATTR:function(a){return a[1]=a[1].replace(K,""),a[3]=(a[4]||a[5]||"").replace(K,""),a[2]==="~="&&(a[3]=" "+a[3]+" "),a.slice(0,4)},CHILD:function(a){return a[1]=a[1].toLowerCase(),a[1]==="nth"?(a[2]||Z.error(a[0]),a[3]=+(a[3]?a[4]+(a[5]||1):2*(a[2]==="even"||a[2]==="odd")),a[4]=+(a[6]+a[7]||a[2]==="odd")):a[2]&&Z.error(a[0]),a},PSEUDO:function(a){var b,c=a[4];return L.CHILD.test(a[0])?null:(c&&(b=D.exec(c))&&b.pop()&&(a[0]=a[0].slice(0,b[0].length-c.length-1),c=b[0].slice(0,-1)),a.splice(2,3,c||a[3]),a)}},filter:{ID:g?function(a){return a=a.replace(K,""),function(b){return b.getAttribute("id")===a}}:function(a){return a=a.replace(K,""),function(b){var c=typeof b.getAttributeNode!==j
&&b.getAttributeNode("id");return c&&c.value===a}},TAG:function(a){return a==="*"?function(){return!0}:(a=a.replace(K,"").toLowerCase(),function(b){return b.nodeName&&b.nodeName.toLowerCase()===a})},CLASS:function(a){var b=M[a];return b||(b=M[a]=new RegExp("(^|"+r+")"+a+"("+r+"|$)"),N.push(a),N.length>$.cacheLength&&delete M[N.shift()]),function(a){return b.test(a.className||typeof a.getAttribute!==j&&a.getAttribute("class")||"")}},ATTR:function(a,b,c){return b?function(d){var e=Z.attr(d,a),f=e+"";if(e==null)return b==="!=";switch(b){case"=":return f===c;case"!=":return f!==c;case"^=":return c&&f.indexOf(c)===0;case"*=":return c&&f.indexOf(c)>-1;case"$=":return c&&f.substr(f.length-c.length)===c;case"~=":return(" "+f+" ").indexOf(c)>-1;case"|=":return f===c||f.substr(0,c.length+1)===c+"-"}}:function(b){return Z.attr(b,a)!=null}},CHILD:function(a,b,c,d){if(a==="nth"){var e=m++;return function(a){var b,f,g=0,h=a;if(c===1&&d===0)return!0;b=a.parentNode;if(b&&(b[q]!==e||!a.sizset)){for(
h=b.firstChild;h;h=h.nextSibling)if(h.nodeType===1){h.sizset=++g;if(h===a)break}b[q]=e}return f=a.sizset-d,c===0?f===0:f%c===0&&f/c>=0}}return function(b){var c=b;switch(a){case"only":case"first":while(c=c.previousSibling)if(c.nodeType===1)return!1;if(a==="first")return!0;c=b;case"last":while(c=c.nextSibling)if(c.nodeType===1)return!1;return!0}}},PSEUDO:function(a,b,c,d){var e=$.pseudos[a]||$.pseudos[a.toLowerCase()];return e||Z.error("unsupported pseudo: "+a),e.sizzleFilter?e(b,c,d):e}},pseudos:{not:Q(function(a,b,c){var d=bl(a.replace(A,"$1"),b,c);return function(a){return!d(a)}}),enabled:function(a){return a.disabled===!1},disabled:function(a){return a.disabled===!0},checked:function(a){var b=a.nodeName.toLowerCase();return b==="input"&&!!a.checked||b==="option"&&!!a.selected},selected:function(a){return a.parentNode&&a.parentNode.selectedIndex,a.selected===!0},parent:function(a){return!$.pseudos.empty(a)},empty:function(a){var b;a=a.firstChild;while(a){if(a.nodeName>"@"||(b=a.no
deType)===3||b===4)return!1;a=a.nextSibling}return!0},contains:Q(function(a){return function(b){return(b.textContent||b.innerText||bc(b)).indexOf(a)>-1}}),has:Q(function(a){return function(b){return Z(a,b).length>0}}),header:function(a){return I.test(a.nodeName)},text:function(a){var b,c;return a.nodeName.toLowerCase()==="input"&&(b=a.type)==="text"&&((c=a.getAttribute("type"))==null||c.toLowerCase()===b)},radio:R("radio"),checkbox:R("checkbox"),file:R("file"),password:R("password"),image:R("image"),submit:S("submit"),reset:S("reset"),button:function(a){var b=a.nodeName.toLowerCase();return b==="input"&&a.type==="button"||b==="button"},input:function(a){return J.test(a.nodeName)},focus:function(a){var b=a.ownerDocument;return a===b.activeElement&&(!b.hasFocus||b.hasFocus())&&(!!a.type||!!a.href)},active:function(a){return a===a.ownerDocument.activeElement}},setFilters:{first:function(a,b,c){return c?a.slice(1):[a[0]]},last:function(a,b,c){var d=a.pop();return c?a:[d]},even:function(
a,b,c){var d=[],e=c?1:0,f=a.length;for(;e<f;e=e+2)d.push(a[e]);return d},odd:function(a,b,c){var d=[],e=c?0:1,f=a.length;for(;e<f;e=e+2)d.push(a[e]);return d},lt:function(a,b,c){return c?a.slice(+b):a.slice(0,+b)},gt:function(a,b,c){return c?a.slice(0,+b+1):a.slice(+b+1)},eq:function(a,b,c){var d=a.splice(+b,1);return c?a:d}}};$.setFilters.nth=$.setFilters.eq,$.filters=$.pseudos,X||($.attrHandle={href:function(a){return a.getAttribute("href",2)},type:function(a){return a.getAttribute("type")}}),V&&($.order.push("NAME"),$.find.NAME=function(a,b){if(typeof b.getElementsByName!==j)return b.getElementsByName(a)}),Y&&($.order.splice(1,0,"CLASS"),$.find.CLASS=function(a,b,c){if(typeof b.getElementsByClassName!==j&&!c)return b.getElementsByClassName(a)});try{n.call(i.childNodes,0)[0].nodeType}catch(_){n=function(a){var b,c=[];for(;b=this[a];a++)c.push(b);return c}}var ba=Z.isXML=function(a){var b=a&&(a.ownerDocument||a).documentElement;return b?b.nodeName!=="HTML":!1},bb=Z.contains=i.compa
reDocumentPosition?function(a,b){return!!(a.compareDocumentPosition(b)&16)}:i.contains?function(a,b){var c=a.nodeType===9?a.documentElement:a,d=b.parentNode;return a===d||!!(d&&d.nodeType===1&&c.contains&&c.contains(d))}:function(a,b){while(b=b.parentNode)if(b===a)return!0;return!1},bc=Z.getText=function(a){var b,c="",d=0,e=a.nodeType;if(e){if(e===1||e===9||e===11){if(typeof a.textContent=="string")return a.textContent;for(a=a.firstChild;a;a=a.nextSibling)c+=bc(a)}else if(e===3||e===4)return a.nodeValue}else for(;b=a[d];d++)c+=bc(b);return c};Z.attr=function(a,b){var c,d=ba(a);return d||(b=b.toLowerCase()),$.attrHandle[b]?$.attrHandle[b](a):U||d?a.getAttribute(b):(c=a.getAttributeNode(b),c?typeof a[b]=="boolean"?a[b]?b:null:c.specified?c.value:null:null)},Z.error=function(a){throw new Error("Syntax error, unrecognized expression: "+a)},[0,0].sort(function(){return l=0}),i.compareDocumentPosition?e=function(a,b){return a===b?(k=!0,0):(!a.compareDocumentPosition||!b.compareDocumentPos
ition?a.compareDocumentPosition:a.compareDocumentPosition(b)&4)?-1:1}:(e=function(a,b){if(a===b)return k=!0,0;if(a.sourceIndex&&b.sourceIndex)return a.sourceIndex-b.sourceIndex;var c,d,e=[],g=[],h=a.parentNode,i=b.parentNode,j=h;if(h===i)return f(a,b);if(!h)return-1;if(!i)return 1;while(j)e.unshift(j),j=j.parentNode;j=i;while(j)g.unshift(j),j=j.parentNode;c=e.length,d=g.length;for(var l=0;l<c&&l<d;l++)if(e[l]!==g[l])return f(e[l],g[l]);return l===c?f(a,g[l],-1):f(e[l],b,1)},f=function(a,b,c){if(a===b)return c;var d=a.nextSibling;while(d){if(d===b)return-1;d=d.nextSibling}return 1}),Z.uniqueSort=function(a){var b,c=1;if(e){k=l,a.sort(e);if(k)for(;b=a[c];c++)b===a[c-1]&&a.splice(c--,1)}return a};var bl=Z.compile=function(a,b,c){var d,e,f,g=O[a];if(g&&g.context===b)return g;e=bg(a,b,c);for(f=0;d=e[f];f++)e[f]=bj(d,b,c);return g=O[a]=bk(e),g.context=b,g.runs=g.dirruns=0,P.push(a),P.length>$.cacheLength&&delete O[P.shift()],g};Z.matches=function(a,b){return Z(a,null,null,b)},Z.matchesSel
ector=function(a,b){return Z(b,null,null,[a]).length>0};var bm=function(a,b,e,f,g){a=a.replace(A,"$1");var h,i,j,k,l,m,p,q,r,s=a.match(C),t=a.match(E),u=b.nodeType;if(L.POS.test(a))return bf(a,b,e,f,s);if(f)h=n.call(f,0);else if(s&&s.length===1){if(t.length>1&&u===9&&!g&&(s=L.ID.exec(t[0]))){b=$.find.ID(s[1],b,g)[0];if(!b)return e;a=a.slice(t.shift().length)}q=(s=G.exec(t[0]))&&!s.index&&b.parentNode||b,r=t.pop(),m=r.split(":not")[0];for(j=0,k=$.order.length;j<k;j++){p=$.order[j];if(s=L[p].exec(m)){h=$.find[p]((s[1]||"").replace(K,""),q,g);if(h==null)continue;m===r&&(a=a.slice(0,a.length-r.length)+m.replace(L[p],""),a||o.apply(e,n.call(h,0)));break}}}if(a){i=bl(a,b,g),d=i.dirruns++,h==null&&(h=$.find.TAG("*",G.test(a)&&b.parentNode||b));for(j=0;l=h[j];j++)c=i.runs++,i(l,b)&&e.push(l)}return e};h.querySelectorAll&&function(){var a,b=bm,c=/'|\\/g,d=/\=[\x20\t\r\n\f]*([^'"\]]*)[\x20\t\r\n\f]*\]/g,e=[],f=[":active"],g=i.matchesSelector||i.mozMatchesSelector||i.webkitMatchesSelector||i.o
MatchesSelector||i.msMatchesSelector;T(function(a){a.innerHTML="<select><option selected></option></select>",a.querySelectorAll("[selected]").length||e.push("\\["+r+"*(?:checked|disabled|ismap|multiple|readonly|selected|value)"),a.querySelectorAll(":checked").length||e.push(":checked")}),T(function(a){a.innerHTML="<p test=''></p>",a.querySelectorAll("[test^='']").length&&e.push("[*^$]="+r+"*(?:\"\"|'')"),a.innerHTML="<input type='hidden'>",a.querySelectorAll(":enabled").length||e.push(":enabled",":disabled")}),e=e.length&&new RegExp(e.join("|")),bm=function(a,d,f,g,h){if(!g&&!h&&(!e||!e.test(a)))if(d.nodeType===9)try{return o.apply(f,n.call(d.querySelectorAll(a),0)),f}catch(i){}else if(d.nodeType===1&&d.nodeName.toLowerCase()!=="object"){var j=d.getAttribute("id"),k=j||q,l=G.test(a)&&d.parentNode||d;j?k=k.replace(c,"\\$&"):d.setAttribute("id",k);try{return o.apply(f,n.call(l.querySelectorAll(a.replace(C,"[id='"+k+"'] $&")),0)),f}catch(i){}finally{j||d.removeAttribute("id")}}return b
(a,d,f,g,h)},g&&(T(function(b){a=g.call(b,"div");try{g.call(b,"[test!='']:sizzle"),f.push($.match.PSEUDO)}catch(c){}}),f=new RegExp(f.join("|")),Z.matchesSelector=function(b,c){c=c.replace(d,"='$1']");if(!ba(b)&&!f.test(c)&&(!e||!e.test(c)))try{var h=g.call(b,c);if(h||a||b.document&&b.document.nodeType!==11)return h}catch(i){}return Z(c,null,null,[b]).length>0})}(),Z.attr=p.attr,p.find=Z,p.expr=Z.selectors,p.expr[":"]=p.expr.pseudos,p.unique=Z.uniqueSort,p.text=Z.getText,p.isXMLDoc=Z.isXML,p.contains=Z.contains}(a);var bc=/Until$/,bd=/^(?:parents|prev(?:Until|All))/,be=/^.[^:#\[\.,]*$/,bf=p.expr.match.needsContext,bg={children:!0,contents:!0,next:!0,prev:!0};p.fn.extend({find:function(a){var b,c,d,e,f,g,h=this;if(typeof a!="string")return p(a).filter(function(){for(b=0,c=h.length;b<c;b++)if(p.contains(h[b],this))return!0});g=this.pushStack("","find",a);for(b=0,c=this.length;b<c;b++){d=g.length,p.find(a,this[b],g);if(b>0)for(e=d;e<g.length;e++)for(f=0;f<d;f++)if(g[f]===g[e]){g.splice
(e--,1);break}}return g},has:function(a){var b,c=p(a,this),d=c.length;return this.filter(function(){for(b=0;b<d;b++)if(p.contains(this,c[b]))return!0})},not:function(a){return this.pushStack(bj(this,a,!1),"not",a)},filter:function(a){return this.pushStack(bj(this,a,!0),"filter",a)},is:function(a){return!!a&&(typeof a=="string"?bf.test(a)?p(a,this.context).index(this[0])>=0:p.filter(a,this).length>0:this.filter(a).length>0)},closest:function(a,b){var c,d=0,e=this.length,f=[],g=bf.test(a)||typeof a!="string"?p(a,b||this.context):0;for(;d<e;d++){c=this[d];while(c&&c.ownerDocument&&c!==b&&c.nodeType!==11){if(g?g.index(c)>-1:p.find.matchesSelector(c,a)){f.push(c);break}c=c.parentNode}}return f=f.length>1?p.unique(f):f,this.pushStack(f,"closest",a)},index:function(a){return a?typeof a=="string"?p.inArray(this[0],p(a)):p.inArray(a.jquery?a[0]:a,this):this[0]&&this[0].parentNode?this.prevAll().length:-1},add:function(a,b){var c=typeof a=="string"?p(a,b):p.makeArray(a&&a.nodeType?[a]:a),d=p.
merge(this.get(),c);return this.pushStack(bh(c[0])||bh(d[0])?d:p.unique(d))},addBack:function(a){return this.add(a==null?this.prevObject:this.prevObject.filter(a))}}),p.fn.andSelf=p.fn.addBack,p.each({parent:function(a){var b=a.parentNode;return b&&b.nodeType!==11?b:null},parents:function(a){return p.dir(a,"parentNode")},parentsUntil:function(a,b,c){return p.dir(a,"parentNode",c)},next:function(a){return bi(a,"nextSibling")},prev:function(a){return bi(a,"previousSibling")},nextAll:function(a){return p.dir(a,"nextSibling")},prevAll:function(a){return p.dir(a,"previousSibling")},nextUntil:function(a,b,c){return p.dir(a,"nextSibling",c)},prevUntil:function(a,b,c){return p.dir(a,"previousSibling",c)},siblings:function(a){return p.sibling((a.parentNode||{}).firstChild,a)},children:function(a){return p.sibling(a.firstChild)},contents:function(a){return p.nodeName(a,"iframe")?a.contentDocument||a.contentWindow.document:p.merge([],a.childNodes)}},function(a,b){p.fn[a]=function(c,d){var e=p.
map(this,b,c);return bc.test(a)||(d=c),d&&typeof d=="string"&&(e=p.filter(d,e)),e=this.length>1&&!bg[a]?p.unique(e):e,this.length>1&&bd.test(a)&&(e=e.reverse()),this.pushStack(e,a,k.call(arguments).join(","))}}),p.extend({filter:function(a,b,c){return c&&(a=":not("+a+")"),b.length===1?p.find.matchesSelector(b[0],a)?[b[0]]:[]:p.find.matches(a,b)},dir:function(a,c,d){var e=[],f=a[c];while(f&&f.nodeType!==9&&(d===b||f.nodeType!==1||!p(f).is(d)))f.nodeType===1&&e.push(f),f=f[c];return e},sibling:function(a,b){var c=[];for(;a;a=a.nextSibling)a.nodeType===1&&a!==b&&c.push(a);return c}});var bl="abbr|article|aside|audio|bdi|canvas|data|datalist|details|figcaption|figure|footer|header|hgroup|mark|meter|nav|output|progress|section|summary|time|video",bm=/ jQuery\d+="(?:null|\d+)"/g,bn=/^\s+/,bo=/<(?!area|br|col|embed|hr|img|input|link|meta|param)(([\w:]+)[^>]*)\/>/gi,bp=/<([\w:]+)/,bq=/<tbody/i,br=/<|&#?\w+;/,bs=/<(?:script|style|link)/i,bt=/<(?:script|object|embed|option|style)/i,bu=new Reg
Exp("<(?:"+bl+")[\\s/>]","i"),bv=/^(?:checkbox|radio)$/,bw=/checked\s*(?:[^=]|=\s*.checked.)/i,bx=/\/(java|ecma)script/i,by=/^\s*<!(?:\[CDATA\[|\-\-)|[\]\-]{2}>\s*$/g,bz={option:[1,"<select multiple='multiple'>","</select>"],legend:[1,"<fieldset>","</fieldset>"],thead:[1,"<table>","</table>"],tr:[2,"<table><tbody>","</tbody></table>"],td:[3,"<table><tbody><tr>","</tr></tbody></table>"],col:[2,"<table><tbody></tbody><colgroup>","</colgroup></table>"],area:[1,"<map>","</map>"],_default:[0,"",""]},bA=bk(e),bB=bA.appendChild(e.createElement("div"));bz.optgroup=bz.option,bz.tbody=bz.tfoot=bz.colgroup=bz.caption=bz.thead,bz.th=bz.td,p.support.htmlSerialize||(bz._default=[1,"X<div>","</div>"]),p.fn.extend({text:function(a){return p.access(this,function(a){return a===b?p.text(this):this.empty().append((this[0]&&this[0].ownerDocument||e).createTextNode(a))},null,a,arguments.length)},wrapAll:function(a){if(p.isFunction(a))return this.each(function(b){p(this).wrapAll(a.call(this,b))});if(this[
0]){var b=p(a,this[0].ownerDocument).eq(0).clone(!0);this[0].parentNode&&b.insertBefore(this[0]),b.map(function(){var a=this;while(a.firstChild&&a.firstChild.nodeType===1)a=a.firstChild;return a}).append(this)}return this},wrapInner:function(a){return p.isFunction(a)?this.each(function(b){p(this).wrapInner(a.call(this,b))}):this.each(function(){var b=p(this),c=b.contents();c.length?c.wrapAll(a):b.append(a)})},wrap:function(a){var b=p.isFunction(a);return this.each(function(c){p(this).wrapAll(b?a.call(this,c):a)})},unwrap:function(){return this.parent().each(function(){p.nodeName(this,"body")||p(this).replaceWith(this.childNodes)}).end()},append:function(){return this.domManip(arguments,!0,function(a){(this.nodeType===1||this.nodeType===11)&&this.appendChild(a)})},prepend:function(){return this.domManip(arguments,!0,function(a){(this.nodeType===1||this.nodeType===11)&&this.insertBefore(a,this.firstChild)})},before:function(){if(!bh(this[0]))return this.domManip(arguments,!1,function(
a){this.parentNode.insertBefore(a,this)});if(arguments.length){var a=p.clean(arguments);return this.pushStack(p.merge(a,this),"before",this.selector)}},after:function(){if(!bh(this[0]))return this.domManip(arguments,!1,function(a){this.parentNode.insertBefore(a,this.nextSibling)});if(arguments.length){var a=p.clean(arguments);return this.pushStack(p.merge(this,a),"after",this.selector)}},remove:function(a,b){var c,d=0;for(;(c=this[d])!=null;d++)if(!a||p.filter(a,[c]).length)!b&&c.nodeType===1&&(p.cleanData(c.getElementsByTagName("*")),p.cleanData([c])),c.parentNode&&c.parentNode.removeChild(c);return this},empty:function(){var a,b=0;for(;(a=this[b])!=null;b++){a.nodeType===1&&p.cleanData(a.getElementsByTagName("*"));while(a.firstChild)a.removeChild(a.firstChild)}return this},clone:function(a,b){return a=a==null?!1:a,b=b==null?a:b,this.map(function(){return p.clone(this,a,b)})},html:function(a){return p.access(this,function(a){var c=this[0]||{},d=0,e=this.length;if(a===b)return c.nod
eType===1?c.innerHTML.replace(bm,""):b;if(typeof a=="string"&&!bs.test(a)&&(p.support.htmlSerialize||!bu.test(a))&&(p.support.leadingWhitespace||!bn.test(a))&&!bz[(bp.exec(a)||["",""])[1].toLowerCase()]){a=a.replace(bo,"<$1></$2>");try{for(;d<e;d++)c=this[d]||{},c.nodeType===1&&(p.cleanData(c.getElementsByTagName("*")),c.innerHTML=a);c=0}catch(f){}}c&&this.empty().append(a)},null,a,arguments.length)},replaceWith:function(a){return bh(this[0])?this.length?this.pushStack(p(p.isFunction(a)?a():a),"replaceWith",a):this:p.isFunction(a)?this.each(function(b){var c=p(this),d=c.html();c.replaceWith(a.call(this,b,d))}):(typeof a!="string"&&(a=p(a).detach()),this.each(function(){var b=this.nextSibling,c=this.parentNode;p(this).remove(),b?p(b).before(a):p(c).append(a)}))},detach:function(a){return this.remove(a,!0)},domManip:function(a,c,d){a=[].concat.apply([],a);var e,f,g,h,i=0,j=a[0],k=[],l=this.length;if(!p.support.checkClone&&l>1&&typeof j=="string"&&bw.test(j))return this.each(function()
{p(this).domManip(a,c,d)});if(p.isFunction(j))return this.each(function(e){var f=p(this);a[0]=j.call(this,e,c?f.html():b),f.domManip(a,c,d)});if(this[0]){e=p.buildFragment(a,this,k),g=e.fragment,f=g.firstChild,g.childNodes.length===1&&(g=f);if(f){c=c&&p.nodeName(f,"tr");for(h=e.cacheable||l-1;i<l;i++)d.call(c&&p.nodeName(this[i],"table")?bC(this[i],"tbody"):this[i],i===h?g:p.clone(g,!0,!0))}g=f=null,k.length&&p.each(k,function(a,b){b.src?p.ajax?p.ajax({url:b.src,type:"GET",dataType:"script",async:!1,global:!1,"throws":!0}):p.error("no ajax"):p.globalEval((b.text||b.textContent||b.innerHTML||"").replace(by,"")),b.parentNode&&b.parentNode.removeChild(b)})}return this}}),p.buildFragment=function(a,c,d){var f,g,h,i=a[0];return c=c||e,c=(c[0]||c).ownerDocument||c[0]||c,typeof c.createDocumentFragment=="undefined"&&(c=e),a.length===1&&typeof i=="string"&&i.length<512&&c===e&&i.charAt(0)==="<"&&!bt.test(i)&&(p.support.checkClone||!bw.test(i))&&(p.support.html5Clone||!bu.test(i))&&(g=!0,f=p
.fragments[i],h=f!==b),f||(f=c.createDocumentFragment(),p.clean(a,c,f,d),g&&(p.fragments[i]=h&&f)),{fragment:f,cacheable:g}},p.fragments={},p.each({appendTo:"append",prependTo:"prepend",insertBefore:"before",insertAfter:"after",replaceAll:"replaceWith"},function(a,b){p.fn[a]=function(c){var d,e=0,f=[],g=p(c),h=g.length,i=this.length===1&&this[0].parentNode;if((i==null||i&&i.nodeType===11&&i.childNodes.length===1)&&h===1)return g[b](this[0]),this;for(;e<h;e++)d=(e>0?this.clone(!0):this).get(),p(g[e])[b](d),f=f.concat(d);return this.pushStack(f,a,g.selector)}}),p.extend({clone:function(a,b,c){var d,e,f,g;p.support.html5Clone||p.isXMLDoc(a)||!bu.test("<"+a.nodeName+">")?g=a.cloneNode(!0):(bB.innerHTML=a.outerHTML,bB.removeChild(g=bB.firstChild));if((!p.support.noCloneEvent||!p.support.noCloneChecked)&&(a.nodeType===1||a.nodeType===11)&&!p.isXMLDoc(a)){bE(a,g),d=bF(a),e=bF(g);for(f=0;d[f];++f)e[f]&&bE(d[f],e[f])}if(b){bD(a,g);if(c){d=bF(a),e=bF(g);for(f=0;d[f];++f)bD(d[f],e[f])}}return
d=e=null,g},clean:function(a,b,c,d){var f,g,h,i,j,k,l,m,n,o,q,r,s=0,t=[];if(!b||typeof b.createDocumentFragment=="undefined")b=e;for(g=b===e&&bA;(h=a[s])!=null;s++){typeof h=="number"&&(h+="");if(!h)continue;if(typeof h=="string")if(!br.test(h))h=b.createTextNode(h);else{g=g||bk(b),l=l||g.appendChild(b.createElement("div")),h=h.replace(bo,"<$1></$2>"),i=(bp.exec(h)||["",""])[1].toLowerCase(),j=bz[i]||bz._default,k=j[0],l.innerHTML=j[1]+h+j[2];while(k--)l=l.lastChild;if(!p.support.tbody){m=bq.test(h),n=i==="table"&&!m?l.firstChild&&l.firstChild.childNodes:j[1]==="<table>"&&!m?l.childNodes:[];for(f=n.length-1;f>=0;--f)p.nodeName(n[f],"tbody")&&!n[f].childNodes.length&&n[f].parentNode.removeChild(n[f])}!p.support.leadingWhitespace&&bn.test(h)&&l.insertBefore(b.createTextNode(bn.exec(h)[0]),l.firstChild),h=l.childNodes,l=g.lastChild}h.nodeType?t.push(h):t=p.merge(t,h)}l&&(g.removeChild(l),h=l=g=null);if(!p.support.appendChecked)for(s=0;(h=t[s])!=null;s++)p.nodeName(h,"input")?bG(h):type
of h.getElementsByTagName!="undefined"&&p.grep(h.getElementsByTagName("input"),bG);if(c){q=function(a){if(!a.type||bx.test(a.type))return d?d.push(a.parentNode?a.parentNode.removeChild(a):a):c.appendChild(a)};for(s=0;(h=t[s])!=null;s++)if(!p.nodeName(h,"script")||!q(h))c.appendChild(h),typeof h.getElementsByTagName!="undefined"&&(r=p.grep(p.merge([],h.getElementsByTagName("script")),q),t.splice.apply(t,[s+1,0].concat(r)),s+=r.length)}return t},cleanData:function(a,b){var c,d,e,f,g=0,h=p.expando,i=p.cache,j=p.support.deleteExpando,k=p.event.special;for(;(e=a[g])!=null;g++)if(b||p.acceptData(e)){d=e[h],c=d&&i[d];if(c){if(c.events)for(f in c.events)k[f]?p.event.remove(e,f):p.removeEvent(e,f,c.handle);i[d]&&(delete i[d],j?delete e[h]:e.removeAttribute?e.removeAttribute(h):e[h]=null,p.deletedIds.push(d))}}}}),function(){var a,b;p.uaMatch=function(a){a=a.toLowerCase();var b=/(chrome)[ \/]([\w.]+)/.exec(a)||/(webkit)[ \/]([\w.]+)/.exec(a)||/(opera)(?:.*version|)[ \/]([\w.]+)/.exec(a)||/(ms
ie) ([\w.]+)/.exec(a)||a.indexOf("compatible")<0&&/(mozilla)(?:.*? rv:([\w.]+)|)/.exec(a)||[];return{browser:b[1]||"",version:b[2]||"0"}},a=p.uaMatch(g.userAgent),b={},a.browser&&(b[a.browser]=!0,b.version=a.version),b.webkit&&(b.safari=!0),p.browser=b,p.sub=function(){function a(b,c){return new a.fn.init(b,c)}p.extend(!0,a,this),a.superclass=this,a.fn=a.prototype=this(),a.fn.constructor=a,a.sub=this.sub,a.fn.init=function c(c,d){return d&&d instanceof p&&!(d instanceof a)&&(d=a(d)),p.fn.init.call(this,c,d,b)},a.fn.init.prototype=a.fn;var b=a(e);return a}}();var bH,bI,bJ,bK=/alpha\([^)]*\)/i,bL=/opacity=([^)]*)/,bM=/^(top|right|bottom|left)$/,bN=/^margin/,bO=new RegExp("^("+q+")(.*)$","i"),bP=new RegExp("^("+q+")(?!px)[a-z%]+$","i"),bQ=new RegExp("^([-+])=("+q+")","i"),bR={},bS={position:"absolute",visibility:"hidden",display:"block"},bT={letterSpacing:0,fontWeight:400,lineHeight:1},bU=["Top","Right","Bottom","Left"],bV=["Webkit","O","Moz","ms"],bW=p.fn.toggle;p.fn.extend({css:funct
ion(a,c){return p.access(this,function(a,c,d){return d!==b?p.style(a,c,d):p.css(a,c)},a,c,arguments.length>1)},show:function(){return bZ(this,!0)},hide:function(){return bZ(this)},toggle:function(a,b){var c=typeof a=="boolean";return p.isFunction(a)&&p.isFunction(b)?bW.apply(this,arguments):this.each(function(){(c?a:bY(this))?p(this).show():p(this).hide()})}}),p.extend({cssHooks:{opacity:{get:function(a,b){if(b){var c=bH(a,"opacity");return c===""?"1":c}}}},cssNumber:{fillOpacity:!0,fontWeight:!0,lineHeight:!0,opacity:!0,orphans:!0,widows:!0,zIndex:!0,zoom:!0},cssProps:{"float":p.support.cssFloat?"cssFloat":"styleFloat"},style:function(a,c,d,e){if(!a||a.nodeType===3||a.nodeType===8||!a.style)return;var f,g,h,i=p.camelCase(c),j=a.style;c=p.cssProps[i]||(p.cssProps[i]=bX(j,i)),h=p.cssHooks[c]||p.cssHooks[i];if(d===b)return h&&"get"in h&&(f=h.get(a,!1,e))!==b?f:j[c];g=typeof d,g==="string"&&(f=bQ.exec(d))&&(d=(f[1]+1)*f[2]+parseFloat(p.css(a,c)),g="number");if(d==null||g==="number"&&is
NaN(d))return;g==="number"&&!p.cssNumber[i]&&(d+="px");if(!h||!("set"in h)||(d=h.set(a,d,e))!==b)try{j[c]=d}catch(k){}},css:function(a,c,d,e){var f,g,h,i=p.camelCase(c);return c=p.cssProps[i]||(p.cssProps[i]=bX(a.style,i)),h=p.cssHooks[c]||p.cssHooks[i],h&&"get"in h&&(f=h.get(a,!0,e)),f===b&&(f=bH(a,c)),f==="normal"&&c in bT&&(f=bT[c]),d||e!==b?(g=parseFloat(f),d||p.isNumeric(g)?g||0:f):f},swap:function(a,b,c){var d,e,f={};for(e in b)f[e]=a.style[e],a.style[e]=b[e];d=c.call(a);for(e in b)a.style[e]=f[e];return d}}),a.getComputedStyle?bH=function(a,b){var c,d,e,f,g=getComputedStyle(a,null),h=a.style;return g&&(c=g[b],c===""&&!p.contains(a.ownerDocument.documentElement,a)&&(c=p.style(a,b)),bP.test(c)&&bN.test(b)&&(d=h.width,e=h.minWidth,f=h.maxWidth,h.minWidth=h.maxWidth=h.width=c,c=g.width,h.width=d,h.minWidth=e,h.maxWidth=f)),c}:e.documentElement.currentStyle&&(bH=function(a,b){var c,d,e=a.currentStyle&&a.currentStyle[b],f=a.style;return e==null&&f&&f[b]&&(e=f[b]),bP.test(e)&&!bM.te
st(b)&&(c=f.left,d=a.runtimeStyle&&a.runtimeStyle.left,d&&(a.runtimeStyle.left=a.currentStyle.left),f.left=b==="fontSize"?"1em":e,e=f.pixelLeft+"px",f.left=c,d&&(a.runtimeStyle.left=d)),e===""?"auto":e}),p.each(["height","width"],function(a,b){p.cssHooks[b]={get:function(a,c,d){if(c)return a.offsetWidth!==0||bH(a,"display")!=="none"?ca(a,b,d):p.swap(a,bS,function(){return ca(a,b,d)})},set:function(a,c,d){return b$(a,c,d?b_(a,b,d,p.support.boxSizing&&p.css(a,"boxSizing")==="border-box"):0)}}}),p.support.opacity||(p.cssHooks.opacity={get:function(a,b){return bL.test((b&&a.currentStyle?a.currentStyle.filter:a.style.filter)||"")?.01*parseFloat(RegExp.$1)+"":b?"1":""},set:function(a,b){var c=a.style,d=a.currentStyle,e=p.isNumeric(b)?"alpha(opacity="+b*100+")":"",f=d&&d.filter||c.filter||"";c.zoom=1;if(b>=1&&p.trim(f.replace(bK,""))===""&&c.removeAttribute){c.removeAttribute("filter");if(d&&!d.filter)return}c.filter=bK.test(f)?f.replace(bK,e):f+" "+e}}),p(function(){p.support.reliableMarg
inRight||(p.cssHooks.marginRight={get:function(a,b){return p.swap(a,{display:"inline-block"},function(){if(b)return bH(a,"marginRight")})}}),!p.support.pixelPosition&&p.fn.position&&p.each(["top","left"],function(a,b){p.cssHooks[b]={get:function(a,c){if(c){var d=bH(a,b);return bP.test(d)?p(a).position()[b]+"px":d}}}})}),p.expr&&p.expr.filters&&(p.expr.filters.hidden=function(a){return a.offsetWidth===0&&a.offsetHeight===0||!p.support.reliableHiddenOffsets&&(a.style&&a.style.display||bH(a,"display"))==="none"},p.expr.filters.visible=function(a){return!p.expr.filters.hidden(a)}),p.each({margin:"",padding:"",border:"Width"},function(a,b){p.cssHooks[a+b]={expand:function(c){var d,e=typeof c=="string"?c.split(" "):[c],f={};for(d=0;d<4;d++)f[a+bU[d]+b]=e[d]||e[d-2]||e[0];return f}},bN.test(a)||(p.cssHooks[a+b].set=b$)});var cc=/%20/g,cd=/\[\]$/,ce=/\r?\n/g,cf=/^(?:color|date|datetime|datetime-local|email|hidden|month|number|password|range|search|tel|text|time|url|week)$/i,cg=/^(?:select|t
extarea)/i;p.fn.extend({serialize:function(){return p.param(this.serializeArray())},serializeArray:function(){return this.map(function(){return this.elements?p.makeArray(this.elements):this}).filter(function(){return this.name&&!this.disabled&&(this.checked||cg.test(this.nodeName)||cf.test(this.type))}).map(function(a,b){var c=p(this).val();return c==null?null:p.isArray(c)?p.map(c,function(a,c){return{name:b.name,value:a.replace(ce,"\r\n")}}):{name:b.name,value:c.replace(ce,"\r\n")}}).get()}}),p.param=function(a,c){var d,e=[],f=function(a,b){b=p.isFunction(b)?b():b==null?"":b,e[e.length]=encodeURIComponent(a)+"="+encodeURIComponent(b)};c===b&&(c=p.ajaxSettings&&p.ajaxSettings.traditional);if(p.isArray(a)||a.jquery&&!p.isPlainObject(a))p.each(a,function(){f(this.name,this.value)});else for(d in a)ch(d,a[d],c,f);return e.join("&").replace(cc,"+")};var ci,cj,ck=/#.*$/,cl=/^(.*?):[ \t]*([^\r\n]*)\r?$/mg,cm=/^(?:about|app|app\-storage|.+\-extension|file|res|widget):$/,cn=/^(?:GET|HEAD)$/
,co=/^\/\//,cp=/\?/,cq=/<script\b[^<]*(?:(?!<\/script>)<[^<]*)*<\/script>/gi,cr=/([?&])_=[^&]*/,cs=/^([\w\+\.\-]+:)(?:\/\/([^\/?#:]*)(?::(\d+)|)|)/,ct=p.fn.load,cu={},cv={},cw=["*/"]+["*"];try{ci=f.href}catch(cx){ci=e.createElement("a"),ci.href="",ci=ci.href}cj=cs.exec(ci.toLowerCase())||[],p.fn.load=function(a,c,d){if(typeof a!="string"&&ct)return ct.apply(this,arguments);if(!this.length)return this;var e,f,g,h=this,i=a.indexOf(" ");return i>=0&&(e=a.slice(i,a.length),a=a.slice(0,i)),p.isFunction(c)?(d=c,c=b):typeof c=="object"&&(f="POST"),p.ajax({url:a,type:f,dataType:"html",data:c,complete:function(a,b){d&&h.each(d,g||[a.responseText,b,a])}}).done(function(a){g=arguments,h.html(e?p("<div>").append(a.replace(cq,"")).find(e):a)}),this},p.each("ajaxStart ajaxStop ajaxComplete ajaxError ajaxSuccess ajaxSend".split(" "),function(a,b){p.fn[b]=function(a){return this.on(b,a)}}),p.each(["get","post"],function(a,c){p[c]=function(a,d,e,f){return p.isFunction(d)&&(f=f||e,e=d,d=b),p.ajax({ty
pe:c,url:a,data:d,success:e,dataType:f})}}),p.extend({getScript:function(a,c){return p.get(a,b,c,"script")},getJSON:function(a,b,c){return p.get(a,b,c,"json")},ajaxSetup:function(a,b){return b?cA(a,p.ajaxSettings):(b=a,a=p.ajaxSettings),cA(a,b),a},ajaxSettings:{url:ci,isLocal:cm.test(cj[1]),global:!0,type:"GET",contentType:"application/x-www-form-urlencoded; charset=UTF-8",processData:!0,async:!0,accepts:{xml:"application/xml, text/xml",html:"text/html",text:"text/plain",json:"application/json, text/javascript","*":cw},contents:{xml:/xml/,html:/html/,json:/json/},responseFields:{xml:"responseXML",text:"responseText"},converters:{"* text":a.String,"text html":!0,"text json":p.parseJSON,"text xml":p.parseXML},flatOptions:{context:!0,url:!0}},ajaxPrefilter:cy(cu),ajaxTransport:cy(cv),ajax:function(a,c){function y(a,c,f,i){var k,s,t,u,w,y=c;if(v===2)return;v=2,h&&clearTimeout(h),g=b,e=i||"",x.readyState=a>0?4:0,f&&(u=cB(l,x,f));if(a>=200&&a<300||a===304)l.ifModified&&(w=x.getResponseHea
der("Last-Modified"),w&&(p.lastModified[d]=w),w=x.getResponseHeader("Etag"),w&&(p.etag[d]=w)),a===304?(y="notmodified",k=!0):(k=cC(l,u),y=k.state,s=k.data,t=k.error,k=!t);else{t=y;if(!y||a)y="error",a<0&&(a=0)}x.status=a,x.statusText=""+(c||y),k?o.resolveWith(m,[s,y,x]):o.rejectWith(m,[x,y,t]),x.statusCode(r),r=b,j&&n.trigger("ajax"+(k?"Success":"Error"),[x,l,k?s:t]),q.fireWith(m,[x,y]),j&&(n.trigger("ajaxComplete",[x,l]),--p.active||p.event.trigger("ajaxStop"))}typeof a=="object"&&(c=a,a=b),c=c||{};var d,e,f,g,h,i,j,k,l=p.ajaxSetup({},c),m=l.context||l,n=m!==l&&(m.nodeType||m instanceof p)?p(m):p.event,o=p.Deferred(),q=p.Callbacks("once memory"),r=l.statusCode||{},t={},u={},v=0,w="canceled",x={readyState:0,setRequestHeader:function(a,b){if(!v){var c=a.toLowerCase();a=u[c]=u[c]||a,t[a]=b}return this},getAllResponseHeaders:function(){return v===2?e:null},getResponseHeader:function(a){var c;if(v===2){if(!f){f={};while(c=cl.exec(e))f[c[1].toLowerCase()]=c[2]}c=f[a.toLowerCase()]}return
c===b?null:c},overrideMimeType:function(a){return v||(l.mimeType=a),this},abort:function(a){return a=a||w,g&&g.abort(a),y(0,a),this}};o.promise(x),x.success=x.done,x.error=x.fail,x.complete=q.add,x.statusCode=function(a){if(a){var b;if(v<2)for(b in a)r[b]=[r[b],a[b]];else b=a[x.status],x.always(b)}return this},l.url=((a||l.url)+"").replace(ck,"").replace(co,cj[1]+"//"),l.dataTypes=p.trim(l.dataType||"*").toLowerCase().split(s),l.crossDomain==null&&(i=cs.exec(l.url.toLowerCase()),l.crossDomain=!(!i||i[1]==cj[1]&&i[2]==cj[2]&&(i[3]||(i[1]==="http:"?80:443))==(cj[3]||(cj[1]==="http:"?80:443)))),l.data&&l.processData&&typeof l.data!="string"&&(l.data=p.param(l.data,l.traditional)),cz(cu,l,c,x);if(v===2)return x;j=l.global,l.type=l.type.toUpperCase(),l.hasContent=!cn.test(l.type),j&&p.active++===0&&p.event.trigger("ajaxStart");if(!l.hasContent){l.data&&(l.url+=(cp.test(l.url)?"&":"?")+l.data,delete l.data),d=l.url;if(l.cache===!1){var z=p.now(),A=l.url.replace(cr,"$1_="+z);l.url=A+(A===
l.url?(cp.test(l.url)?"&":"?")+"_="+z:"")}}(l.data&&l.hasContent&&l.contentType!==!1||c.contentType)&&x.setRequestHeader("Content-Type",l.contentType),l.ifModified&&(d=d||l.url,p.lastModified[d]&&x.setRequestHeader("If-Modified-Since",p.lastModified[d]),p.etag[d]&&x.setRequestHeader("If-None-Match",p.etag[d])),x.setRequestHeader("Accept",l.dataTypes[0]&&l.accepts[l.dataTypes[0]]?l.accepts[l.dataTypes[0]]+(l.dataTypes[0]!=="*"?", "+cw+"; q=0.01":""):l.accepts["*"]);for(k in l.headers)x.setRequestHeader(k,l.headers[k]);if(!l.beforeSend||l.beforeSend.call(m,x,l)!==!1&&v!==2){w="abort";for(k in{success:1,error:1,complete:1})x[k](l[k]);g=cz(cv,l,c,x);if(!g)y(-1,"No Transport");else{x.readyState=1,j&&n.trigger("ajaxSend",[x,l]),l.async&&l.timeout>0&&(h=setTimeout(function(){x.abort("timeout")},l.timeout));try{v=1,g.send(t,y)}catch(B){if(v<2)y(-1,B);else throw B}}return x}return x.abort()},active:0,lastModified:{},etag:{}});var cD=[],cE=/\?/,cF=/(=)\?(?=&|$)|\?\?/,cG=p.now();p.ajaxSetup({j
sonp:"callback",jsonpCallback:function(){var a=cD.pop()||p.expando+"_"+cG++;return this[a]=!0,a}}),p.ajaxPrefilter("json jsonp",function(c,d,e){var f,g,h,i=c.data,j=c.url,k=c.jsonp!==!1,l=k&&cF.test(j),m=k&&!l&&typeof i=="string"&&!(c.contentType||"").indexOf("application/x-www-form-urlencoded")&&cF.test(i);if(c.dataTypes[0]==="jsonp"||l||m)return f=c.jsonpCallback=p.isFunction(c.jsonpCallback)?c.jsonpCallback():c.jsonpCallback,g=a[f],l?c.url=j.replace(cF,"$1"+f):m?c.data=i.replace(cF,"$1"+f):k&&(c.url+=(cE.test(j)?"&":"?")+c.jsonp+"="+f),c.converters["script json"]=function(){return h||p.error(f+" was not called"),h[0]},c.dataTypes[0]="json",a[f]=function(){h=arguments},e.always(function(){a[f]=g,c[f]&&(c.jsonpCallback=d.jsonpCallback,cD.push(f)),h&&p.isFunction(g)&&g(h[0]),h=g=b}),"script"}),p.ajaxSetup({accepts:{script:"text/javascript, application/javascript, application/ecmascript, application/x-ecmascript"},contents:{script:/javascript|ecmascript/},converters:{"text script":fu
nction(a){return p.globalEval(a),a}}}),p.ajaxPrefilter("script",function(a){a.cache===b&&(a.cache=!1),a.crossDomain&&(a.type="GET",a.global=!1)}),p.ajaxTransport("script",function(a){if(a.crossDomain){var c,d=e.head||e.getElementsByTagName("head")[0]||e.documentElement;return{send:function(f,g){c=e.createElement("script"),c.async="async",a.scriptCharset&&(c.charset=a.scriptCharset),c.src=a.url,c.onload=c.onreadystatechange=function(a,e){if(e||!c.readyState||/loaded|complete/.test(c.readyState))c.onload=c.onreadystatechange=null,d&&c.parentNode&&d.removeChild(c),c=b,e||g(200,"success")},d.insertBefore(c,d.firstChild)},abort:function(){c&&c.onload(0,1)}}}});var cH,cI=a.ActiveXObject?function(){for(var a in cH)cH[a](0,1)}:!1,cJ=0;p.ajaxSettings.xhr=a.ActiveXObject?function(){return!this.isLocal&&cK()||cL()}:cK,function(a){p.extend(p.support,{ajax:!!a,cors:!!a&&"withCredentials"in a})}(p.ajaxSettings.xhr()),p.support.ajax&&p.ajaxTransport(function(c){if(!c.crossDomain||p.support.cors){v
ar d;return{send:function(e,f){var g,h,i=c.xhr();c.username?i.open(c.type,c.url,c.async,c.username,c.password):i.open(c.type,c.url,c.async);if(c.xhrFields)for(h in c.xhrFields)i[h]=c.xhrFields[h];c.mimeType&&i.overrideMimeType&&i.overrideMimeType(c.mimeType),!c.crossDomain&&!e["X-Requested-With"]&&(e["X-Requested-With"]="XMLHttpRequest");try{for(h in e)i.setRequestHeader(h,e[h])}catch(j){}i.send(c.hasContent&&c.data||null),d=function(a,e){var h,j,k,l,m;try{if(d&&(e||i.readyState===4)){d=b,g&&(i.onreadystatechange=p.noop,cI&&delete cH[g]);if(e)i.readyState!==4&&i.abort();else{h=i.status,k=i.getAllResponseHeaders(),l={},m=i.responseXML,m&&m.documentElement&&(l.xml=m);try{l.text=i.responseText}catch(a){}try{j=i.statusText}catch(n){j=""}!h&&c.isLocal&&!c.crossDomain?h=l.text?200:404:h===1223&&(h=204)}}}catch(o){e||f(-1,o)}l&&f(h,j,l,k)},c.async?i.readyState===4?setTimeout(d,0):(g=++cJ,cI&&(cH||(cH={},p(a).unload(cI)),cH[g]=d),i.onreadystatechange=d):d()},abort:function(){d&&d(0,1)}}}});
var cM,cN,cO=/^(?:toggle|show|hide)$/,cP=new RegExp("^(?:([-+])=|)("+q+")([a-z%]*)$","i"),cQ=/queueHooks$/,cR=[cX],cS={"*":[function(a,b){var c,d,e,f=this.createTween(a,b),g=cP.exec(b),h=f.cur(),i=+h||0,j=1;if(g){c=+g[2],d=g[3]||(p.cssNumber[a]?"":"px");if(d!=="px"&&i){i=p.css(f.elem,a,!0)||c||1;do e=j=j||".5",i=i/j,p.style(f.elem,a,i+d),j=f.cur()/h;while(j!==1&&j!==e)}f.unit=d,f.start=i,f.end=g[1]?i+(g[1]+1)*c:c}return f}]};p.Animation=p.extend(cV,{tweener:function(a,b){p.isFunction(a)?(b=a,a=["*"]):a=a.split(" ");var c,d=0,e=a.length;for(;d<e;d++)c=a[d],cS[c]=cS[c]||[],cS[c].unshift(b)},prefilter:function(a,b){b?cR.unshift(a):cR.push(a)}}),p.Tween=cY,cY.prototype={constructor:cY,init:function(a,b,c,d,e,f){this.elem=a,this.prop=c,this.easing=e||"swing",this.options=b,this.start=this.now=this.cur(),this.end=d,this.unit=f||(p.cssNumber[c]?"":"px")},cur:function(){var a=cY.propHooks[this.prop];return a&&a.get?a.get(this):cY.propHooks._default.get(this)},run:function(a){var b,c=cY.prop
Hooks[this.prop];return this.pos=b=p.easing[this.easing](a,this.options.duration*a,0,1,this.options.duration),this.now=(this.end-this.start)*b+this.start,this.options.step&&this.options.step.call(this.elem,this.now,this),c&&c.set?c.set(this):cY.propHooks._default.set(this),this}},cY.prototype.init.prototype=cY.prototype,cY.propHooks={_default:{get:function(a){var b;return a.elem[a.prop]==null||!!a.elem.style&&a.elem.style[a.prop]!=null?(b=p.css(a.elem,a.prop,!1,""),!b||b==="auto"?0:b):a.elem[a.prop]},set:function(a){p.fx.step[a.prop]?p.fx.step[a.prop](a):a.elem.style&&(a.elem.style[p.cssProps[a.prop]]!=null||p.cssHooks[a.prop])?p.style(a.elem,a.prop,a.now+a.unit):a.elem[a.prop]=a.now}}},cY.propHooks.scrollTop=cY.propHooks.scrollLeft={set:function(a){a.elem.nodeType&&a.elem.parentNode&&(a.elem[a.prop]=a.now)}},p.each(["toggle","show","hide"],function(a,b){var c=p.fn[b];p.fn[b]=function(d,e,f){return d==null||typeof d=="boolean"||!a&&p.isFunction(d)&&p.isFunction(e)?c.apply(this,argum
ents):this.animate(cZ(b,!0),d,e,f)}}),p.fn.extend({fadeTo:function(a,b,c,d){return this.filter(bY).css("opacity",0).show().end().animate({opacity:b},a,c,d)},animate:function(a,b,c,d){var e=p.isEmptyObject(a),f=p.speed(b,c,d),g=function(){var b=cV(this,p.extend({},a),f);e&&b.stop(!0)};return e||f.queue===!1?this.each(g):this.queue(f.queue,g)},stop:function(a,c,d){var e=function(a){var b=a.stop;delete a.stop,b(d)};return typeof a!="string"&&(d=c,c=a,a=b),c&&a!==!1&&this.queue(a||"fx",[]),this.each(function(){var b=!0,c=a!=null&&a+"queueHooks",f=p.timers,g=p._data(this);if(c)g[c]&&g[c].stop&&e(g[c]);else for(c in g)g[c]&&g[c].stop&&cQ.test(c)&&e(g[c]);for(c=f.length;c--;)f[c].elem===this&&(a==null||f[c].queue===a)&&(f[c].anim.stop(d),b=!1,f.splice(c,1));(b||!d)&&p.dequeue(this,a)})}}),p.each({slideDown:cZ("show"),slideUp:cZ("hide"),slideToggle:cZ("toggle"),fadeIn:{opacity:"show"},fadeOut:{opacity:"hide"},fadeToggle:{opacity:"toggle"}},function(a,b){p.fn[a]=function(a,c,d){return this.a
nimate(b,a,c,d)}}),p.speed=function(a,b,c){var d=a&&typeof a=="object"?p.extend({},a):{complete:c||!c&&b||p.isFunction(a)&&a,duration:a,easing:c&&b||b&&!p.isFunction(b)&&b};d.duration=p.fx.off?0:typeof d.duration=="number"?d.duration:d.duration in p.fx.speeds?p.fx.speeds[d.duration]:p.fx.speeds._default;if(d.queue==null||d.queue===!0)d.queu
<TRUNCATED>
[39/47] incubator-systemml git commit: [SYSML-328] Markdown for
additional algorithms
Posted by du...@apache.org.
[SYSML-328] Markdown for additional algorithms
Highlighted mandatory and optional arguments to scripts
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/2a2520bd
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/2a2520bd
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/2a2520bd
Branch: refs/heads/gh-pages
Commit: 2a2520bd8aa108c204d7bbdee470c61580127084
Parents: b053ce6
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Thu Oct 22 17:03:20 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Thu Oct 22 17:03:20 2015 -0700
----------------------------------------------------------------------
_config.yml | 2 +-
algorithms-bibliography.md | 16 +
algorithms-classification.md | 850 ++++++++++++++++---
algorithms-clustering.md | 75 +-
algorithms-descriptive-statistics.md | 100 ++-
algorithms-matrix-factorization.md | 345 +++++++-
algorithms-reference.md | 12 +
algorithms-regression.md | 440 ++++++++--
algorithms-survival-analysis.md | 709 ++++++++++++++++
...-quartiles-median-and-interquartile-mean.png | Bin 26477 -> 0 bytes
...-quartiles-median-and-interquartile-mean.png | Bin 0 -> 26477 bytes
img/algorithms-reference/example-tree.png | Bin 0 -> 14758 bytes
12 files changed, 2225 insertions(+), 324 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2a2520bd/_config.yml
----------------------------------------------------------------------
diff --git a/_config.yml b/_config.yml
index 9ea5762..0419bff 100644
--- a/_config.yml
+++ b/_config.yml
@@ -13,4 +13,4 @@ include:
- _modules
# These allow the documentation to be updated with newer releases
-SYSTEMML_VERSION: 5.2
+SYSTEMML_VERSION: 0.8.0
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2a2520bd/algorithms-bibliography.md
----------------------------------------------------------------------
diff --git a/algorithms-bibliography.md b/algorithms-bibliography.md
index c583dc6..1012d09 100644
--- a/algorithms-bibliography.md
+++ b/algorithms-bibliography.md
@@ -4,6 +4,8 @@ title: SystemML Algorithms Reference - Bibliography
displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
---
+# 7. Bibliography
+
AcockStavig1979, Alan C. Acock and Gordon
R. Stavig, A Measure of Association for Nonparametric
Statistics, Social Forces, Oxford University
@@ -35,6 +37,10 @@ Annual ACM-SIAM Symposium on Discrete Algorithms (SODA 2007),
January 7--9, 2007, New Orleans, LA,
USA, 1027--1035.
+Breiman2001, L. Breiman. Random forests. Machine Learning, 45(1):5–32, 2001.
+
+BreimanFOS1984, L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth, 1984.
+
Chapelle2007, Olivier Chapelle, Training a Support Vector Machine in the Primal, Neural Computation, 2007.
Cochran1954, William G. Cochran,
@@ -42,6 +48,8 @@ Some Methods for Strengthening the Common $\chi^2$ Tests,
Biometrics, Volume 10, Number 4, December
1954, 417--451.
+Collett2003, D. Collett. Modelling Survival Data in Medical Research, Second Edition. Chapman & Hall/CRC Texts in Statistical Science. Taylor & Francis, 2003.
+
Gill2000, Jeff Gill, Generalized Linear
Models: A Unified Approach, Sage University Papers Series on
Quantitative Applications in the Social Sciences, Number 07-134,
@@ -79,6 +87,8 @@ Jorge Nocedal and Stephen Wright, Springer Series
in Operations Research and Financial Engineering, 664,
Second Edition, Springer, 2006.
+PandaHBB2009, B. Panda, J. Herbach, S. Basu, and R. J. Bayardo. PLANET: massively parallel learning of tree ensembles with mapreduce. PVLDB, 2(2):1426– 1437, 2009.
+
Russell2009, S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, Prentice Hall, 2009.
Scholkopf1995, B. Scholkopf, C. Burges and V. Vapnik, Extracting Support Data for a Given Task, International Conference on Knowledge Discovery and Data Mining (ICDM), 1995.
@@ -91,3 +101,9 @@ June 7, 1946, Volume 103, Number 2684,
Vetterling1992,
W. T. Vetterling and B. P. Flannery,
Multidimensions in Numerical Recipes in C - The Art in Scientific Computing, W. H. Press and S. A. Teukolsky (eds.), Cambridge University Press, 1992.
+
+ZhouWSP08,
+Y. Zhou, D. M. Wilkinson, R. Schreiber, and R. Pan. Large-scale parallel collaborative filtering for the Netflix prize.
+In Algorithmic Aspects in Information and Management, 4th International Conference, AAIM 2008, Shanghai, China, June 23-25, 2008. Proceedings, pages 337–348, 2008.
+
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2a2520bd/algorithms-classification.md
----------------------------------------------------------------------
diff --git a/algorithms-classification.md b/algorithms-classification.md
index 8a16cd0..d6215df 100644
--- a/algorithms-classification.md
+++ b/algorithms-classification.md
@@ -4,7 +4,6 @@ title: SystemML Algorithms Reference - Classification
displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
---
-
# 2. Classification
@@ -86,7 +85,7 @@ $$
The goal of the regression
is to estimate the parameter matrix $B$ from the provided dataset
-$(X, Y) = (x_i, y_i)_{i=1}^n$ by maximizing the product of over the
+$(X, Y) = (x_i, y_i)_{i=1}^n$ by maximizing the product of $$Prob[y_i\mid x_i; B]$$ over the
observed labels $y_i$. Taking its logarithm, negating, and adding a
regularization term gives us a minimization objective:
@@ -110,16 +109,16 @@ Eqs. (1) and (2).
### Usage
hadoop jar SystemML.jar -f MultiLogReg.dml
- -nvargs X=file
- Y=file
- B=file
- Log=file
- icpt=int
- reg=double
- tol=double
- moi=int
- mii=int
- fmt=format
+ -nvargs X=<file>
+ Y=<file>
+ B=<file>
+ Log=[file]
+ icpt=[int]
+ reg=[double]
+ tol=[double]
+ moi=[int]
+ mii=[int]
+ fmt=[format]
### Arguments
@@ -128,11 +127,12 @@ Eqs. (1) and (2).
constitutes one feature vector.
**Y**: Location to read the input one-column matrix of category labels that
-correspond to feature vectors in X. Note the following:\
-– Each non-baseline category label must be a positive integer.\
-– If all labels are positive, the largest represents the baseline
-category.\
-– If non-positive labels such as $-1$ or $0$ are present, then they
+correspond to feature vectors in X. Note the following:
+
+ * Each non-baseline category label must be a positive integer.
+ * If all labels are positive, the largest represents the baseline
+category.
+ * If non-positive labels such as $-1$ or $0$ are present, then they
represent the (same) baseline category and are converted to label
$\max(\texttt{Y})\,{+}\,1$.
@@ -143,30 +143,31 @@ The size of B is $(m\,{+}\,1)\times (k\,{-}\,1)$ with the
intercepts or $m \times (k\,{-}\,1)$ without the intercepts, one column
per non-baseline category and one row per feature.
-**Log**: (default: " ") Location to store iteration-specific variables for monitoring
+**Log**: (default: `" "`) Location to store iteration-specific variables for monitoring
and debugging purposes, see
[**Table 5**](algorithms-classification.html#table5)
for details.
-**icpt**: (default: 0) Intercept and shifting/rescaling of the features in $X$:\
-0 = no intercept (hence no $\beta_0$), no
-shifting/rescaling of the features;\
-1 = add intercept, but do not shift/rescale the features
-in $X$;\
-2 = add intercept, shift/rescale the features in $X$ to
+**icpt**: (default: `0`) Intercept and shifting/rescaling of the features in $X$:
+
+ * 0 = no intercept (hence no $\beta_0$), no
+shifting/rescaling of the features;
+ * 1 = add intercept, but do not shift/rescale the features
+in $X$;
+ * 2 = add intercept, shift/rescale the features in $X$ to
mean 0, variance 1
-**reg**: (default: 0.0) L2-regularization parameter (lambda)
+**reg**: (default: `0.0`) L2-regularization parameter (lambda)
-**tol**: (default: 0.000001) Tolerance (epsilon) used in the convergence criterion
+**tol**: (default: `0.000001`) Tolerance ($\epsilon$) used in the convergence criterion
-**moi**: (default: 100) Maximum number of outer (Fisher scoring) iterations
+**moi**: (default: `100`) Maximum number of outer (Fisher scoring) iterations
-**mii**: (default: 0) Maximum number of inner (conjugate gradient) iterations, or 0
+**mii**: (default: `0`) Maximum number of inner (conjugate gradient) iterations, or 0
if no maximum limit provided
-**fmt**: (default: `"text"`) Matrix file output format, such as text,
-mm, or csv; see read/write functions in
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
SystemML Language Reference for details.
@@ -189,8 +190,8 @@ SystemML Language Reference for details.
<a name="table5" />
**Table 5**: The `Log` file for multinomial logistic regression
-contains the following iteration variables in CSV format, each line
-containing triple (Name, Iteration\#, Value) with Iteration\# being 0
+contains the following iteration variables in `CSV` format, each line
+containing triple (`Name`, `Iteration#`, `Value`) with `Iteration#` being 0
for initial values.
@@ -242,7 +243,7 @@ f \,\,&=\,\, - \,\,{\textstyle\sum} \,\,Y \cdot (X B)\, + \,
{\textstyle\sum}\,\log\big(\exp(XB)\,1_{k\times 1}\big) \,+ \,
(1/2)\,\, {\textstyle\sum} \,\,\Lambda \cdot B \cdot B\end{aligned}$$
-where operations $\cdot\,$, $/$, $\exp$, and $\log$ are applied
+where operations $\cdot\,$, `/`, `exp`, and `log` are applied
cellwise, and $\textstyle\sum$ denotes the sum of all cells in a matrix.
The gradient of $f$ with respect to $B$ can be represented as a matrix
too:
@@ -278,12 +279,12 @@ $0.5\sqrt{m}\,/ \max_i \|x_i\|_2$
and updated as described
in [[Lin2008]](algorithms-bibliography.html).
Users can specify the maximum number of the outer
-and the inner iterations with input parameters moi and
-mii, respectively. The iterative minimizer terminates
+and the inner iterations with input parameters `moi` and
+`mii`, respectively. The iterative minimizer terminates
successfully if
$$\|\nabla f\|_2 < \varepsilon \|\nabla f_{B=0} \|_2$$
, where ${\varepsilon}> 0$ is a tolerance supplied by the user via input
-parameter tol.
+parameter `tol`.
### Returns
@@ -292,14 +293,14 @@ The estimated regression parameters (the
$$\hat{\beta}_{j, l}$$)
are
populated into a matrix and written to an HDFS file whose path/name was
-provided as the “B” input argument. Only the non-baseline
+provided as the `B` input argument. Only the non-baseline
categories ($1\leq l \leq k\,{-}\,1$) have their
$$\hat{\beta}_{j, l}$$
in the output; to add the baseline category, just append a column of zeros.
-If icpt=0 in the input command line, no intercepts are used
-and B has size
+If `icpt=0` in the input command line, no intercepts are used
+and `B` has size
$m\times (k\,{-}\,1)$; otherwise
-B has size
+`B` has size
$(m\,{+}\,1)\times (k\,{-}\,1)$
and the
intercepts are in the
@@ -319,33 +320,37 @@ original features.
#### Description
Support Vector Machines are used to model the relationship between a
-categorical dependent variable y and one or more explanatory variables
-denoted X. This implementation learns (and predicts with) a binary class
-support vector machine (y with domain size 2).
+categorical dependent variable `y` and one or more explanatory variables
+denoted `X`. This implementation learns (and predicts with) a binary class
+support vector machine (`y` with domain size `2`).
#### Usage
+**Binary-Class Support Vector Machines**:
+
hadoop jar SystemML.jar -f l2-svm.dml
- -nvargs X=file
- Y=file
- icpt=int
- tol=double
- reg=double
- maxiter=int
- model=file
- Log=file
- fmt=format
+ -nvargs X=<file>
+ Y=<file>
+ icpt=[int]
+ tol=[double]
+ reg=[double]
+ maxiter=[int]
+ model=<file>
+ Log=<file>
+ fmt=[format]
+
+**Binary-Class Support Vector Machines Prediction**:
hadoop jar SystemML.jar -f l2-svm-predict.dml
- -nvargs X=file
- Y=file
- icpt=int
- model=file
- scores=file
- accuracy=file
- confusion=file
- fmt=format
+ -nvargs X=<file>
+ Y=[file]
+ icpt=[int]
+ model=<file>
+ scores=[file]
+ accuracy=[file]
+ confusion=[file]
+ fmt=[format]
#### Arguments
@@ -354,24 +359,24 @@ support vector machine (y with domain size 2).
row constitutes one feature vector.
**Y**: Location to read the one-column matrix of (categorical) labels
-that correspond to feature vectors in X. Binary class labels can be
-expressed in one of two choices: $\pm 1$ or $1/2$. Note that, this
+that correspond to feature vectors in `X`. Binary class labels can be
+expressed in one of two choices: $\pm 1$ or $1/2$. Note that this
argument is optional for prediction.
-**icpt**: (default: 0) If set to 1 then a constant bias
-column is added to X.
+**icpt**: (default: `0`) If set to `1` then a constant bias
+column is added to `X`.
-**tol**: (default: 0.001) Procedure terminates early if the
+**tol**: (default: `0.001`) Procedure terminates early if the
reduction in objective function value is less than tolerance times
the initial objective function value.
-**reg**: (default: 1) Regularization constant. See details
+**reg**: (default: `1`) Regularization constant. See details
to find out where lambda appears in the objective function. If one
-were interested in drawing an analogy with the C parameter in C-SVM,
-then C = 2/lambda. Usually, cross validation is employed to
+were interested in drawing an analogy with the `C` parameter in C-SVM,
+then `C = 2/lambda`. Usually, cross validation is employed to
determine the optimum value of lambda.
-**maxiter**: (default: 100) The maximum number
+**maxiter**: (default: `100`) The maximum number
of iterations.
**model**: Location (on HDFS) that contains the learnt weights.
@@ -380,21 +385,24 @@ of iterations.
function value etc.) that depict progress across iterations
while training.
-**fmt**: (default: `"text"`) Specifies the output format.
-Choice of comma-separated values (csv) or as a sparse-matrix (text).
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
**scores**: Location (on HDFS) to store scores for a held-out test set.
-Note that, this is an optional argument.
+Note that this is an optional argument.
**accuracy**: Location (on HDFS) to store the accuracy computed on a
-held-out test set. Note that, this is an optional argument.
+held-out test set. Note that this is an optional argument.
**confusion**: Location (on HDFS) to store the confusion matrix computed
-using a held-out test set. Note that, this is an optional argument.
+using a held-out test set. Note that this is an optional argument.
#### Examples
+**Binary-Class Support Vector Machines**:
+
hadoop jar SystemML.jar -f l2-svm.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/y.mtx
@@ -406,6 +414,8 @@ using a held-out test set. Note that, this is an optional argument.
model=/user/ml/weights.csv
Log=/user/ml/Log.csv
+**Binary-Class Support Vector Machines Prediction**:
+
hadoop jar SystemML.jar -f l2-svm-predict.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/y.mtx
@@ -431,7 +441,7 @@ $y_i$, $w$ is the vector of parameters and $\lambda$ is the
regularization constant specified by the user.
To account for the missing bias term, one may augment the data with a
-column of constants which is achieved by setting the intercept argument to 1
+column of constants which is achieved by setting the intercept argument to `1`
[[Hsieh2008]](algorithms-bibliography.html).
This implementation optimizes the primal directly
@@ -445,8 +455,8 @@ Newton minimization in the direction of the gradient.
The learnt weights produced by `l2-svm.dml` are populated into a single
column matrix and written to file on HDFS (see model in section
-Arguments). The number of rows in this matrix is ncol(X) if intercept
-was set to 0 during invocation and ncol(X) + 1 otherwise. The bias term,
+Arguments). The number of rows in this matrix is `ncol(X)` if intercept
+was set to `0` during invocation and `ncol(X) + 1` otherwise. The bias term,
if used, is placed in the last row. Depending on what arguments are
provided during invocation, `l2-svm-predict.dml` may compute one or more
of scores, accuracy and confusion matrix in the output format
@@ -461,34 +471,38 @@ specified.
#### Description
Support Vector Machines are used to model the relationship between a
-categorical dependent variable y and one or more explanatory variables
-denoted X. This implementation supports dependent variables that have
-domain size greater or equal to 2 and hence is not restricted to binary
+categorical dependent variable `y` and one or more explanatory variables
+denoted `X`. This implementation supports dependent variables that have
+domain size greater or equal to `2` and hence is not restricted to binary
class labels.
#### Usage
+**Multi-Class Support Vector Machines**:
+
hadoop jar SystemML.jar -f m-svm.dml
- -nvargs X=file
- Y=file
- icpt=int
- tol=double
- reg=double
- maxiter=int
- model=file
- Log=file
- fmt=format
+ -nvargs X=<file>
+ Y=<file>
+ icpt=[int]
+ tol=[double]
+ reg=[double]
+ maxiter=[int]
+ model=<file>
+ Log=<file>
+ fmt=[format]
+
+**Multi-Class Support Vector Machines Prediction**:
hadoop jar SystemML.jar -f m-svm-predict.dml
- -nvargs X=file
- Y=file
- icpt=int
- model=file
- scores=file
- accuracy=file
- confusion=file
- fmt=format
+ -nvargs X=<file>
+ Y=[file]
+ icpt=[int]
+ model=<file>
+ scores=[file]
+ accuracy=[file]
+ confusion=[file]
+ fmt=[format]
#### Arguments
@@ -498,23 +512,23 @@ class labels.
**Y**: Location (on HDFS) containing a 1-column matrix specifying the
categorical dependent variable (label). Labels are assumed to be
- contiguously numbered from 1 $\ldots$ \#classes. Note that, this
+ contiguously numbered from 1 $\ldots$ \#classes. Note that this
argument is optional for prediction.
-**icpt**: (default: 0) If set to 1 then a constant bias
- column is added to X.
+**icpt**: (default: `0`) If set to `1` then a constant bias
+ column is added to `X`.
-**tol**: (default: 0.001) Procedure terminates early if the
+**tol**: (default: `0.001`) Procedure terminates early if the
reduction in objective function value is less than tolerance times
the initial objective function value.
-**reg**: (default: 1) Regularization constant. See details
- to find out where lambda appears in the objective function. If one
- were interested in drawing an analogy with C-SVM, then C = 2/lambda.
+**reg**: (default: `1`) Regularization constant. See details
+ to find out where `lambda` appears in the objective function. If one
+ were interested in drawing an analogy with C-SVM, then `C = 2/lambda`.
Usually, cross validation is employed to determine the optimum value
- of lambda.
+ of `lambda`.
-**maxiter**: (default: 100) The maximum number
+**maxiter**: (default: `100`) The maximum number
of iterations.
**model**: Location (on HDFS) that contains the learnt weights.
@@ -523,21 +537,24 @@ class labels.
function value etc.) that depict progress across iterations
while training.
-**fmt**: (default: `"text"`) Specifies the output format.
- Choice of comma-separated values (csv) or as a sparse-matrix (text).
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
**scores**: Location (on HDFS) to store scores for a held-out test set.
- Note that, this is an optional argument.
+ Note that this is an optional argument.
**accuracy**: Location (on HDFS) to store the accuracy computed on a
- held-out test set. Note that, this is an optional argument.
+ held-out test set. Note that this is an optional argument.
**confusion**: Location (on HDFS) to store the confusion matrix computed
- using a held-out test set. Note that, this is an optional argument.
+ using a held-out test set. Note that this is an optional argument.
#### Examples
+**Multi-Class Support Vector Machines**:
+
hadoop jar SystemML.jar -f m-svm.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/y.mtx
@@ -549,6 +566,8 @@ class labels.
model=/user/ml/weights.csv
Log=/user/ml/Log.csv
+**Multi-Class Support Vector Machines Prediction**:
+
hadoop jar SystemML.jar -f m-svm-predict.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/y.mtx
@@ -595,8 +614,8 @@ Newton minimization in the direction of the gradient.
The learnt weights produced by `m-svm.dml` are populated into a matrix
that has as many columns as there are classes in the training data, and
written to file provided on HDFS (see model in section Arguments). The
-number of rows in this matrix is ncol(X) if intercept was set to 0
-during invocation and ncol(X) + 1 otherwise. The bias terms, if used,
+number of rows in this matrix is `ncol(X)` if intercept was set to `0`
+during invocation and `ncol(X) + 1` otherwise. The bias terms, if used,
are placed in the last row. Depending on what arguments are provided
during invocation, `m-svm-predict.dml` may compute one or more of scores,
accuracy and confusion matrix in the output format specified.
@@ -615,24 +634,28 @@ applicable when all features are counts of categorical values.
#### Usage
+**Naive Bayes**:
+
hadoop jar SystemML.jar -f naive-bayes.dml
- -nvargs X=file
- Y=file
- laplace=double
- prior=file
- conditionals=file
- accuracy=file
- fmt=format
+ -nvargs X=<file>
+ Y=<file>
+ laplace=[double]
+ prior=<file>
+ conditionals=<file>
+ accuracy=<file>
+ fmt=[format]
+
+**Naive Bayes Prediction**:
hadoop jar SystemML.jar -f naive-bayes-predict.dml
- -nvargs X=file
- Y=file
- prior=file
- conditionals=file
- fmt=format
- accuracy=file
- confusion=file
- probabilities=file
+ -nvargs X=<file>
+ Y=[file]
+ prior=<file>
+ conditionals=<file>
+ fmt=[format]
+ accuracy=[file]
+ confusion=[file]
+ probabilities=[file]
### Arguments
@@ -641,12 +664,12 @@ applicable when all features are counts of categorical values.
row constitutes one feature vector.
**Y**: Location (on HDFS) to read the one-column matrix of (categorical)
- labels that correspond to feature vectors in X. Classes are assumed
- to be contiguously labeled beginning from 1. Note that, this
+ labels that correspond to feature vectors in `X`. Classes are assumed
+ to be contiguously labeled beginning from `1`. Note that this
argument is optional for prediction.
-**laplace**: (default: 1) Laplace smoothing specified by
- the user to avoid creation of 0 probabilities.
+**laplace**: (default: `1`) Laplace smoothing specified by
+ the user to avoid creation of `0` probabilities.
**prior**: Location (on HDFS) that contains the class
prior probabilites.
@@ -654,24 +677,27 @@ applicable when all features are counts of categorical values.
**conditionals**: Location (on HDFS) that contains the class conditional
feature distributions.
-**fmt** (default: `"text"`): Specifies the output format.
- Choice of comma-separated values (csv) or as a sparse-matrix (text).
+**fmt** (default: `"text"`): Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
**probabilities**: Location (on HDFS) to store class membership
- probabilities for a held-out test set. Note that, this is an
+ probabilities for a held-out test set. Note that this is an
optional argument.
**accuracy**: Location (on HDFS) to store the training accuracy during
learning and testing accuracy from a held-out test set
- during prediction. Note that, this is an optional argument
+ during prediction. Note that this is an optional argument
for prediction.
**confusion**: Location (on HDFS) to store the confusion matrix computed
- using a held-out test set. Note that, this is an optional argument.
+ using a held-out test set. Note that this is an optional argument.
### Examples
+**Naive Bayes**:
+
hadoop jar SystemML.jar -f naive-bayes.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/y.mtx
@@ -680,6 +706,8 @@ applicable when all features are counts of categorical values.
conditionals=/user/ml/conditionals.csv
accuracy=/user/ml/accuracy.csv
+**Naive Bayes Prediction**:
+
hadoop jar SystemML.jar -f naive-bayes-predict.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/y.mtx
@@ -715,7 +743,7 @@ where $\mathcal{C}$ is the set of classes.
Given a fully labeled training dataset, it is possible to learn a naive
Bayes model using simple counting (group-by aggregates). To compute the
class conditional probabilities, it is usually advisable to avoid
-setting $\theta_{iy}$ to 0. One way to achieve this is using additive
+setting $\theta_{iy}$ to `0`. One way to achieve this is using additive
smoothing or Laplace smoothing. Some authors have argued that this
should in fact be add-one smoothing. This implementation uses add-one
smoothing by default but lets the user specify her/his own constant, if
@@ -736,4 +764,548 @@ during invocation, `naive-bayes-predict.dml` may compute one or more of
probabilities, accuracy and confusion matrix in the output format
specified.
-
\ No newline at end of file
+
+* * *
+
+## 2.4. Decision Trees
+
+### Description
+
+Decision tree (for classification) is a classifier that is considered
+more interpretable than other statistical classifiers. This
+implementation is well-suited to handle large-scale data and builds a
+(binary) decision tree in parallel.
+
+
+### Usage
+
+**Decision Tree**:
+
+ hadoop jar SystemML.jar -f decision-tree.dml
+ -nvargs X=<file>
+ Y=<file>
+ R=[file]
+ M=<file>
+ bins=[int]
+ depth=[int]
+ num_leaf=[int]
+ num_samples=[int]
+ impurity=[Gini|entropy]
+ O=[file]
+ S_map=[file]
+ C_map=[file]
+ fmt=[format]
+
+**Decision Tree Prediction**:
+
+ hadoop jar SystemML.jar -f decision-tree-predict.dml
+ -nvargs X=<file>
+ Y=[file]
+ R=[file]
+ M=<file>
+ P=<file>
+ A=[file]
+ CM=[file]
+ fmt=[format]
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read the matrix of feature vectors; each row
+constitutes one feature vector. Note that categorical features in $X$
+need to be both recoded and dummy coded.
+
+**Y**: Location (on HDFS) to read the matrix of (categorical) labels that
+correspond to feature vectors in $X$. Note that class labels are assumed
+to be both recoded and dummy coded. This argument is optional for
+prediction.
+
+**R**: (default: `" "`) Location (on HDFS) to read matrix $R$ which for each feature
+in $X$ contains column-ids (first column), start indices (second
+column), and end indices (third column). If $R$ is not provided by
+default all features are assumed to be continuous-valued.
+
+**M**: Location (on HDFS) to write matrix $M$ containing the learned decision
+tree (see below for the schema)
+
+**bins**: (default: `20`) Number of thresholds to choose for each continuous-valued
+feature (determined by equi-height binning).
+
+**depth**: (default: `25`) Maximum depth of the learned tree
+
+**num_leaf**: (default: `10`) Parameter that controls pruning. The tree is not expanded if
+a node receives less than `num_leaf` training examples.
+
+**num_samples**: (default: `3000`) Parameter that decides when to switch to in-memory building
+of subtrees. If a node $v$ receives less than `num_samples`
+training examples then this implementation switches to an in-memory
+subtree building procedure to build the subtree under $v$ in its
+entirety.
+
+**impurity**: (default: `"Gini"`) Impurity measure used at internal nodes of the tree for
+selecting which features to split on. Possible value are `entropy` or
+`Gini`.
+
+**O**: (default: `" "`) Location (on HDFS) to store the training accuracy (%). Note
+that this argument is optional.
+
+**A**: (default: `" "`) Location (on HDFS) to store the testing accuracy (%) from a
+held-out test set during prediction. Note that this argument is
+optional.
+
+**P**: Location (on HDFS) to store predictions for a held-out test set
+
+**CM**: (default: `" "`) Location (on HDFS) to store the confusion matrix computed
+using a held-out test set. Note that this argument is optional.
+
+**S_map**: (default: `" "`) Location (on HDFS) to write the mappings from the
+continuous-valued feature-ids to the global feature-ids in $X$ (see
+below for details). Note that this argument is optional.
+
+**C_map**: (default: `" "`) Location (on HDFS) to write the mappings from the categorical
+feature-ids to the global feature-ids in $X$ (see below for details).
+Note that this argument is optional.
+
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
+
+
+### Examples
+
+**Decision Tree**:
+
+ hadoop jar SystemML.jar -f decision-tree.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ R=/user/ml/R.csv
+ M=/user/ml/model.csv
+ bins=20
+ depth=25
+ num_leaf=10
+ num_samples=3000
+ impurity=Gini
+ fmt=csv
+
+**Decision Tree Prediction**:
+
+ hadoop jar SystemML.jar -f decision-tree-predict.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ R=/user/ml/R.csv
+ M=/user/ml/model.csv
+ P=/user/ml/predictions.csv
+ A=/user/ml/accuracy.csv
+ CM=/user/ml/confusion.csv
+ fmt=csv
+
+
+
+### Details
+
+Decision trees [[BreimanFOS1984]](algorithms-bibliography.html) are simple models of classification
+that, due to their structure, are easy to interpret. Given an example
+feature vector, each node in the learned tree runs a simple test on it.
+Based on the result of the test, the example is either diverted to the
+left subtree or to the right subtree. Once the example reaches a leaf,
+then the label stored at the leaf is returned as the prediction for the
+example.
+
+Building a decision tree from a fully labeled training set entails
+choosing appropriate splitting tests for each internal node in the tree
+and this is usually performed in a top-down manner. The splitting test
+(denoted by $s$) requires first choosing a feature $j$ and depending on
+the type of $j$, either a threshold $\sigma$, in case $j$ is
+continuous-valued, or a subset of values $S \subseteq \text{Dom}(j)$
+where $\text{Dom}(j)$ denotes domain of $j$, in case it is categorical.
+For continuous-valued features the test is thus of form $x_j < \sigma$
+and for categorical features it is of form $x_j \in S$, where $x_j$
+denotes the $j$th feature value of feature vector $x$. One way to
+determine which test to include, is to compare impurities of the tree
+nodes induced by the test. The *node impurity* measures the
+homogeneity of the labels at the node. This implementation supports two
+commonly used impurity measures (denoted by $\mathcal{I}$):
+*Entropy* $$\mathcal{E}=\sum_{i=1}^{C}-f_i \log f_i$$, as
+well as *Gini impurity*
+$$\mathcal{G}=\sum_{i=1}^{C}f_i (1-f_i)$$, where $C$ denotes the number of
+unique labels and $f_i$ is the frequency of label $i$. Once the impurity
+at the tree nodes has been obtained, the *best split* is
+chosen from a set of possible splits that maximizes the
+*information gain* at the node, i.e.,
+$$\arg\max_{s}\mathcal{IG}(X,s)$$, where $\mathcal{IG}(X,s)$ denotes the
+information gain when the splitting test $s$ partitions the feature
+matrix $X$. Assuming that $s$ partitions $X$ that contains $N$ feature
+vectors into $$X_\text{left}$$ and $$X_\text{right}$$ each including
+$$N_\text{left}$$ and $$N_\text{right}$$ feature vectors, respectively,
+$$\mathcal{IG}(X,s)$$ is given by
+
+$$\mathcal{IG}(X,s)=\mathcal{I}(X)-\frac{N_\text{left}}{N}\mathcal{I}(X_\text{left})-\frac{N_\text{right}}{N}\mathcal{I}(X_\text{right})$$
+
+where $\mathcal{I}\in\{\mathcal{E},\mathcal{G}\}$. In the following we
+discuss the implementation details specific to
+`decision-tree.dml`.
+
+**Input format.** In general implementations of the decision tree
+algorithm do not require categorical features to be dummy coded. For
+improved efficiency and reducing the training time, our implementation
+however assumes dummy coded categorical features and dummy coded class
+labels.
+
+**Tree construction.** Learning a decision tree on large-scale data has
+received some attention in the literature. The current implementation
+includes logic for choosing tests for multiple nodes that belong to the
+same level in the decision tree in parallel (breadth-first expansion)
+and for building entire subtrees under multiple nodes in parallel
+(depth-first subtree building). Empirically it has been demonstrated
+that it is advantageous to perform breadth-first expansion for the nodes
+belonging to the top levels of the tree and to perform depth-first
+subtree building for nodes belonging to the lower levels of the
+tree [[PandaHBB2009]](algorithms-bibliography.html). The parameter `num_samples` controls
+when we switch to depth-first subtree building. Any node in the decision
+tree that receives $\leq$ `num_samples` training examples,
+the subtree under it is built in its entirety in one shot.
+
+**Stopping rule and pruning.** The splitting of data at the internal
+nodes stops when at least one the following criteria is satisfied:
+
+ * the depth of the internal node reaches the input parameter
+ `depth` controlling the maximum depth of the learned
+ tree, or
+ * no candidate split achieves information gain.
+
+This implementation also allows for some automated pruning via the
+argument `num_leaf`. If a node receives $\leq$
+`num_leaf` training examples, then a leaf is built in its
+place.
+
+**Continuous-valued features.** For a continuous-valued feature $j$ the
+number of candidate thresholds $\sigma$ to choose from is of the order
+of the number of examples present in the training set. Since for
+large-scale data this can result in a large number of candidate
+thresholds, the user can limit this number via the arguments
+`bins` which controls the number of candidate thresholds
+considered for each continuous-valued feature. For each
+continuous-valued feature, the implementation computes an equi-height
+histogram to generate one candidate threshold per equi-height bin.
+
+**Categorical features.** In order to determine the best value subset to
+split on in the case of categorical features, this implementation
+greedily includes values from the feature’s domain until the information
+gain stops improving. In particular, for a categorical feature $j$ the
+$|Dom(j)|$ feature values are sorted by impurity and the resulting split
+candidates $|Dom(j)|-1$ are examined; the sequence of feature values
+which results in the maximum information gain is then selected.
+
+**Description of the model.** The learned decision tree is represented
+in a matrix $M$ that contains at least 6 rows. Each column in the matrix
+contains the parameters relevant to a single node in the tree. Note that
+for building the tree model, our implementation splits the feature
+matrix $X$ into $$X_\text{cont}$$ containing continuous-valued features
+and $$X_\text{cat}$$ containing categorical features. In the following,
+the continuous-valued (resp. categorical) feature-ids correspond to the
+indices of the features in $$X_\text{cont}$$ (resp. $$X_\text{cat}$$).
+Moreover, we refer to an internal node as a continuous-valued
+(categorical) node if the feature that this nodes looks at is
+continuous-valued (categorical). Below is a description of what each row
+in the matrix contains.
+
+ * Row 1: stores the node-ids. These ids correspond to the node-ids in
+ a complete binary tree.
+ * Row 2: for internal nodes stores the offsets (the number of columns)
+ in $M$ to the left child, and otherwise `0`.
+ * Row 3: stores the feature index of the feature (id of a
+ continuous-valued feature in $$X_\text{cont}$$ if the feature is
+ continuous-valued or id of a categorical feature in $$X_\text{cat}$$
+ if the feature is categorical) that this node looks at if the node
+ is an internal node, otherwise `0`.
+ * Row 4: store the type of the feature that this node looks at if the
+ node is an internal node: `1` for continuous-valued and `2` for
+ categorical features, otherwise the label this leaf node is supposed
+ to predict.
+ * Row 5: for the internal nodes contains `1` if the feature chosen for
+ the node is continuous-valued, or the size of the subset of values
+ used for splitting at the node stored in rows 6,7,$\ldots$ if the
+ feature chosen for the node is categorical. For the leaf nodes, Row
+ 5 contains the number of misclassified training examples reaching at
+ this node.
+ * Row 6,7,$\ldots$: for the internal nodes, row 6 stores the threshold
+ to which the example’s feature value is compared if the feature
+ chosen for this node is continuous-valued, otherwise if the feature
+ chosen for this node is categorical rows 6,7,$\ldots$ store the
+ value subset chosen for the node. For the leaf nodes, row 6 contains
+ `1` if the node is impure and the number of training examples at the
+ node is greater than `num_leaf`, otherwise `0`.
+
+As an example, [**Figure 2**](algorithms-classification.html#figure2) shows a decision tree with $5$ nodes and
+its matrix representation.
+
+* * *
+
+<a name="figure2" />
+
+**Figure 2**: (a) An example tree and its (b) matrix representation. $x$ denotes an example and $x_j$ is the value of the $j$th continuous-valued (resp. categorical) feature in $$X_\text{cont}$$ (resp. $$X_\text{cat}$$). In this example all leaf nodes are pure and no training example is misclassified.
+
+(a) 
+
+(b)
+
+| | Col 1 | Col 2 | Col 3 | Col 4 | Col 5
+|-------|-------|-------|-------|-------|-------
+| Row 1 | 1 | 2 | 3 | 6 | 7
+| Row 2 | 1 | 0 | 1 | 0 | 0
+| Row 3 | 3 | 5 | 0 | 0 | 0
+| Row 4 | 1 | 1 | 2 | 2 | 1
+| Row 5 | 1 | 0 | 2 | 0 | 0
+| Row 6 | 0.45 | 0 | 2 | 0 | 0
+| Row 7 | | | 3 | |
+
+
+* * *
+
+### Returns
+
+The matrix corresponding to the learned model as well as the training
+accuracy (if requested) is written to a file in the format specified.
+See details where the structure of the model matrix is described. Recall
+that in our implementation $X$ is split into $$X_\text{cont}$$ and
+$$X_\text{cat}$$. If requested, the mappings of the continuous-valued
+feature-ids in $$X_\text{cont}$$ (stored at `S_map`) and the
+categorical feature-ids in $$X_\text{cat}$$ (stored at
+`C_map`) to the global feature-ids in $X$ will be provided.
+Depending on what arguments are provided during invocation, the
+`decision-tree-predict.dml` script may compute one or more of
+predictions, accuracy and confusion matrix in the requested output
+format.
+
+
+* * *
+
+
+## 2.5. Random Forests
+
+### Description
+
+Random forest is one of the most successful machine learning methods for
+classification and regression. It is an ensemble learning method that
+creates a model composed of a set of tree models. This implementation is
+well-suited to handle large-scale data and builds a random forest model
+for classification in parallel.
+
+
+### Usage
+
+**Random Forest**:
+
+ hadoop jar SystemML.jar -f random-forest.dml
+ -nvargs X=<file>
+ Y=<file>
+ R=[file]
+ M=<file>
+ bins=[int]
+ depth=[int]
+ num_leaf=[int]
+ num_samples=[int]
+ num_trees=[int]
+ subsamp_rate=[double]
+ feature_subset=[double]
+ impurity=[Gini|entropy]
+ C=[file]
+ S_map=[file]
+ C_map=[file]
+ fmt=[format]
+
+**Random Forest Prediction**:
+
+ hadoop jar SystemML.jar -f random-forest-predict.dml
+ -nvargs X=<file>
+ Y=[file]
+ R=[file]
+ M=<file>
+ C=[file]
+ P=<file>
+ A=[file]
+ OOB=[file]
+ CM=[file]
+ fmt=[format]
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read the matrix of feature vectors; each row
+constitutes one feature vector. Note that categorical features in $X$
+need to be both recoded and dummy coded.
+
+**Y**: Location (on HDFS) to read the matrix of (categorical) labels that
+correspond to feature vectors in $X$. Note that classes are assumed to
+be both recoded and dummy coded. This argument is optional for
+prediction.
+
+**R**: (default: `" "`) Location (on HDFS) to read matrix $R$ which for each feature
+in $X$ contains column-ids (first column), start indices (second
+column), and end indices (third column). If $R$ is not provided by
+default all features are assumed to be continuous-valued.
+
+**M**: Location (on HDFS) to write matrix $M$ containing the learned random
+forest (see [Decision Trees](algorithms-classification.html#decision-trees) and below for the schema)
+
+**bins**: (default: `20`) Number of thresholds to choose for each continuous-valued
+feature (determined by equi-height binning).
+
+**depth**: (default: `25`) Maximum depth of the learned trees in the random forest model
+
+**num_leaf**: (default: `10`) Parameter that controls pruning. The tree is not expanded if
+a node receives less than `num_leaf` training examples.
+
+**num_samples**: (default: `3000`) Parameter that decides when to switch to in-memory building
+of the subtrees in each tree of the random forest model. If a node $v$
+receives less than `num_samples` training examples then this
+implementation switches to an in-memory subtree building procedure to
+build the subtree under $v$ in its entirety.
+
+**num_trees**: (default: `10`) Number of trees to be learned in the random forest model
+
+**subsamp_rate**: (default: `1.0`) Parameter controlling the size of each tree in the random
+forest model; samples are selected from a Poisson distribution with
+parameter `subsamp_rate`.
+
+**feature_subset**: (default: `0.5`) Parameter that controls the number of feature used as
+candidates for splitting at each tree node as a power of the number of
+features in the data, i.e., assuming the training set has $D$ features
+$$D^{\tt feature\_subset}$$ are used at each tree node.
+
+**impurity**: (default: `"Gini"`) Impurity measure used at internal nodes of the trees in the
+random forest model for selecting which features to split on. Possible
+value are `entropy` or `Gini`.
+
+**C**: (default: `" "`) Location (on HDFS) to store the number of counts (generated
+according to a Poisson distribution with parameter
+`subsamp_rate`) for each feature vector. Note that this
+argument is optional. If Out-Of-Bag (`OOB`) error estimate needs to be
+computed this parameter is passed as input to
+`random-forest-predict.dml`.
+
+**A**: (default: `" "`) Location (on HDFS) to store the testing accuracy (%) from a
+held-out test set during prediction. Note that this argument is
+optional.
+
+**OOB**: (default: `" "`) Location (on HDFS) to store the Out-Of-Bag (`OOB`) error
+estimate of the training set. Note that the matrix of sample counts
+(stored at `C`) needs to be provided for computing `OOB` error
+estimate. Note that this argument is optional.
+
+**P**: Location (on HDFS) to store predictions for a held-out test set
+
+**CM**: (default: `" "`) Location (on HDFS) to store the confusion matrix computed
+using a held-out test set. Note that this argument is optional.
+
+**S_map**: (default: `" "`) Location (on HDFS) to write the mappings from the
+continuous-valued feature-ids to the global feature-ids in $X$ (see
+below for details). Note that this argument is optional.
+
+**C_map**: (default: `" "`) Location (on HDFS) to write the mappings from the categorical
+feature-ids to the global feature-ids in $X$ (see below for details).
+Note that this argument is optional.
+
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
+
+
+### Examples
+
+**Random Forest**:
+
+ hadoop jar SystemML.jar -f random-forest.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ R=/user/ml/R.csv
+ M=/user/ml/model.csv
+ bins=20
+ depth=25
+ num_leaf=10
+ num_samples=3000
+ num_trees=10
+ impurity=Gini
+ fmt=csv
+
+**Random Forest Prediction**:
+
+To compute predictions:
+
+ hadoop jar SystemML.jar -f random-forest-predict.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ R=/user/ml/R.csv
+ M=/user/ml/model.csv
+ P=/user/ml/predictions.csv
+ A=/user/ml/accuracy.csv
+ CM=/user/ml/confusion.csv
+ fmt=csv
+
+
+### Details
+
+Random forests [[Breiman2001]](algorithms-bibliography.html)
+are learning algorithms for ensembles
+of decision trees. The main idea is to build a number of decision trees
+on bootstrapped training samples, i.e., by taking repeatedly samples
+from a (single) training set. Moreover, instead of considering all the
+features when building the trees only a random subset of the
+features—typically $\approx \sqrt{D}$, where $D$ is the number of
+features—is chosen each time a split test at a tree node is performed.
+This procedure *decorrelates* the trees and makes it less
+prone to overfitting. To build decision trees we utilize the techniques
+discussed in [Decision Trees](algorithms-classification.html#decision-trees) and proposed
+in [[PandaHBB2009]](algorithms-bibliography.html); the implementation details are similar to those of
+the decision trees script. Below we review some features of our
+implementation which differ from `decision-tree.dml`.
+
+**Bootstrapped sampling.** Each decision tree is fitted to a
+bootstrapped training set sampled with replacement (WR). To improve
+efficiency, we generate $N$ sample counts according to a Poisson
+distribution with parameter `subsamp_rate`, where $N$
+denotes the total number of training points. These sample counts
+approximate WR sampling when $N$ is large enough and are generated
+upfront for each decision tree.
+
+**Bagging.** Decision trees suffer from *high variance*
+resulting in different models whenever trained on a random subsets of
+the data points. *Bagging* is a general-purpose method to
+reduce the variance of a statistical learning method like decision
+trees. In the context of decision trees (for classification), for a
+given test feature vector the prediction is computed by taking a
+*majority vote*: the overall prediction is the most
+commonly occurring class among all the tree predictions.
+
+**Out-Of-Bag error estimation.** Note that each bagged tree in a random
+forest model is trained on a subset (around $\frac{2}{3}$) of the
+observations (i.e., feature vectors). The remaining ($\frac{1}{3}$ of
+the) observations not used for training is called the
+*Out-Of-Bag* (`OOB`) observations. This gives us a
+straightforward way to estimate the test error: to predict the class
+label of each test observation $i$ we use the trees in which $i$ was
+`OOB`. Our `random-forest-predict.dml` script provides the `OOB`
+error estimate for a given training set if requested.
+
+**Description of the model.** Similar to decision trees, the learned
+random forest model is presented in a matrix $M$ with at least `7` rows.
+The information stored in the model is similar to that of decision trees
+with the difference that the tree-ids are stored in the second row and
+rows $2,3,\ldots$ from the decision tree model are shifted by one. See
+[Decision Trees](algorithms-classification.html#decision-trees) for a description of the model.
+
+
+### Returns
+
+The matrix corresponding to the learned model is written to a file in
+the format specified. See [Decision Trees](algorithms-classification.html#decision-trees) where the
+details about the structure of the model matrix is described. Similar to
+`decision-tree.dml`, $X$ is split into $$X_\text{cont}$$ and
+$$X_\text{cat}$$. If requested, the mappings of the continuous feature-ids
+in $$X_\text{cont}$$ (stored at `S_map`) as well as the
+categorical feature-ids in $$X_\text{cat}$$ (stored at
+`C_map`) to the global feature-ids in $X$ will be provided.
+The `random-forest-predict.dml` script may compute one or
+more of predictions, accuracy, confusion matrix, and `OOB` error estimate
+in the requested output format depending on the input arguments used.
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2a2520bd/algorithms-clustering.md
----------------------------------------------------------------------
diff --git a/algorithms-clustering.md b/algorithms-clustering.md
index c84a2ff..a443083 100644
--- a/algorithms-clustering.md
+++ b/algorithms-clustering.md
@@ -4,7 +4,6 @@ title: SystemML Algorithms Reference - Clustering
displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
---
-
# 3. Clustering
@@ -75,9 +74,7 @@ not the global minimum and varies depending on the initial centroids. We
implement multiple parallel runs with different initial centroids and
report the best result.
-### Scoring
-
-Our scoring script evaluates the clustering output by comparing it with
+**Scoring.** Our scoring script evaluates the clustering output by comparing it with
a known category assignment. Since cluster labels have no prior
correspondence to the categories, we cannot count “correct” and “wrong”
cluster assignments. Instead, we quantify them in two ways:
@@ -96,30 +93,34 @@ apart is a “false negative” etc.
### Usage
+**K-Means**:
+
hadoop jar SystemML.jar -f Kmeans.dml
- -nvargs X=file
- C=file
- k=int
- runs=int
- maxi=int
- tol=double
- samp=int
- isY=int
- Y=file
- fmt=format
- verb=int
+ -nvargs X=<file>
+ C=[file]
+ k=<int>
+ runs=[int]
+ maxi=[int]
+ tol=[double]
+ samp=[int]
+ isY=[int]
+ Y=[file]
+ fmt=[format]
+ verb=[int]
+
+**K-Means Prediction**:
hadoop jar SystemML.jar -f Kmeans-predict.dml
- -nvargs X=file
- C=file
- spY=file
- prY=file
- fmt=format
- O=file
+ -nvargs X=[file]
+ C=[file]
+ spY=[file]
+ prY=[file]
+ fmt=[format]
+ O=[file]
-### Arguments - K-means
+### Arguments - K-Means
**X**: Location to read matrix $X$ with the input data records as rows
@@ -128,31 +129,31 @@ cluster centroids as rows
**k**: Number of clusters (and centroids)
-**runs**: (default: 10) Number of parallel runs, each run with different initial
+**runs**: (default: `10`) Number of parallel runs, each run with different initial
centroids
-**maxi**: (default: 1000) Maximum number of iterations per run
+**maxi**: (default: `1000`) Maximum number of iterations per run
-**tol**: (default: 0.000001) Tolerance (epsilon) for single-iteration WCSS\_C change ratio
+**tol**: (default: `0.000001`) Tolerance (epsilon) for single-iteration WCSS\_C change ratio
-**samp**: (default: 50) Average number of records per centroid in data samples used
+**samp**: (default: `50`) Average number of records per centroid in data samples used
in the centroid initialization procedure
**Y**: (default: `"Y.mtx"`) Location to store the one-column matrix $Y$ with the best
available mapping of records to clusters (defined by the output
centroids)
-**isY**: (default: 0) 0 = do not write matrix $Y$, 1 = write $Y$
+**isY**: (default: `0`) `0` = do not write matrix $Y$, `1` = write $Y$
**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
`mm`, or `csv`; see read/write functions in
SystemML Language Reference for details.
-**verb**: (default: 0) 0 = do not print per-iteration statistics for
-each run, 1 = print them (the “verbose” option)
+**verb**: (default: `0`) `0` = do not print per-iteration statistics for
+each run, `1` = print them (the “verbose” option)
-### Arguments - K-means Scoring/Prediction
+### Arguments - K-Means Prediction
**X**: (default: `" "`) Location to read matrix $X$ with the input data records as
rows, optional when `prY` input is provided
@@ -174,7 +175,7 @@ categories
**fmt**: (default: `"text"`) Matrix file output format for `prY`, such as
`text`, `mm`, or `csv`; see read/write
-functions in SystemML Language Reference for details
+functions in SystemML Language Reference for details.
**0**: (default: `" "`) Location to write the output statistics defined in
[**Table 6**](algorithms-clustering.html#table6), by default print them to the
@@ -183,6 +184,8 @@ standard output
### Examples
+**K-Means**:
+
hadoop jar SystemML.jar -f Kmeans.dml
-nvargs X=/user/ml/X.mtx
k=5
@@ -201,6 +204,8 @@ standard output
Y=/user/ml/Yout.mtx
verb=1
+**K-Means Prediction**:
+
To predict Y given X and C:
hadoop jar SystemML.jar -f Kmeans-predict.dml
@@ -465,11 +470,11 @@ assignment.
We output the $k$ centroids for the best available clustering,
i. e. whose WCSS is the smallest of all successful runs. The centroids
are written as the rows of the $k\,{\times}\,m$-matrix into the output
-file whose path/name was provided as the “C” input
-argument. If the input parameter “isY” was set
-to 1, we also output the one-column matrix with the cluster
+file whose path/name was provided as the `C` input
+argument. If the input parameter `isY` was set
+to `1`, we also output the one-column matrix with the cluster
assignment for all the records. This assignment is written into the file
-whose path/name was provided as the “Y” input argument. The
+whose path/name was provided as the `Y` input argument. The
best WCSS value, as well as some information about the performance of
the other runs, is printed during the script execution. The scoring
script `Kmeans-predict.dml` prints all its results in a
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2a2520bd/algorithms-descriptive-statistics.md
----------------------------------------------------------------------
diff --git a/algorithms-descriptive-statistics.md b/algorithms-descriptive-statistics.md
index 6c56344..eafb88e 100644
--- a/algorithms-descriptive-statistics.md
+++ b/algorithms-descriptive-statistics.md
@@ -4,9 +4,9 @@ title: SystemML Algorithms Reference - Descriptive Statistics
displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
---
-
# 1. Descriptive Statistics
+
Descriptive statistics are used to quantitatively describe the main
characteristics of the data. They provide meaningful summaries computed
over different observations or data records collected in a study. These
@@ -99,7 +99,10 @@ to compute the mean of a categorical attribute like ‘Hair Color’.
### Usage
- hadoop jar SystemML.jar -f Univar-Stats.dml -nvargs X=file TYPES=file STATS=file
+ hadoop jar SystemML.jar -f Univar-Stats.dml
+ -nvargs X=<file>
+ TYPES=<file>
+ STATS=<file>
### Arguments
@@ -119,7 +122,10 @@ be stored. The format of the output matrix is defined by
### Examples
- hadoop jar SystemML.jar -f Univar-Stats.dml -nvargs X=/user/ml/X.mtx TYPES=/user/ml/types.mtx STATS=/user/ml/stats.mtx
+ hadoop jar SystemML.jar -f Univar-Stats.dml
+ -nvargs X=/user/ml/X.mtx
+ TYPES=/user/ml/types.mtx
+ STATS=/user/ml/stats.mtx
* * *
@@ -180,6 +186,8 @@ $v_i = \texttt{X[}i\texttt{,}\,\texttt{idx]}$. Let
$v^s = (v^s_1, v^s_2, \ldots, v^s_n)$ be the same values in the sorted
order, preserving duplicates: $v^s_1 \leq v^s_2 \leq \ldots \leq v^s_n$.
+* * *
+
<a name="figure1" />
**Figure 1**: The computation of quartiles, median, and interquartile mean from the
empirical distribution function of the 10-point
@@ -188,8 +196,9 @@ the graph has height $1{/}n = 0.1$. Values $$q_{25\%}$$, $$q_{50\%}$$, and $$q_
the $1^{\textrm{st}}$, $2^{\textrm{nd}}$, and $3^{\textrm{rd}}$ quartiles correspondingly;
value $\mu$ denotes the median. Values $\phi_1$ and $\phi_2$ show the partial contribution
of border points (quartiles) $v_3=3.7$ and $v_8=6.4$ into the interquartile mean.
-
+
+* * *
#### Central Tendency Measures
@@ -485,7 +494,7 @@ over all categories.
The output matrix containing all computed statistics is of size
$17$ rows and as many columns as in the input matrix `X`. Each row
corresponds to a particular statistic, according to the convention
-specified in Table \[table:univars\]. The first $14$ statistics are
+specified in [**Table 1**](algorithms-descriptive-statistics.html#table1). The first $14$ statistics are
applicable for *scale* columns, and the last $3$ statistics are
applicable for categorical, i.e. nominal and ordinal, columns.
@@ -515,7 +524,13 @@ attributes like ‘Hair Color’.
### Usage
- hadoop jar SystemML.jar -f bivar-stats.dml -nvargs X=file index1=file index2=file types1=file types2=file OUTDIR=directory
+ hadoop jar SystemML.jar -f bivar-stats.dml
+ -nvargs X=<file>
+ index1=<file>
+ index2=<file>
+ types1=<file>
+ types2=<file>
+ OUTDIR=<directory>
### Arguments
@@ -559,7 +574,13 @@ are defined in [**Table 2**](algorithms-descriptive-statistics.html#table2).
### Examples
- hadoop jar SystemML.jar -f bivar-stats.dml -nvargs X=/user/ml/X.mtx index1=/user/ml/S1.mtx index2=/user/ml/S2.mtx types1=/user/ml/K1.mtx types2=/user/ml/K2.mtx OUTDIR=/user/ml/stats.mtx
+ hadoop jar SystemML.jar -f bivar-stats.dml
+ -nvargs X=/user/ml/X.mtx
+ index1=/user/ml/S1.mtx
+ index2=/user/ml/S2.mtx
+ types1=/user/ml/K1.mtx
+ types2=/user/ml/K2.mtx
+ OUTDIR=/user/ml/stats.mtx
* * *
@@ -609,7 +630,7 @@ types in the same order as their indices in the `index1` and `index2`
files. The types must be provided as per the following convention:
1 = scale, 2 = nominal, 3 = ordinal.
-The script orgainizes its results into (potentially) four output
+The script organizes its results into (potentially) four output
matrices, one per each type combination. The types of bivariate
statistics are defined using the types of the columns that were used for
their arguments, with “ordinal” sometimes retrogressing to “nominal.”
@@ -1025,7 +1046,15 @@ becomes reversed and amplified (from $+0.1$ to $-0.5$) if we ignore the months.
### Usage
- hadoop jar SystemML.jar -f stratstats.dml -nvargs X=file Xcid=file Y=file Ycid=file S=file Scid=int O=file fmt=format
+ hadoop jar SystemML.jar -f stratstats.dml
+ -nvargs X=<file>
+ Xcid=[file]
+ Y=[file]
+ Ycid=[file]
+ S=[file]
+ Scid=[int]
+ O=<file>
+ fmt=[format]
### Arguments
@@ -1033,25 +1062,25 @@ becomes reversed and amplified (from $+0.1$ to $-0.5$) if we ignore the months.
**X**: Location (on HDFS) to read matrix $X$ whose columns we want to use as
the 1-st covariate (i.e. as the feature variable)
-**Xcid**: (default: " ") Location to read the single-row matrix that lists all index
+**Xcid**: (default: `" "`) Location to read the single-row matrix that lists all index
numbers of the $X$-columns used as the 1-st covariate; the default value
means “use all $X$-columns”
-**Y**: (default: " ") Location to read matrix $Y$ whose columns we want to use as
+**Y**: (default: `" "`) Location to read matrix $Y$ whose columns we want to use as
the 2-nd covariate (i.e. as the response variable); the default value
means “use $X$ in place of $Y$”
-**Ycid**: (default: " ") Location to read the single-row matrix that lists all index
+**Ycid**: (default: `" "`) Location to read the single-row matrix that lists all index
numbers of the $Y$-columns used as the 2-nd covariate; the default value
means “use all $Y$-columns”
-**S**: (default: " ") Location to read matrix $S$ that has the stratum column.
+**S**: (default: `" "`) Location to read matrix $S$ that has the stratum column.
Note: the stratum column must contain small positive integers; all
fractional values are rounded; stratum IDs of value ${\leq}\,0$ or NaN
are treated as missing. The default value for S means “use
$X$ in place of $S$”
-**Scid**: (default: 1) The index number of the stratum column in $S$
+**Scid**: (default: `1`) The index number of the stratum column in $S$
**O**: Location to store the output matrix defined in
[**Table 4**](algorithms-descriptive-statistics.html#table4).
@@ -1204,15 +1233,27 @@ SystemML Language Reference for details.
### Examples
- hadoop jar SystemML.jar -f stratstats.dml -nvargs X=/user/ml/X.mtx Xcid=/user/ml/Xcid.mtx Y=/user/ml/Y.mtx Ycid=/user/ml/Ycid.mtx S=/user/ml/S.mtx Scid=2 O=/user/ml/Out.mtx fmt=csv
+ hadoop jar SystemML.jar -f stratstats.dml
+ -nvargs X=/user/ml/X.mtx
+ Xcid=/user/ml/Xcid.mtx
+ Y=/user/ml/Y.mtx
+ Ycid=/user/ml/Ycid.mtx
+ S=/user/ml/S.mtx Scid=2
+ O=/user/ml/Out.mtx
+ fmt=csv
- hadoop jar SystemML.jar -f stratstats.dml -nvargs X=/user/ml/Data.mtx Xcid=/user/ml/Xcid.mtx Ycid=/user/ml/Ycid.mtx Scid=7 O=/user/ml/Out.mtx
+ hadoop jar SystemML.jar -f stratstats.dml
+ -nvargs X=/user/ml/Data.mtx
+ Xcid=/user/ml/Xcid.mtx
+ Ycid=/user/ml/Ycid.mtx
+ Scid=7
+ O=/user/ml/Out.mtx
### Details
Suppose we have $n$ records of format $(i, x, y)$, where
-$i\in\{1,\ldots, k\}$ is a stratum number and $(x, y)$ are two numerical
+$$i\in\{1,\ldots, k\}$$ is a stratum number and $(x, y)$ are two numerical
covariates. We want to analyze conditional linear relationship between
$y$ and $x$ conditioned by $i$. Note that $x$, but not $y$, may
represent a categorical variable if we assign a numerical value to each
@@ -1220,8 +1261,12 @@ category, for example 0 and 1 for two categories.
We assume a linear regression model for $y$:
-$$y_{i,j} \,=\, \alpha_i + \beta x_{i,j} + {\varepsilon}_{i,j}\,, \quad\textrm{where}\,\,\,\,
-{\varepsilon}_{i,j} \sim Normal(0, \sigma^2)$$
+$$
+\begin{equation}
+y_{i,j} \,=\, \alpha_i + \beta x_{i,j} + {\varepsilon}_{i,j}\,, \quad\textrm{where}\,\,\,\,
+{\varepsilon}_{i,j} \sim Normal(0, \sigma^2)
+\end{equation}
+$$
Here $i = 1\ldots k$ is a stratum number and
$j = 1\ldots n_i$ is a record number in stratum $i$; by $n_i$ we denote
@@ -1238,9 +1283,13 @@ $\beta$ is known, the best estimate for $\alpha_i$ is
$\bar{y}_i - \beta \bar{x}_i$, which gives the prediction error
sum-of-squares of
-$$\sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(y_{i,j} - \beta x_{i,j} - (\bar{y}_i - \beta \bar{x}_i)\big)^2
+$$
+\begin{equation}
+\sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(y_{i,j} - \beta x_{i,j} - (\bar{y}_i - \beta \bar{x}_i)\big)^2
\,\,=\,\, \beta^{2\,}V_x \,-\, 2\beta \,V_{x,y} \,+\, V_y
-\label{eqn:stratsumsq}$$
+\label{eqn:stratsumsq}
+\end{equation}
+$$
where $V_x$, $V_y$, and $V_{x, y}$ are,
correspondingly, the “stratified” sample estimates of variance
@@ -1263,10 +1312,9 @@ closer to $$x_{i,j}$$ and $$y_{i,j}$$ than the global means. The stratified
variance estimates the uncertainty in $$x_{i,j}$$ and $$y_{i,j}$$ given
their stratum $i$.
-Minimizing over $\beta$ the error sum-of-squares
+Minimizing over $\beta$ the error sum-of-squares (2)
gives us the regression slope estimate $\hat{\beta} = V_{x,y} / V_x$,
-with the error sum-of-squares
-becoming the residual sum-of-squares (RSS):
+with (2) becoming the residual sum-of-squares (RSS):
$$\mathrm{RSS} \,\,=\, \,
\sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(y_{i,j} -
@@ -1276,10 +1324,10 @@ $$\mathrm{RSS} \,\,=\, \,
The quantity
$$\hat{R}^2 = V_{x,y}^2 / (V_x V_y)$$, called *$R$-squared*, estimates the
fraction of stratified variance in $$y_{i,j}$$ explained by covariate
-$$x_{i, j}$$ in the linear regression model. We
+$$x_{i, j}$$ in the linear regression model (1). We
define *stratified correlation* as the square root of $\hat{R}^2$ taken
with the sign of $V_{x,y}$. We also use RSS to estimate the residual
-standard deviation $\sigma$ in the linear regression model that models the
+standard deviation $\sigma$ in (1) that models the
prediction error of $$y_{i,j}$$ given $$x_{i,j}$$ and the stratum:
$$\hat{\beta}\, =\, \frac{V_{x,y}}{V_x}; \,\,\,\, \hat{R} \,=\, \frac{V_{x,y}}{\sqrt{V_x V_y}};
[34/47] incubator-systemml git commit: [SYSML-273] Initial DML and
PyDML programming guide
Posted by du...@apache.org.
[SYSML-273] Initial DML and PyDML programming guide
Guide to introduce users to DML and PyDML. Aims to highlight
the similarities and differences of DML and PyDML with
side-by-side comparisons. Covers topics such as script
invocation, configuration files, matrix output formats,
command-line argument, default values, data types,
control statements, built-in functions, user-defined
functions, and matrix operations. DML and PyDML coverage
is not comprehensive but rather aims to convey the basics
that can a user started with DML and PyDML in as short a
time as possible.
The document currently does not include a best practices section.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/3bae2a65
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/3bae2a65
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/3bae2a65
Branch: refs/heads/gh-pages
Commit: 3bae2a65bc75a0df999e5ce241e98c5e42cdf8c7
Parents: 43da6de
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Wed Sep 30 10:34:35 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Wed Sep 30 10:34:35 2015 -0700
----------------------------------------------------------------------
_layouts/global.html | 2 +-
dml-and-pydml-programming-guide.md | 871 +++++++++++++++++++
...bug-configuration-hello-world-main-class.png | Bin 0 -> 241282 bytes
...figuration-hello-world-program-arguments.png | Bin 0 -> 224406 bytes
index.md | 6 +-
js/main.js | 13 +-
6 files changed, 881 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/3bae2a65/_layouts/global.html
----------------------------------------------------------------------
diff --git a/_layouts/global.html b/_layouts/global.html
index 7b2deea..6d50bb1 100644
--- a/_layouts/global.html
+++ b/_layouts/global.html
@@ -36,7 +36,7 @@
<li><a href="http://www.github.com/SparkTC/systemml">SystemML GitHub README</a></li>
<li><a href="quick-start-guide.html">Quick Start Guide</a></li>
- <!-- <li><a href="programming-guide.html">Programming Guide</a></li> -->
+ <li><a href="dml-and-pydml-programming-guide.html">DML and PyDML Programming Guide</a></li>
<li><a href="algorithms-reference.html">Algorithms Reference</a></li>
<li><a href="dml-language-reference.html">DML Language Reference</a></li>
<li class="divider"></li>
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/3bae2a65/dml-and-pydml-programming-guide.md
----------------------------------------------------------------------
diff --git a/dml-and-pydml-programming-guide.md b/dml-and-pydml-programming-guide.md
new file mode 100644
index 0000000..2fae62a
--- /dev/null
+++ b/dml-and-pydml-programming-guide.md
@@ -0,0 +1,871 @@
+---
+layout: global
+title: DML and PyDML Programming Guide
+description: DML and PyDML Programming Guide
+---
+
+* This will become a table of contents (this text will be scraped).
+{:toc}
+
+<br/>
+
+
+# Overview
+
+SystemML enables *flexible*, scalable machine learning. This flexibility is achieved
+through the specification of a high-level declarative machine learning language
+that comes in two flavors, one with an R-like syntax (DML) and one with
+a Python-like syntax (PyDML).
+
+Algorithm scripts written in DML and PyDML can be run on Hadoop, on Spark, or
+in Standalone mode. No script modifications are required to change between modes.
+SystemML automatically performs advanced
+optimizations based on data and cluster characteristics, so much of the need to manually
+tweak algorithms is largely reduced or eliminated.
+
+This SystemML Programming Guide serves as a starting point for writing DML and PyDML
+scripts.
+
+
+# Script Invocation
+
+DML and PyDML scripts can be invoked in a variety of ways. Suppose that we have `hello.dml` and
+`hello.pydml` scripts containing the following:
+
+ print('hello ' + $1)
+
+One way to begin working with SystemML is to build the project and unpack the standalone distribution,
+which features the `runStandaloneSystemML.sh` and `runStandaloneSystemML.bat` scripts. The name of the DML or PyDML script
+is passed as the first argument to these scripts, along with a variety of arguments.
+
+ ./runStandaloneSystemML.sh hello.dml -args world
+ ./runStandaloneSystemML.sh hello.pydml -python -args world
+
+For DML and PyDML script invocations that take multiple arguments, a common technique is to create
+a standard script that invokes `runStandaloneSystemML.sh` or `runStandaloneSystemML.bat` with the arguments specified.
+
+SystemML itself is written in Java and is managed using Maven. As a result, SystemML can readily be
+imported into a standard development environment such as Eclipse.
+The `DMLScript` class serves as the main entrypoint to SystemML. Executing
+`DMLScript` with no arguments displays usage information. A script file can be specified using the `-f` argument.
+
+In Eclipse, a Debug Configuration can be created with `DMLScript` as the Main class and any arguments specified as
+Program arguments. A PyDML script requires the addition of a `-python` switch.
+
+<div class="codetabs">
+
+<div data-lang="Eclipse Debug Configuration - Main" markdown="1">
+
+</div>
+
+<div data-lang="Eclipse Debug Configuration - Arguments" markdown="1">
+
+</div>
+
+</div>
+
+SystemML contains a default set of configuration information. In addition to this, SystemML looks for a default `./SystemML-config.xml` file in the working directory, where overriding configuration information can be specified. Furthermore, a config file can be specified using the `-config` argument, as in this example:
+
+ -f hello.dml -config=src/main/standalone/SystemML-config.xml -args world
+
+When operating in a distributed environment, it is *highly recommended* that cluster-specific configuration information
+is provided to SystemML via a configuration file for optimal performance.
+
+
+# Data Types
+
+SystemML has four value data types. In DML, these are: **double**, **integer**,
+**string**, and **boolean**. In PyDML, these are: **float**, **int**,
+**str**, and **bool**. In normal usage, the data type of a variable is implicit
+based on its value. Mathematical operations typically operate on
+doubles/floats, whereas integers/ints are typically useful for tasks such as
+iteration and accessing elements in a matrix.
+
+<div class="codetabs">
+
+<div data-lang="DML" markdown="1">
+{% highlight r %}
+aDouble = 3.0
+bInteger = 2
+print('aDouble = ' + aDouble)
+print('bInteger = ' + bInteger)
+print('aDouble + bInteger = ' + (aDouble + bInteger))
+print('bInteger ^ 3 = ' + (bInteger ^ 3))
+print('aDouble ^ 2 = ' + (aDouble ^ 2))
+
+cBoolean = TRUE
+print('cBoolean = ' + cBoolean)
+print('(2 < 1) = ' + (2 < 1))
+
+dString = 'Hello'
+eString = dString + ' World'
+print('dString = ' + dString)
+print('eString = ' + eString)
+{% endhighlight %}
+</div>
+
+<div data-lang="PyDML" markdown="1">
+{% highlight python %}
+aFloat = 3.0
+bInt = 2
+print('aFloat = ' + aFloat)
+print('bInt = ' + bInt)
+print('aFloat + bInt = ' + (aFloat + bInt))
+print('bInt ** 3 = ' + (bInt ** 3))
+print('aFloat ** 2 = ' + (aFloat ** 2))
+
+cBool = True
+print('cBool = ' + cBool)
+print('(2 < 1) = ' + (2 < 1))
+
+dStr = 'Hello'
+eStr = dStr + ' World'
+print('dStr = ' + dStr)
+print('eStr = ' + eStr)
+{% endhighlight %}
+</div>
+
+<div data-lang="DML Result" markdown="1">
+ aDouble = 3.0
+ bInteger = 2
+ aDouble + bInteger = 5.0
+ bInteger ^ 3 = 8.0
+ aDouble ^ 2 = 9.0
+ cBoolean = TRUE
+ (2 < 1) = FALSE
+ dString = Hello
+ eString = Hello World
+</div>
+
+<div data-lang="PyDML Result" markdown="1">
+ aFloat = 3.0
+ bInt = 2
+ aFloat + bInt = 5.0
+ bInt ** 3 = 8.0
+ aFloat ** 2 = 9.0
+ cBool = TRUE
+ (2 < 1) = FALSE
+ dStr = Hello
+ eStr = Hello World
+</div>
+
+</div>
+
+
+# Matrix Basics
+
+## Creating a Matrix
+
+A matrix can be created in DML using the **`matrix()`** function and in PyDML using the **`full()`**
+function. In the example below, a matrix element is still considered to be of the matrix data type,
+so the value is cast to a scalar in order to print it. Matrix element values are of type **double**/**float**.
+*Note that matrices index from 1 in both DML and PyDML.*
+
+<div class="codetabs">
+
+<div data-lang="DML" markdown="1">
+{% highlight r %}
+m = matrix("1 2 3 4 5 6 7 8 9 10 11 12", rows=4, cols=3)
+for (i in 1:nrow(m)) {
+ for (j in 1:ncol(m)) {
+ n = m[i,j]
+ print('[' + i + ',' + j + ']:' + as.scalar(n))
+ }
+}
+{% endhighlight %}
+</div>
+
+<div data-lang="PyDML" markdown="1">
+{% highlight python %}
+m = full("1 2 3 4 5 6 7 8 9 10 11 12", rows=4, cols=3)
+for (i in 1:nrow(m)):
+ for (j in 1:ncol(m)):
+ n = m[i,j]
+ print('[' + i + ',' + j + ']:' + scalar(n))
+{% endhighlight %}
+</div>
+
+<div data-lang="Result" markdown="1">
+ [1,1]:1.0
+ [1,2]:2.0
+ [1,3]:3.0
+ [2,1]:4.0
+ [2,2]:5.0
+ [2,3]:6.0
+ [3,1]:7.0
+ [3,2]:8.0
+ [3,3]:9.0
+ [4,1]:10.0
+ [4,2]:11.0
+ [4,3]:12.0
+</div>
+
+</div>
+
+For additional information about the **`matrix()`** and **`full()`** functions, please see the
+DML Language Reference ([Matrix Construction](dml-language-reference.html#matrix-construction-manipulation-and-aggregation-built-in-functions)) and the
+PyDML Language Reference (Matrix Construction).
+
+
+## Saving a Matrix
+
+A matrix can be saved using the **`write()`** function in DML and the **`save()`** function in PyDML. SystemML supports four
+different formats: **`text`** (`i,j,v`), **`mm`** (`Matrix Market`), **`csv`** (`delimiter-separated values`), and **`binary`**.
+
+<div class="codetabs">
+
+<div data-lang="DML" markdown="1">
+{% highlight r %}
+m = matrix("1 2 3 0 0 0 7 8 9 0 0 0", rows=4, cols=3)
+write(m, "m.txt", format="text")
+write(m, "m.mm", format="mm")
+write(m, "m.csv", format="csv")
+write(m, "m.binary", format="binary")
+{% endhighlight %}
+</div>
+
+<div data-lang="PyDML" markdown="1">
+{% highlight python %}
+m = full("1 2 3 0 0 0 7 8 9 0 0 0", rows=4, cols=3)
+save(m, "m.txt", format="text")
+save(m, "m.mm", format="mm")
+save(m, "m.csv", format="csv")
+save(m, "m.binary", format="binary")
+{% endhighlight %}
+</div>
+
+</div>
+
+Saving a matrix automatically creates a metadata file for each format except for Matrix Market, since Matrix Market contains
+metadata within the *.mm file. All formats are text-based except binary. The contents of the resulting files are shown here.
+
+<div class="codetabs">
+
+<div data-lang="m.txt" markdown="1">
+ 1 1 1.0
+ 1 2 2.0
+ 1 3 3.0
+ 3 1 7.0
+ 3 2 8.0
+ 3 3 9.0
+</div>
+
+<div data-lang="m.txt.mtd" markdown="1">
+ {
+ "data_type": "matrix"
+ ,"value_type": "double"
+ ,"rows": 4
+ ,"cols": 3
+ ,"nnz": 6
+ ,"format": "text"
+ ,"description": { "author": "SystemML" }
+ }
+</div>
+
+<div data-lang="m.mm" markdown="1">
+ %%MatrixMarket matrix coordinate real general
+ 4 3 6
+ 1 1 1.0
+ 1 2 2.0
+ 1 3 3.0
+ 3 1 7.0
+ 3 2 8.0
+ 3 3 9.0
+</div>
+
+<div data-lang="m.csv" markdown="1">
+ 1.0,2.0,3.0
+ 0,0,0
+ 7.0,8.0,9.0
+ 0,0,0
+</div>
+
+<div data-lang="m.csv.mtd" markdown="1">
+ {
+ "data_type": "matrix"
+ ,"value_type": "double"
+ ,"rows": 4
+ ,"cols": 3
+ ,"nnz": 6
+ ,"format": "csv"
+ ,"header": false
+ ,"sep": ","
+ ,"description": { "author": "SystemML" }
+ }
+</div>
+
+<div data-lang="m.binary" markdown="1">
+ Not text-based
+</div>
+
+<div data-lang="m.binary.mtd" markdown="1">
+ {
+ "data_type": "matrix"
+ ,"value_type": "double"
+ ,"rows": 4
+ ,"cols": 3
+ ,"rows_in_block": 1000
+ ,"cols_in_block": 1000
+ ,"nnz": 6
+ ,"format": "binary"
+ ,"description": { "author": "SystemML" }
+ }
+</div>
+
+</div>
+
+
+## Loading a Matrix
+
+A matrix can be loaded using the **`read()`** function in DML and the **`load()`** function in PyDML. As with saving, SystemML supports four
+formats: **`text`** (`i,j,v`), **`mm`** (`Matrix Market`), **`csv`** (`delimiter-separated values`), and **`binary`**. To read a file, a corresponding
+metadata file is required, except for the Matrix Market format.
+
+<div class="codetabs">
+
+<div data-lang="DML" markdown="1">
+{% highlight r %}
+m = read("m.txt")
+print("min:" + min(m))
+print("max:" + max(m))
+print("sum:" + sum(m))
+mRowSums = rowSums(m)
+for (i in 1:nrow(mRowSums)) {
+ print("row " + i + " sum:" + as.scalar(mRowSums[i,1]))
+}
+mColSums = colSums(m)
+for (i in 1:ncol(mColSums)) {
+ print("col " + i + " sum:" + as.scalar(mColSums[1,i]))
+}
+{% endhighlight %}
+</div>
+
+<div data-lang="PyDML" markdown="1">
+{% highlight python %}
+m = load("m.txt")
+print("min:" + min(m))
+print("max:" + max(m))
+print("sum:" + sum(m))
+mRowSums = rowSums(m)
+for (i in 1:nrow(mRowSums)):
+ print("row " + i + " sum:" + scalar(mRowSums[i,1]))
+mColSums = colSums(m)
+for (i in 1:ncol(mColSums)):
+ print("col " + i + " sum:" + scalar(mColSums[1,i]))
+{% endhighlight %}
+</div>
+
+<div data-lang="Result" markdown="1">
+ min:0.0
+ max:9.0
+ sum:30.0
+ row 1 sum:6.0
+ row 2 sum:0.0
+ row 3 sum:24.0
+ row 4 sum:0.0
+ col 1 sum:8.0
+ col 2 sum:10.0
+ col 3 sum:12.0
+</div>
+
+</div>
+
+
+## Matrix Operations
+
+DML and PyDML offer a rich set of operators and built-in functions to perform various operations on matrices and scalars.
+Operators and built-in functions are described in great detail in the DML Language Reference
+([Expressions](dml-language-reference.html#expressions), [Built-In Functions](dml-language-reference.html#built-in-functions))
+and the PyDML Language Reference
+(Expressions, Built-In Functions).
+
+In this example, we create a matrix A. Next, we create another matrix B by adding 4 to each element in A. Next, we flip
+B by taking its transpose. We then multiply A and B, represented by matrix C. We create a matrix D with the same number
+of rows and columns as C, and initialize its elements to 5. We then subtract D from C and divide the values of its elements
+by 2 and assign the resulting matrix to D.
+
+This example also shows a user-defined function called `printMatrix()`, which takes a string and matrix as arguments and returns
+nothing.
+
+<div class="codetabs">
+
+<div data-lang="DML" markdown="1">
+{% highlight r %}
+printMatrix = function(string which, matrix[double] mat) {
+ print(which)
+ for (i in 1:nrow(mat)) {
+ colVals = '| '
+ for (j in 1:ncol(mat)) {
+ n = mat[i,j]
+ colVals = colVals + as.scalar(n) + ' | '
+ }
+ print(colVals)
+ }
+}
+
+A = matrix("1 2 3 4 5 6", rows=3, cols=2)
+z = printMatrix('Matrix A:', A)
+B = A + 4
+B = t(B)
+z = printMatrix('Matrix B:', B)
+C = A %*% B
+z = printMatrix('Matrix C:', C)
+D = matrix(5, rows=nrow(C), cols=ncol(C))
+D = (C - D) / 2
+z = printMatrix('Matrix D:', D)
+
+{% endhighlight %}
+</div>
+
+<div data-lang="PyDML" markdown="1">
+{% highlight python %}
+def printMatrix(which: str, mat: matrix[float]):
+ print(which)
+ for (i in 1:nrow(mat)):
+ colVals = '| '
+ for (j in 1:ncol(mat)):
+ n = mat[i,j]
+ colVals = colVals + scalar(n) + ' | '
+ print(colVals)
+
+A = full("1 2 3 4 5 6", rows=3, cols=2)
+z = printMatrix('Matrix A:', A)
+B = A + 4
+B = transpose(B)
+z = printMatrix('Matrix B:', B)
+C = dot(A, B)
+z = printMatrix('Matrix C:', C)
+D = full(5, rows=nrow(C), cols=ncol(C))
+D = (C - D) / 2
+z = printMatrix('Matrix D:', D)
+
+{% endhighlight %}
+</div>
+
+<div data-lang="Result" markdown="1">
+ Matrix A:
+ | 1.0 | 2.0 |
+ | 3.0 | 4.0 |
+ | 5.0 | 6.0 |
+ Matrix B:
+ | 5.0 | 7.0 | 9.0 |
+ | 6.0 | 8.0 | 10.0 |
+ Matrix C:
+ | 17.0 | 23.0 | 29.0 |
+ | 39.0 | 53.0 | 67.0 |
+ | 61.0 | 83.0 | 105.0 |
+ Matrix D:
+ | 6.0 | 9.0 | 12.0 |
+ | 17.0 | 24.0 | 31.0 |
+ | 28.0 | 39.0 | 50.0 |
+</div>
+
+</div>
+
+
+## Matrix Indexing
+
+The elements in a matrix can be accessed by their row and column indices. In the example below, we have 3x3 matrix A.
+First, we access the element at the third row and third column. Next, we obtain a row slice (vector) of the matrix by
+specifying row 2 and leaving the column blank. We obtain a column slice (vector) by leaving the row blank and specifying
+column 3. After that, we obtain a submatrix via range indexing, where we specify rows 2 to 3, separated by a colon, and columns
+1 to 2, separated by a colon.
+
+<div class="codetabs">
+
+<div data-lang="DML" markdown="1">
+{% highlight r %}
+printMatrix = function(string which, matrix[double] mat) {
+ print(which)
+ for (i in 1:nrow(mat)) {
+ colVals = '| '
+ for (j in 1:ncol(mat)) {
+ n = mat[i,j]
+ colVals = colVals + as.scalar(n) + ' | '
+ }
+ print(colVals)
+ }
+}
+
+A = matrix("1 2 3 4 5 6 7 8 9", rows=3, cols=3)
+z = printMatrix('Matrix A:', A)
+B = A[3,3]
+z = printMatrix('Matrix B:', B)
+C = A[2,]
+z = printMatrix('Matrix C:', C)
+D = A[,3]
+z = printMatrix('Matrix D:', D)
+E = A[2:3,1:2]
+z = printMatrix('Matrix E:', E)
+
+{% endhighlight %}
+</div>
+
+<div data-lang="PyDML" markdown="1">
+{% highlight python %}
+def printMatrix(which: str, mat: matrix[float]):
+ print(which)
+ for (i in 1:nrow(mat)):
+ colVals = '| '
+ for (j in 1:ncol(mat)):
+ n = mat[i,j]
+ colVals = colVals + scalar(n) + ' | '
+ print(colVals)
+
+A = full("1 2 3 4 5 6 7 8 9", rows=3, cols=3)
+z = printMatrix('Matrix A:', A)
+B = A[3,3]
+z = printMatrix('Matrix B:', B)
+C = A[2,]
+z = printMatrix('Matrix C:', C)
+D = A[,3]
+z = printMatrix('Matrix D:', D)
+E = A[2:3,1:2]
+z = printMatrix('Matrix E:', E)
+
+{% endhighlight %}
+</div>
+
+<div data-lang="Result" markdown="1">
+ Matrix A:
+ | 1.0 | 2.0 | 3.0 |
+ | 4.0 | 5.0 | 6.0 |
+ | 7.0 | 8.0 | 9.0 |
+ Matrix B:
+ | 9.0 |
+ Matrix C:
+ | 4.0 | 5.0 | 6.0 |
+ Matrix D:
+ | 3.0 |
+ | 6.0 |
+ | 9.0 |
+ Matrix E:
+ | 4.0 | 5.0 |
+ | 7.0 | 8.0 |
+</div>
+
+</div>
+
+
+# Control Statements
+
+DML and PyDML both feature `if` and `if-else` conditional statements. In addition, DML features `else-if` which avoids the
+need for nested conditional statements.
+
+DML and PyDML feature 3 loop statements: `while`, `for`, and `parfor` (parallel for). In the example, note that the
+`print` statements within the `parfor` loop can occur in any order since the iterations occur in parallel rather than
+sequentially as in a regular `for` loop. The `parfor` statement can include several optional parameters, as described
+in the DML Language Reference ([ParFor Statement](dml-language-reference.html#parfor-statement)) and PyDML Language Reference (ParFor Statement).
+
+<div class="codetabs">
+
+<div data-lang="DML" markdown="1">
+{% highlight r %}
+i = 1
+while (i < 3) {
+ if (i == 1) {
+ print('hello')
+ } else {
+ print('world')
+ }
+ i = i + 1
+}
+
+A = matrix("1 2 3 4 5 6", rows=3, cols=2)
+
+for (i in 1:nrow(A)) {
+ print("for A[" + i + ",1]:" + as.scalar(A[i,1]))
+}
+
+parfor(i in 1:nrow(A)) {
+ print("parfor A[" + i + ",1]:" + as.scalar(A[i,1]))
+}
+{% endhighlight %}
+</div>
+
+<div data-lang="PyDML" markdown="1">
+{% highlight python %}
+i = 1
+while (i < 3):
+ if (i == 1):
+ print('hello')
+ else:
+ print('world')
+ i = i + 1
+
+A = full("1 2 3 4 5 6", rows=3, cols=2)
+
+for (i in 1:nrow(A)):
+ print("for A[" + i + ",1]:" + scalar(A[i,1]))
+
+parfor(i in 1:nrow(A)):
+ print("parfor A[" + i + ",1]:" + scalar(A[i,1]))
+{% endhighlight %}
+</div>
+
+<div data-lang="Result" markdown="1">
+ hello
+ world
+ for A[1,1]:1.0
+ for A[2,1]:3.0
+ for A[3,1]:5.0
+ parfor A[2,1]:3.0
+ parfor A[1,1]:1.0
+ parfor A[3,1]:5.0
+</div>
+
+</div>
+
+
+# User-Defined Functions
+
+Functions encapsulate useful functionality in SystemML. In addition to built-in functions, users can define their own functions.
+Functions take 0 or more parameters and return 0 or more values.
+Currently, if a function returns nothing, it still needs to be assigned to a variable.
+
+<div class="codetabs">
+
+<div data-lang="DML" markdown="1">
+{% highlight r %}
+doSomething = function(matrix[double] mat) return (matrix[double] ret) {
+ additionalCol = matrix(1, rows=nrow(mat), cols=1) # 1x3 matrix with 1 values
+ ret = append(mat, additionalCol) # append column to matrix
+ ret = append(ret, seq(0, 2, 1)) # append column (0,1,2) to matrix
+ ret = append(ret, rowMaxs(ret)) # append column of max row values to matrix
+ ret = append(ret, rowSums(ret)) # append column of row sums to matrix
+}
+
+A = rand(rows=3, cols=2, min=0, max=2) # random 3x2 matrix with values 0 to 2
+B = doSomething(A)
+write(A, "A.csv", format="csv")
+write(B, "B.csv", format="csv")
+{% endhighlight %}
+</div>
+
+<div data-lang="PyDML" markdown="1">
+{% highlight python %}
+def doSomething(mat: matrix[float]) -> (ret: matrix[float]):
+ additionalCol = full(1, rows=nrow(mat), cols=1) # 1x3 matrix with 1 values
+ ret = append(mat, additionalCol) # append column to matrix
+ ret = append(ret, seq(0, 2, 1)) # append column (0,1,2) to matrix
+ ret = append(ret, rowMaxs(ret)) # append column of max row values to matrix
+ ret = append(ret, rowSums(ret)) # append column of row sums to matrix
+
+A = rand(rows=3, cols=2, min=0, max=2) # random 3x2 matrix with values 0 to 2
+B = doSomething(A)
+save(A, "A.csv", format="csv")
+save(B, "B.csv", format="csv")
+{% endhighlight %}
+</div>
+
+</div>
+
+In the above example, a 3x2 matrix of random doubles between 0 and 2 is created using the **`rand()`** function.
+Additional parameters can be passed to **`rand()`** to control sparsity and other matrix characteristics.
+
+Matrix A is passed to the `doSomething` function. A column of 1 values is appended to the matrix. A column
+consisting of the values `(0, 1, 2)` is appended to the matrix. Next, a column consisting of the maximum row values
+is appended to the matrix. A column consisting of the row sums is appended to the matrix, and this resulting
+matrix is returned to variable B. Matrix A is output to the `A.csv` file and matrix B is saved as the `B.csv` file.
+
+
+<div class="codetabs">
+
+<div data-lang="A.csv" markdown="1">
+ 1.6091961493071,0.7088614208099939
+ 0.5984862383600267,1.5732118950764993
+ 0.2947607068519842,1.9081406573366781
+</div>
+
+<div data-lang="B.csv" markdown="1">
+ 1.6091961493071,0.7088614208099939,1.0,0,1.6091961493071,4.927253719424194
+ 0.5984862383600267,1.5732118950764993,1.0,1.0,1.5732118950764993,5.744910028513026
+ 0.2947607068519842,1.9081406573366781,1.0,2.0,2.0,7.202901364188662
+</div>
+
+</div>
+
+
+# Command-Line Arguments and Default Values
+
+Command-line arguments can be passed to DML and PyDML scripts either as named arguments or as positional arguments. Named
+arguments are the preferred technique. Named arguments can be passed utilizing the `-nvargs` switch, and positional arguments
+can be passed using the `-args` switch.
+
+Default values can be set using the **`ifdef()`** function.
+
+In the example below, a matrix is read from the file system using named argument `M`. The number of rows to print is specified
+using the `rowsToPrint` argument, which defaults to 2 if no argument is supplied. Likewise, the number of columns is
+specified using `colsToPrint` with a default value of 2.
+
+<div class="codetabs">
+
+<div data-lang="DML" markdown="1">
+{% highlight r %}
+
+fileM = $M
+
+numRowsToPrint = ifdef($rowsToPrint, 2) # default to 2
+numColsToPrint = ifdef($colsToPrint, 2) # default to 2
+
+m = read(fileM)
+
+for (i in 1:numRowsToPrint) {
+ for (j in 1:numColsToPrint) {
+ print('[' + i + ',' + j + ']:' + as.scalar(m[i,j]))
+ }
+}
+
+{% endhighlight %}
+</div>
+
+<div data-lang="PyDML" markdown="1">
+{% highlight python %}
+
+fileM = $M
+
+numRowsToPrint = ifdef($rowsToPrint, 2) # default to 2
+numColsToPrint = ifdef($colsToPrint, 2) # default to 2
+
+m = load(fileM)
+
+for (i in 1:numRowsToPrint):
+ for (j in 1:numColsToPrint):
+ print('[' + i + ',' + j + ']:' + scalar(m[i,j]))
+
+{% endhighlight %}
+</div>
+
+<div data-lang="DML Named Arguments and Results" markdown="1">
+ Example #1 Arguments:
+ -f ex.dml -nvargs M=M.txt rowsToPrint=1 colsToPrint=3
+
+ Example #1 Results:
+ [1,1]:1.0
+ [1,2]:2.0
+ [1,3]:3.0
+
+ Example #2 Arguments:
+ -f ex.dml -nvargs M=M.txt
+
+ Example #2 Results:
+ [1,1]:1.0
+ [1,2]:2.0
+ [2,1]:0.0
+ [2,2]:0.0
+
+</div>
+
+<div data-lang="PyDML Named Arguments and Results" markdown="1">
+ Example #1 Arguments:
+ -f ex.pydml -python -nvargs M=M.txt rowsToPrint=1 colsToPrint=3
+
+ Example #1 Results:
+ [1,1]:1.0
+ [1,2]:2.0
+ [1,3]:3.0
+
+ Example #2 Arguments:
+ -f ex.pydml -python -nvargs M=M.txt
+
+ Example #2 Results:
+ [1,1]:1.0
+ [1,2]:2.0
+ [2,1]:0.0
+ [2,2]:0.0
+
+</div>
+
+</div>
+
+Here, we see identical functionality but with positional arguments.
+
+<div class="codetabs">
+
+<div data-lang="DML" markdown="1">
+{% highlight r %}
+
+fileM = $1
+
+numRowsToPrint = ifdef($2, 2) # default to 2
+numColsToPrint = ifdef($3, 2) # default to 2
+
+m = read(fileM)
+
+for (i in 1:numRowsToPrint) {
+ for (j in 1:numColsToPrint) {
+ print('[' + i + ',' + j + ']:' + as.scalar(m[i,j]))
+ }
+}
+
+{% endhighlight %}
+</div>
+
+<div data-lang="PyDML" markdown="1">
+{% highlight python %}
+
+fileM = $1
+
+numRowsToPrint = ifdef($2, 2) # default to 2
+numColsToPrint = ifdef($3, 2) # default to 2
+
+m = load(fileM)
+
+for (i in 1:numRowsToPrint):
+ for (j in 1:numColsToPrint):
+ print('[' + i + ',' + j + ']:' + scalar(m[i,j]))
+
+{% endhighlight %}
+</div>
+
+<div data-lang="DML Positional Arguments and Results" markdown="1">
+ Example #1 Arguments:
+ -f ex.dml -args M.txt 1 3
+
+ Example #1 Results:
+ [1,1]:1.0
+ [1,2]:2.0
+ [1,3]:3.0
+
+ Example #2 Arguments:
+ -f ex.dml -args M.txt
+
+ Example #2 Results:
+ [1,1]:1.0
+ [1,2]:2.0
+ [2,1]:0.0
+ [2,2]:0.0
+
+</div>
+
+<div data-lang="PyDML Positional Arguments and Results" markdown="1">
+ Example #1 Arguments:
+ -f ex.pydml -python -args M.txt 1 3
+
+ Example #1 Results:
+ [1,1]:1.0
+ [1,2]:2.0
+ [1,3]:3.0
+
+ Example #2 Arguments:
+ -f ex.pydml -python -args M.txt
+
+ Example #2 Results:
+ [1,1]:1.0
+ [1,2]:2.0
+ [2,1]:0.0
+ [2,2]:0.0
+
+</div>
+
+</div>
+
+
+# Additional Information
+
+The [DML Language Reference](dml-language-reference.html) and PyDML Language Reference contain highly detailed information regard DML
+and PyDML.
+
+In addition, many excellent examples of DML and PyDML can be found in the `system-ml/scripts` and
+`system-ml/test/scripts/applications` directories.
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/3bae2a65/img/dml-and-pydml-programming-guide/dmlscript-debug-configuration-hello-world-main-class.png
----------------------------------------------------------------------
diff --git a/img/dml-and-pydml-programming-guide/dmlscript-debug-configuration-hello-world-main-class.png b/img/dml-and-pydml-programming-guide/dmlscript-debug-configuration-hello-world-main-class.png
new file mode 100644
index 0000000..934252e
Binary files /dev/null and b/img/dml-and-pydml-programming-guide/dmlscript-debug-configuration-hello-world-main-class.png differ
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/3bae2a65/img/dml-and-pydml-programming-guide/dmlscript-debug-configuration-hello-world-program-arguments.png
----------------------------------------------------------------------
diff --git a/img/dml-and-pydml-programming-guide/dmlscript-debug-configuration-hello-world-program-arguments.png b/img/dml-and-pydml-programming-guide/dmlscript-debug-configuration-hello-world-program-arguments.png
new file mode 100644
index 0000000..e7e2ecc
Binary files /dev/null and b/img/dml-and-pydml-programming-guide/dmlscript-debug-configuration-hello-world-program-arguments.png differ
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/3bae2a65/index.md
----------------------------------------------------------------------
diff --git a/index.md b/index.md
index 70efd1d..4851630 100644
--- a/index.md
+++ b/index.md
@@ -16,7 +16,7 @@ For more information about SystemML, please consult the following references:
* [SystemML GitHub README](http://www.github.com/SparkTC/systemml)
* [Quick Start Guide](quick-start-guide.html)
-<!-- * [Programming Guide](programming-guide.html) -->
+* [DML and PyDML Programming Guide](dml-and-pydml-programming-guide.html)
* [Algorithms Reference](algorithms-reference.html)
-* [DML (Declarative Machine Learning) Language Reference](dml-language-reference.html)
-* PYDML (Python-Like Declarative Machine Learning) Language Reference - **Coming Soon**
+* [DML (R-like Declarative Machine Learning) Language Reference](dml-language-reference.html)
+* PyDML (Python-Like Declarative Machine Learning) Language Reference - **Coming Soon**
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/3bae2a65/js/main.js
----------------------------------------------------------------------
diff --git a/js/main.js b/js/main.js
index f5d66b1..28e7187 100644
--- a/js/main.js
+++ b/js/main.js
@@ -39,18 +39,19 @@ function codeTabs() {
var lang = $(this).data("lang");
var image = $(this).data("image");
var notabs = $(this).data("notabs");
- var capitalizedLang = lang.substr(0, 1).toUpperCase() + lang.substr(1);
- var id = "tab_" + lang + "_" + counter;
+ var langConv = lang.replace(/\./g, "_");
+ langConv = langConv.replace(/ /g, "_");
+ var id = "tab_" + langConv + "_" + counter;
$(this).attr("id", id);
if (image != null && langImages[lang]) {
var buttonLabel = "<img src='" +langImages[lang] + "' alt='" + capitalizedLang + "' />";
} else if (notabs == null) {
- var buttonLabel = "<b>" + capitalizedLang + "</b>";
+ var buttonLabel = "<b>" + lang + "</b>";
} else {
var buttonLabel = ""
}
tabBar.append(
- '<li><a class="tab_' + lang + '" href="#' + id + '">' + buttonLabel + '</a></li>'
+ '<li><a class="tab_' + langConv + '" href="#' + id + '">' + buttonLabel + '</a></li>'
);
});
@@ -59,11 +60,9 @@ function codeTabs() {
counter++;
});
$("ul.nav-tabs a").click(function (e) {
- // Toggling a tab should switch all tabs corresponding to the same language
- // while retaining the scroll position
e.preventDefault();
var scrollOffset = $(this).offset().top - $(document).scrollTop();
- $("." + $(this).attr('class')).tab('show');
+ $(this).tab('show'); // only switch current tab - makes comparisons feel slightly better on this page
$(document).scrollTop($(this).offset().top - scrollOffset);
});
}
[44/47] incubator-systemml git commit: [SYSML-364] Remove snapshot
references in MLContext guide
Posted by du...@apache.org.
[SYSML-364] Remove snapshot references in MLContext guide
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/d80220f9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/d80220f9
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/d80220f9
Branch: refs/heads/gh-pages
Commit: d80220f985a299a7f3950ced274448662689d546
Parents: ad66f73
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Sat Oct 31 23:44:23 2015 -0300
Committer: Luciano Resende <lr...@apache.org>
Committed: Sat Oct 31 23:44:23 2015 -0300
----------------------------------------------------------------------
mlcontext-programming-guide.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d80220f9/mlcontext-programming-guide.md
----------------------------------------------------------------------
diff --git a/mlcontext-programming-guide.md b/mlcontext-programming-guide.md
index 5f3155a..c7d415d 100644
--- a/mlcontext-programming-guide.md
+++ b/mlcontext-programming-guide.md
@@ -26,13 +26,13 @@ To use SystemML with the Spark Shell, the SystemML jar can be referenced using t
Instructions to build the SystemML jar can be found in the [SystemML GitHub README](http://www.github.com/SparkTC/systemml).
{% highlight bash %}
-./bin/spark-shell --executor-memory 4G --driver-memory 4G --jars system-ml-5.2-SNAPSHOT.jar
+./bin/spark-shell --executor-memory 4G --driver-memory 4G --jars SystemML.jar
{% endhighlight %}
Here is an example of Spark Shell with SystemML and YARN.
{% highlight bash %}
-./bin/spark-shell --master yarn-client --num-executors 3 --driver-memory 5G --executor-memory 5G --executor-cores 4 --jars system-ml-5.2-SNAPSHOT.jar
+./bin/spark-shell --master yarn-client --num-executors 3 --driver-memory 5G --executor-memory 5G --executor-cores 4 --jars SystemML.jar
{% endhighlight %}
@@ -511,7 +511,7 @@ this demonstration, it features `SPARK_HOME`, `SPARK_SUBMIT_OPTIONS`, and `ZEPPE
environment variables:
export SPARK_HOME=/Users/example/spark-1.5.1-bin-hadoop2.6
- export SPARK_SUBMIT_OPTIONS="--jars $/Users/example/systemml/system-ml/target/system-ml-5.2-SNAPSHOT.jar"
+ export SPARK_SUBMIT_OPTIONS="--jars $/Users/example/systemml/system-ml/target/SystemML.jar"
export ZEPPELIN_SPARK_USEHIVECONTEXT=false
Start Zeppelin using the `zeppelin.sh` script:
[45/47] incubator-systemml git commit: [SYSML-359] Link to Debugger
Guide
Posted by du...@apache.org.
[SYSML-359] Link to Debugger Guide
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/bf8d44eb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/bf8d44eb
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/bf8d44eb
Branch: refs/heads/gh-pages
Commit: bf8d44eb7fb5eac0197db990278bc25130e7949b
Parents: d80220f
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Tue Nov 3 10:01:57 2015 -0300
Committer: Luciano Resende <lr...@apache.org>
Committed: Tue Nov 3 10:01:57 2015 -0300
----------------------------------------------------------------------
_layouts/global.html | 1 +
debugger-guide.md | 27 +++++++++++++--------------
index.md | 1 +
3 files changed, 15 insertions(+), 14 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/bf8d44eb/_layouts/global.html
----------------------------------------------------------------------
diff --git a/_layouts/global.html b/_layouts/global.html
index 06d8f17..a075461 100644
--- a/_layouts/global.html
+++ b/_layouts/global.html
@@ -38,6 +38,7 @@
<li><a href="quick-start-guide.html">Quick Start Guide</a></li>
<li><a href="dml-and-pydml-programming-guide.html">DML and PyDML Programming Guide</a></li>
<li><a href="mlcontext-programming-guide.html">MLContext Programming Guide</a></li>
+ <li><a href="debugger-guide.html">Debugger Guide</a></li>
<li><a href="algorithms-reference.html">Algorithms Reference</a></li>
<li><a href="dml-language-reference.html">DML Language Reference</a></li>
<li class="divider"></li>
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/bf8d44eb/debugger-guide.md
----------------------------------------------------------------------
diff --git a/debugger-guide.md b/debugger-guide.md
index f343a20..c7067df 100644
--- a/debugger-guide.md
+++ b/debugger-guide.md
@@ -1,12 +1,11 @@
---
layout: global
title: SystemML Debugger Guide
-displayTitle: SystemML Debugger Guide
-displayTitle: <a href="debugger-guide.html">SystemML Debugger Guide</a>
+description: SystemML Debugger Guide
---
-# SystemML Debugger
+## Overview
SystemML supports DML script-level debugging through a command line interface. The SystemML debugger provides functionality typically found in a debugging environment like setting breakpoints, controlling program execution, and inspecting variables. To run a script in debug mode, specify the '-debug' option as shown in below example.
@@ -22,17 +21,17 @@ The following sections describe each command along with example usage.
* [Quit](#quit)
* [List](#list)
* [Step](#step)
- * [Break](#break)
+ * [Break](#break)
* [Delete](#delete)
* [Info break](#info-break)
- * [Continue](#continue)
- * [Run](#run)
- * [Whatis](#whatis)
- * [Print](#print)
- * [Set](#set)
- * [Info frame](#info-frame)
- * [List instruction](#list-instruction)
- * [Step instruction](#step-instruction)
+ * [Continue](#continue)
+ * [Run](#run)
+ * [Whatis](#whatis)
+ * [Print](#print)
+ * [Set](#set)
+ * [Info frame](#info-frame)
+ * [List instruction](#list-instruction)
+ * [Step instruction](#step-instruction)
@@ -237,7 +236,7 @@ Note the run command is not valid if the runtime has already been started. In t
(SystemMLdb) r
Runtime has already started. Try "s" to go to next line, or "c" to continue running your DML script.
(SystemMLdb)
-
+
## Debugger Commands for inspecting or modifying script variables
@@ -372,7 +371,7 @@ So if our test.xml script was executed up to line 4, then the following frame in
(SystemMLdb)
Note only variables that are in scope are included (e.g., the variable C is not part of the frame since not yet in scope).
-
+
## Advanced Debugger Commands
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/bf8d44eb/index.md
----------------------------------------------------------------------
diff --git a/index.md b/index.md
index 21af2b0..a363e5c 100644
--- a/index.md
+++ b/index.md
@@ -18,6 +18,7 @@ For more information about SystemML, please consult the following references:
* [Quick Start Guide](quick-start-guide.html)
* [DML and PyDML Programming Guide](dml-and-pydml-programming-guide.html)
* [MLContext Programming Guide](mlcontext-programming-guide.html)
+* [Debugger Guide](debugger-guide.html)
* [Algorithms Reference](algorithms-reference.html)
* [DML (R-like Declarative Machine Learning) Language Reference](dml-language-reference.html)
* PyDML (Python-Like Declarative Machine Learning) Language Reference - **Coming Soon**
[20/47] incubator-systemml git commit: Dev docs: Overview of matrix
multiplication operators and primitives
Posted by du...@apache.org.
Dev docs: Overview of matrix multiplication operators and primitives
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/66eb331d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/66eb331d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/66eb331d
Branch: refs/heads/gh-pages
Commit: 66eb331dd3e520d59acdd458c1614aa95b78f9ae
Parents: 42373ba
Author: Matthias Boehm <mb...@us.ibm.com>
Authored: Tue Sep 8 09:37:58 2015 -0700
Committer: Matthias Boehm <mb...@us.ibm.com>
Committed: Tue Sep 8 09:37:58 2015 -0700
----------------------------------------------------------------------
devdocs/MatrixMultiplicationOperators.txt | 105 +++++++++++++++++++++++++
1 file changed, 105 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/66eb331d/devdocs/MatrixMultiplicationOperators.txt
----------------------------------------------------------------------
diff --git a/devdocs/MatrixMultiplicationOperators.txt b/devdocs/MatrixMultiplicationOperators.txt
new file mode 100644
index 0000000..ffc38f2
--- /dev/null
+++ b/devdocs/MatrixMultiplicationOperators.txt
@@ -0,0 +1,105 @@
+#####################################################################
+# TITLE: An Overview of Matrix Multiplication Operators in SystemML #
+# DATE MODIFIED: 09/07/2015 #
+#####################################################################
+
+In the following, we give an overview of backend-specific physical matrix multiplication operators in SystemML as well as their internally used matrix multiplication block operations.
+
+A) BASIC MATRIX MULT OPERATORS
+-------------------------------
+
+An AggBinaryOp hop can be compiled into the following physical operators.
+
+* 1) Physical Operators in CP (single node, control program)
+ - MM (basic matrix multiplication) --> mm
+ - MMChain (matrix multiplication chain) --> mmchain
+ - TSMM (transpose-self matrix multiplication) --> tsmm
+ - PMM (permutation matrix multiplication) --> pmm
+
+* 2) Physical Operator in MR (distributed, mapreduce)
+ - MapMM (map-side matrix multiplication, w/|w/o agg) --> mm
+ - MapMMChain (map-side matrix chain multiplication) --> mmchain
+ - TSMM (map-side transpose-self matrix multiplication) --> tsmm
+ - PMM (map-side permutation matrix multiplication) --> pmm
+ - CPMM (cross-product matrix multiplication, 2 jobs) --> mm
+ - RMM (replication-based matrix multiplication, 1 job) --> mm
+
+* 3) Physical Operators in SPARK (distributed, spark)
+ - MapMM (see MR, flatmap/mappartitions/maptopair + --> mm
+ reduce/reducebykey/no_aggregation)
+ - MapMMChain (see MR, mapvalues/maptopair + reduce) --> mmchain
+ - TSMM (see MR, mapvalues + reduce) --> tsmm
+ - PMM (see MR, flatmaptopair + reducebykey) --> pmm
+ - CPMM (see MR, 2 x maptopair + join + maptopair + --> mm
+ reduce/reducebykey)
+ - RMM (see MR, 2 x flatmap + join + maptopair + --> mm
+ reducebykey)
+ - ZIPMM (partitioning-preserving 1-1 zipping mm, --> mm
+ join + mapvalues + reduce)
+
+
+B) COMPLEX MATRIX MULT OPERATORS
+-------------------------------
+
+A QuaternaryOp hop can be compiled into the following physical operators. Note that wsloss, wsigmoid, wdivmm have different semantics though. The main goal of these operators is to prevent the creation of dense "outer" products via selective computation over a sparse driver (sparse matrix and sparse-safe operation).
+
+* 1) Physical Operators in CP (single node, control program)
+ - WSLoss (weighted squared loss) --> wsloss
+ - WSigmoid (weighted sigmoid) --> wsigmoid
+ - WDivMM (weighted divide matrix multiplication) --> wdivmm
+
+* 2) Physical Operator in MR (distributed, mapreduce)
+ - MapWSLoss (map-side weighted squared loss) --> wsloss
+ - RedWSLoss (reduce-side weighted squared loss) --> wsloss
+ - MapWSigmoid (map-side weighted sigmoid) --> wsigmoid
+ - RedWSigmoid (reduce-side weighted sigmoid) --> wsigmoid
+ - MapWDivMM (map-side weighted divide matrix mult) --> wdivmm
+ - RedWDivMM (reduce-side weighted divide matrix mult) --> wdivmm
+
+* 3) Physical Operators in SPARK (distributed, spark)
+ - MapWSLoss (see MR, mappartitions + reduce) --> wsloss
+ - RedWSLoss (see MR, 1/2x flatmaptopair + 1-3x join + --> wsloss
+ maptopair + reduce)
+ - MapWSigmoid (see MR, mappartitions) --> wsigmoid
+ - RedWSigmoid (see MR, 1/2x flatmaptopair + --> wsigmoid
+ 1/2x join + maptopair)
+ - MapWDivMM (see MR, mappartitions + reducebykey ) --> wdivmm
+ - RedWDivMM (see MR, 1/2x flatmaptopair + 1/2x join + --> wdivmm
+ maptopair + reducebykey)
+
+
+C) CORE MATRIX MULT PRIMITIVES LibMatrixMult (incl related script patterns)
+-------------------------------
+* 1) mm (general A %*% B)
+ - sequential / multi-threaded (same block ops, par over rows in A)
+ - dense-dense, dense-sparse, sparse-dense, sparse-sparse, ultra-sparse*
+ - ~20 special cases for matrix-vector, vector-vector, etc
+
+* 2) mmchain ((a) t(X) %*% (X %*% v), (b) t(X) %*% (w * (X %*% v)))
+ - sequential / multi-threaded (same block ops, par over rows in X)
+ - dense / sparse x 2 patterns
+
+* 3) tsmm ((a) t(X) %*% X, (b) X %*% t(X)
+ - sequential / multi-threaded (same block ops, par over rows in R, 2x tasks)
+ - dense / sparse x 2 patterns; special cases for dot products
+
+* 4) pmm (removeEmpty(diag(v), "rows") %*% X)
+ - sequential / multi-threaded (same block ops, par over rows in X)
+ - sparse-sparse, dense-dense, sparse-dense
+
+* 5) wsloss ((a) sum(W*(X-U%*%t(V))^2), (b) sum((X-W*(U%*%t(V)))^2),
+ (c) sum((X -(U%*%t(V)))^2))
+ - sequential / multi-threaded (same block ops, par over rows in W/X)
+ - all dense, sparse-dense factors, sparse/dense-* x 3 patterns
+
+* 6) wsigmoid ((a) W*sigmoid(Y%*%t(X))), (b) W*sigmoid(-(Y%*%t(X))),
+ (c) W*log(sigmoid(Y%*%t(X))), (d) W*log(sigmoid(-(Y%*%t(X)))))
+ - sequential / multi-threaded (same block ops, par over rows in W)
+ - all dense, sparse-dense factors, sparse/dense-* x 4 patterns
+
+* 7) wdivmm ((a) t(t(U)%*%(X/(U%*%t(V)))), (b) (X/(U%*%t(V)))%*%V
+ - sequential / multi-threaded (same block ops, par over rows in X)
+ - all dense, sparse-dense factors, sparse/dense-* x 2 patterns
+
+
+
\ No newline at end of file
[35/47] incubator-systemml git commit: [SYSML-327] Add additional
algorithm tex files to Algorithms Reference
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/53e814f8/Algorithms Reference/StepGLM.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/StepGLM.tex b/Algorithms Reference/StepGLM.tex
new file mode 100644
index 0000000..2afcf59
--- /dev/null
+++ b/Algorithms Reference/StepGLM.tex
@@ -0,0 +1,111 @@
+\subsection{Stepwise Generalized Linear Regression}
+
+\noindent{\bf Description}
+\smallskip
+
+Our stepwise generalized linear regression script selects a model based on the Akaike information criterion (AIC): the model that gives rise to the lowest AIC is provided. Note that currently only the Bernoulli distribution family is supported (see below for details). \\
+
+\smallskip
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+{\tt{}-f }path/\/{\tt{}StepGLM.dml}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} Y=}path/file
+{\tt{} B=}path/file
+{\tt{} S=}path/file
+{\tt{} O=}path/file
+{\tt{} link=}int
+{\tt{} yneg=}double
+{\tt{} icpt=}int
+{\tt{} tol=}double
+{\tt{} disp=}double
+{\tt{} moi=}int
+{\tt{} mii=}int
+{\tt{} thr=}double
+{\tt{} fmt=}format
+
+}
+
+
+\smallskip
+\noindent{\bf Arguments}
+\begin{Description}
+ \item[{\tt X}:]
+ Location (on HDFS) to read the matrix of feature vectors; each row is
+ an example.
+ \item[{\tt Y}:]
+ Location (on HDFS) to read the response matrix, which may have 1 or 2 columns
+ \item[{\tt B}:]
+ Location (on HDFS) to store the estimated regression parameters (the $\beta_j$'s), with the
+ intercept parameter~$\beta_0$ at position {\tt B[}$m\,{+}\,1$, {\tt 1]} if available
+ \item[{\tt S}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to store the selected feature-ids in the order as computed by the algorithm,
+ by default it is standard output.
+ \item[{\tt O}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to write certain summary statistics described in Table~\ref{table:GLM:stats},
+ by default it is standard output.
+ \item[{\tt link}:] (default:\mbox{ }{\tt 2})
+ Link function code to determine the link function~$\eta = g(\mu)$, see Table~\ref{table:commonGLMs}; currently the following link functions are supported: \\
+ {\tt 1} = log,
+ {\tt 2} = logit,
+ {\tt 3} = probit,
+ {\tt 4} = cloglog.
+ \item[{\tt yneg}:] (default:\mbox{ }{\tt 0.0})
+ Response value for Bernoulli ``No'' label, usually 0.0 or -1.0
+ \item[{\tt icpt}:] (default:\mbox{ }{\tt 0})
+ Intercept and shifting/rescaling of the features in~$X$:\\
+ {\tt 0} = no intercept (hence no~$\beta_0$), no shifting/rescaling of the features;\\
+ {\tt 1} = add intercept, but do not shift/rescale the features in~$X$;\\
+ {\tt 2} = add intercept, shift/rescale the features in~$X$ to mean~0, variance~1
+ \item[{\tt tol}:] (default:\mbox{ }{\tt 0.000001})
+ Tolerance (epsilon) used in the convergence criterion: we terminate the outer iterations
+ when the deviance changes by less than this factor; see below for details.
+ \item[{\tt disp}:] (default:\mbox{ }{\tt 0.0})
+ Dispersion parameter, or {\tt 0.0} to estimate it from data
+ \item[{\tt moi}:] (default:\mbox{ }{\tt 200})
+ Maximum number of outer (Fisher scoring) iterations
+ \item[{\tt mii}:] (default:\mbox{ }{\tt 0})
+ Maximum number of inner (conjugate gradient) iterations, or~0 if no maximum
+ limit provided
+ \item[{\tt thr}:] (default:\mbox{ }{\tt 0.01})
+ Threshold to stop the algorithm: if the decrease in the value of the AIC falls below {\tt thr}
+ no further features are being checked and the algorithm stops.
+ \item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+ Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+ see read/write functions in SystemML Language Reference for details.
+\end{Description}
+
+
+\noindent{\bf Details}
+\smallskip
+
+Similar to {\tt StepLinearRegDS.dml} our stepwise GLM script builds a model by iteratively selecting predictive variables
+using a forward selection strategy based on the AIC (\ref{eq:AIC}).
+Note that currently only the Bernoulli distribution family ({\tt fam=2} in Table~\ref{table:commonGLMs}) together with the following link functions are supported: log, logit, probit, and cloglog ({\tt link $\in\{1,2,3,4\}$ } in Table~\ref{table:commonGLMs}).
+
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+Similar to the outputs from {\tt GLM.dml} the stepwise GLM script computes the estimated regression coefficients and stores them in matrix $B$ on HDFS; matrix $B$ follows the same format as the one produced by {\tt GLM.dml} (see Section~\ref{sec:GLM}).
+Additionally, {\tt StepGLM.dml} outputs the variable indices (stored in the 1-column matrix $S$) in the order they have been selected by the algorithm, i.e., $i$th entry in matrix $S$ stores the variable which improves the AIC the most in $i$th iteration.
+If the model with the lowest AIC includes no variables matrix $S$ will be empty.
+Moreover, the estimated summary statistics as defined in Table~\ref{table:GLM:stats}
+are printed out or stored in a file on HDFS (if requested);
+these statistics will be provided only if the selected model is nonempty, i.e., contains at least one variable.
+
+
+\smallskip
+\noindent{\bf Examples}
+\smallskip
+
+{\hangindent=\parindent\noindent\tt
+ \hml -f StepGLM.dml -nvargs X=/user/biadmin/X.mtx Y=/user/biadmin/Y.mtx B=/user/biadmin/B.mtx S=/user/biadmin/selected.csv O=/user/biadmin/stats.csv link=2 yneg=-1.0 icpt=2 tol=0.000001 moi=100 mii=10 thr=0.05 fmt=csv
+
+}
+
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/53e814f8/Algorithms Reference/StepLinRegDS.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/StepLinRegDS.tex b/Algorithms Reference/StepLinRegDS.tex
new file mode 100644
index 0000000..8adf486
--- /dev/null
+++ b/Algorithms Reference/StepLinRegDS.tex
@@ -0,0 +1,101 @@
+\subsection{Stepwise Linear Regression}
+
+\noindent{\bf Description}
+\smallskip
+
+Our stepwise linear regression script selects a linear model based on the Akaike information criterion (AIC):
+the model that gives rise to the lowest AIC is computed. \\
+
+\smallskip
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+{\tt{}-f }path/\/{\tt{}StepLinearRegDS.dml}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} Y=}path/file
+{\tt{} B=}path/file
+{\tt{} S=}path/file
+{\tt{} O=}path/file
+{\tt{} icpt=}int
+{\tt{} thr=}double
+{\tt{} fmt=}format
+
+}
+
+\smallskip
+\noindent{\bf Arguments}
+\begin{Description}
+\item[{\tt X}:]
+Location (on HDFS) to read the matrix of feature vectors, each row contains
+one feature vector.
+\item[{\tt Y}:]
+Location (on HDFS) to read the 1-column matrix of response values
+\item[{\tt B}:]
+Location (on HDFS) to store the estimated regression parameters (the $\beta_j$'s), with the
+intercept parameter~$\beta_0$ at position {\tt B[}$m\,{+}\,1$, {\tt 1]} if available
+\item[{\tt S}:] (default:\mbox{ }{\tt " "})
+Location (on HDFS) to store the selected feature-ids in the order as computed by the algorithm;
+by default the selected feature-ids are forwarded to the standard output.
+\item[{\tt O}:] (default:\mbox{ }{\tt " "})
+Location (on HDFS) to store the CSV-file of summary statistics defined in
+Table~\ref{table:linreg:stats}; by default the summary statistics are forwarded to the standard output.
+\item[{\tt icpt}:] (default:\mbox{ }{\tt 0})
+Intercept presence and shifting/rescaling the features in~$X$:\\
+{\tt 0} = no intercept (hence no~$\beta_0$), no shifting or rescaling of the features;\\
+{\tt 1} = add intercept, but do not shift/rescale the features in~$X$;\\
+{\tt 2} = add intercept, shift/rescale the features in~$X$ to mean~0, variance~1
+\item[{\tt thr}:] (default:\mbox{ }{\tt 0.01})
+Threshold to stop the algorithm: if the decrease in the value of the AIC falls below {\tt thr}
+no further features are being checked and the algorithm stops.
+\item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+see read/write functions in SystemML Language Reference for details.
+\end{Description}
+
+
+\noindent{\bf Details}
+\smallskip
+
+Stepwise linear regression iteratively selects predictive variables in an automated procedure.
+Currently, our implementation supports forward selection: starting from an empty model (without any variable)
+the algorithm examines the addition of each variable based on the AIC as a model comparison criterion. The AIC is defined as
+\begin{equation}
+AIC = -2 \log{L} + 2 edf,\label{eq:AIC}
+\end{equation}
+where $L$ denotes the likelihood of the fitted model and $edf$ is the equivalent degrees of freedom, i.e., the number of estimated parameters.
+This procedure is repeated until including no additional variable improves the model by a certain threshold
+specified in the input parameter {\tt thr}.
+
+For fitting a model in each iteration we use the ``direct solve'' method as in the script {\tt LinearRegDS.dml} discussed in Section~\ref{sec:LinReg}.
+
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+Similar to the outputs from {\tt LinearRegDS.dml} the stepwise linear regression script computes
+the estimated regression coefficients and stores them in matrix $B$ on HDFS.
+The format of matrix $B$ is identical to the one produced by the scripts for linear regression (see Section~\ref{sec:LinReg}).
+Additionally, {\tt StepLinearRegDS.dml} outputs the variable indices (stored in the 1-column matrix $S$)
+in the order they have been selected by the algorithm, i.e., $i$th entry in matrix $S$ corresponds to
+the variable which improves the AIC the most in $i$th iteration.
+If the model with the lowest AIC includes no variables matrix $S$ will be empty (contains one 0).
+Moreover, the estimated summary statistics as defined in Table~\ref{table:linreg:stats}
+are printed out or stored in a file (if requested).
+In the case where an empty model achieves the best AIC these statistics will not be produced.
+
+
+\smallskip
+\noindent{\bf Examples}
+\smallskip
+
+{\hangindent=\parindent\noindent\tt
+ \hml -f StepLinearRegDS.dml -nvargs X=/user/biadmin/X.mtx Y=/user/biadmin/Y.mtx
+ B=/user/biadmin/B.mtx S=/user/biadmin/selected.csv O=/user/biadmin/stats.csv
+ icpt=2 thr=0.05 fmt=csv
+
+}
+
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/53e814f8/Algorithms Reference/SystemML_Algorithms_Reference.bib
----------------------------------------------------------------------
diff --git a/Algorithms Reference/SystemML_Algorithms_Reference.bib b/Algorithms Reference/SystemML_Algorithms_Reference.bib
index 8dce564..878e1dc 100644
--- a/Algorithms Reference/SystemML_Algorithms_Reference.bib
+++ b/Algorithms Reference/SystemML_Algorithms_Reference.bib
@@ -136,3 +136,80 @@
pages = {677--680}
}
+@book{collett2003:kaplanmeier,
+ title={Modelling Survival Data in Medical Research, Second Edition},
+ author={Collett, D.},
+ isbn={9781584883258},
+ lccn={2003040945},
+ series={Chapman \& Hall/CRC Texts in Statistical Science},
+ year={2003},
+ publisher={Taylor \& Francis}
+}
+
+@article{PetoPABCHMMPS1979:kaplanmeier,
+ title = {{Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. analysis and examples.}},
+ author = {Peto, R. and Pike, M. C. and Armitage, P. and Breslow, N. E. and Cox, D. R. and Howard, S. V. and Mantel, N. and McPherson, K. and Peto, J. and Smith, P. G.},
+ journal = {British journal of cancer},
+ number = {1},
+ pages = {1--39},
+ volume = {35},
+ year = {1977}
+}
+
+@inproceedings{ZhouWSP08:als,
+ author = {Yunhong Zhou and
+ Dennis M. Wilkinson and
+ Robert Schreiber and
+ Rong Pan},
+ title = {Large-Scale Parallel Collaborative Filtering for the Netflix Prize},
+ booktitle = {Algorithmic Aspects in Information and Management, 4th International
+ Conference, {AAIM} 2008, Shanghai, China, June 23-25, 2008. Proceedings},
+ pages = {337--348},
+ year = {2008}
+}
+
+@book{BreimanFOS84:dtree,
+ author = {Leo Breiman and
+ J. H. Friedman and
+ R. A. Olshen and
+ C. J. Stone},
+ title = {Classification and Regression Trees},
+ publisher = {Wadsworth},
+ year = {1984},
+ isbn = {0-534-98053-8},
+ timestamp = {Thu, 03 Jan 2002 11:51:52 +0100},
+ biburl = {http://dblp.uni-trier.de/rec/bib/books/wa/BreimanFOS84},
+ bibsource = {dblp computer science bibliography, http://dblp.org}
+}
+
+@article{PandaHBB09:dtree,
+ author = {Biswanath Panda and
+ Joshua Herbach and
+ Sugato Basu and
+ Roberto J. Bayardo},
+ title = {{PLANET:} Massively Parallel Learning of Tree Ensembles with MapReduce},
+ journal = {{PVLDB}},
+ volume = {2},
+ number = {2},
+ pages = {1426--1437},
+ year = {2009},
+ url = {http://www.vldb.org/pvldb/2/vldb09-537.pdf},
+ timestamp = {Wed, 02 Sep 2009 09:21:18 +0200},
+ biburl = {http://dblp.uni-trier.de/rec/bib/journals/pvldb/PandaHBB09},
+ bibsource = {dblp computer science bibliography, http://dblp.org}
+}
+
+@article{Breiman01:rforest,
+ author = {Leo Breiman},
+ title = {Random Forests},
+ journal = {Machine Learning},
+ volume = {45},
+ number = {1},
+ pages = {5--32},
+ year = {2001},
+ url = {http://dx.doi.org/10.1023/A:1010933404324},
+ doi = {10.1023/A:1010933404324},
+ timestamp = {Thu, 26 May 2011 15:25:18 +0200},
+ biburl = {http://dblp.uni-trier.de/rec/bib/journals/ml/Breiman01},
+ bibsource = {dblp computer science bibliography, http://dblp.org}
+}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/53e814f8/Algorithms Reference/SystemML_Algorithms_Reference.pdf
----------------------------------------------------------------------
diff --git a/Algorithms Reference/SystemML_Algorithms_Reference.pdf b/Algorithms Reference/SystemML_Algorithms_Reference.pdf
index 4d4ea6a..4087ba5 100644
Binary files a/Algorithms Reference/SystemML_Algorithms_Reference.pdf and b/Algorithms Reference/SystemML_Algorithms_Reference.pdf differ
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/53e814f8/Algorithms Reference/SystemML_Algorithms_Reference.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/SystemML_Algorithms_Reference.tex b/Algorithms Reference/SystemML_Algorithms_Reference.tex
index 7dccad0..19988fe 100644
--- a/Algorithms Reference/SystemML_Algorithms_Reference.tex
+++ b/Algorithms Reference/SystemML_Algorithms_Reference.tex
@@ -1,10 +1,11 @@
\documentclass[letter]{article}
\usepackage{graphicx,amsmath,amssymb,amsthm,subfigure,color,url,multirow,rotating,comment}
-%\usepackage{tikz}
+\usepackage{tikz}
\usepackage[normalem]{ulem}
\usepackage[np,autolanguage]{numprint}
+\usepackage{tabularx}
-\usepackage[]{hyperref}
+\usepackage[pdftex]{hyperref}
\hypersetup{
unicode=false, % non-Latin characters in Acrobat’s bookmarks
pdftoolbar=true, % show Acrobat’s toolbar?
@@ -92,7 +93,9 @@
\input{NaiveBayes}
-%\input{DecisionTrees}
+\input{DecisionTrees}
+
+\input{RandomForest}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Clustering}
@@ -106,28 +109,36 @@
\input{LinReg}
-\newpage
+\input{StepLinRegDS}
\input{GLM}
-\newpage
+\input{StepGLM}
\input{GLMpredict.tex}
-\newpage
-
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Matrix Factorization}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input{pca}
+\input{ALS.tex}
+
+%%{\color{red}\subsection{GNMF}}
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%{\color{red}\section{Sequence Mining}}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-%%\section{Matrix Factorization}
-%%{\color{red}\subsection{GNMF}}
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\section{Survival Analysis}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\input{KaplanMeier}
+
+\input{Cox}
\bibliographystyle{abbrv}
[41/47] incubator-systemml git commit: [SYSML-347] Invocation Tabs on
Algorithms Reference
Posted by du...@apache.org.
[SYSML-347] Invocation Tabs on Algorithms Reference
Added Hadoop and Spark tabs showing SystemML invocations to
Algorithms Reference.
Updated main.js for handling tabs.
Updated DML and PyDML Programming Guide with respect to tab handling.
Highlighted reference keys in Algorithms Bibliography.
Fixed links in Quick Start Guide.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/b8055201
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/b8055201
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/b8055201
Branch: refs/heads/gh-pages
Commit: b8055201425c0a4073568d6e07d38106138c5d80
Parents: 2a2520b
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Mon Oct 26 09:40:25 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Mon Oct 26 09:40:25 2015 -0700
----------------------------------------------------------------------
algorithms-bibliography.md | 50 +--
algorithms-classification.md | 518 +++++++++++++++++++++++-
algorithms-clustering.md | 136 ++++++-
algorithms-descriptive-statistics.md | 139 ++++++-
algorithms-matrix-factorization.md | 194 ++++++++-
algorithms-regression.md | 641 ++++++++++++++++++++++++------
algorithms-survival-analysis.md | 202 +++++++++-
dml-and-pydml-programming-guide.md | 26 +-
js/main.js | 70 +++-
quick-start-guide.md | 54 +--
10 files changed, 1800 insertions(+), 230 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b8055201/algorithms-bibliography.md
----------------------------------------------------------------------
diff --git a/algorithms-bibliography.md b/algorithms-bibliography.md
index 1012d09..d529c42 100644
--- a/algorithms-bibliography.md
+++ b/algorithms-bibliography.md
@@ -6,103 +6,103 @@ displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference<
# 7. Bibliography
-AcockStavig1979, Alan C. Acock and Gordon
+**[AcockStavig1979]** Alan C. Acock and Gordon
R. Stavig, A Measure of Association for Nonparametric
Statistics, Social Forces, Oxford University
Press, Volume 57, Number 4, June, 1979,
1381--1386.
-AgrawalKSX2002, Rakesh Agrawal and
+**[AgrawalKSX2002]** Rakesh Agrawal and
Jerry Kiernan and Ramakrishnan Srikant and Yirong Xu,
Hippocratic Databases, Proceedings of the 28-th
International Conference on Very Large Data Bases (VLDB 2002),
Hong Kong, China, August 20--23, 2002,
143--154.
-Agresti2002, Alan Agresti, Categorical
+**[Agresti2002]** Alan Agresti, Categorical
Data Analysis, Second Edition, Wiley Series in
Probability and Statistics, Wiley-Interscience
2002, 710.
-AloiseDHP2009, Daniel Aloise and Amit
+**[AloiseDHP2009]** Daniel Aloise and Amit
Deshpande and Pierre Hansen and Preyas Popat, NP-hardness of
Euclidean Sum-of-squares Clustering, Machine Learning,
Kluwer Academic Publishers, Volume 75, Number 2,
May, 2009, 245--248.
-ArthurVassilvitskii2007,
+**[ArthurVassilvitskii2007]**
k-means++: The Advantages of Careful Seeding, David
Arthur and Sergei Vassilvitskii, Proceedings of the 18th
Annual ACM-SIAM Symposium on Discrete Algorithms (SODA 2007),
January 7--9, 2007, New Orleans, LA,
USA, 1027--1035.
-Breiman2001, L. Breiman. Random forests. Machine Learning, 45(1):5–32, 2001.
+**[Breiman2001]** L. Breiman. Random forests. Machine Learning, 45(1):5–32, 2001.
-BreimanFOS1984, L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth, 1984.
+**[BreimanFOS1984]** L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth, 1984.
-Chapelle2007, Olivier Chapelle, Training a Support Vector Machine in the Primal, Neural Computation, 2007.
+**[Chapelle2007]** Olivier Chapelle, Training a Support Vector Machine in the Primal, Neural Computation, 2007.
-Cochran1954, William G. Cochran,
+**[Cochran1954]** William G. Cochran,
Some Methods for Strengthening the Common $\chi^2$ Tests,
Biometrics, Volume 10, Number 4, December
1954, 417--451.
-Collett2003, D. Collett. Modelling Survival Data in Medical Research, Second Edition. Chapman & Hall/CRC Texts in Statistical Science. Taylor & Francis, 2003.
+**[Collett2003]** D. Collett. Modelling Survival Data in Medical Research, Second Edition. Chapman & Hall/CRC Texts in Statistical Science. Taylor & Francis, 2003.
-Gill2000, Jeff Gill, Generalized Linear
+**[Gill2000]** Jeff Gill, Generalized Linear
Models: A Unified Approach, Sage University Papers Series on
Quantitative Applications in the Social Sciences, Number 07-134,
2000, Sage Publications, 101.
-Hartigan1975, John A. Hartigan,
+**[Hartigan1975]** John A. Hartigan,
Clustering Algorithms, John Wiley~&~Sons Inc.,
Probability and Mathematical Statistics, April
1975, 365.
-Hsieh2008, C-J Hsieh, K-W Chang, C-J Lin, S. S. Keerthi and S. Sundararajan, A Dual Coordinate Descent Method for Large-scale Linear SVM, International Conference of Machine Learning (ICML), 2008.
+**[Hsieh2008]** C-J Hsieh, K-W Chang, C-J Lin, S. S. Keerthi and S. Sundararajan, A Dual Coordinate Descent Method for Large-scale Linear SVM, International Conference of Machine Learning (ICML), 2008.
-Lin2008, Chih-Jen Lin and Ruby C. Weng and
+**[Lin2008]** Chih-Jen Lin and Ruby C. Weng and
S. Sathiya Keerthi, Trust Region Newton Method for
Large-Scale Logistic Regression, Journal of Machine Learning
Research, April, 2008, Volume 9, 627--650.
-McCallum1998, A. McCallum and K. Nigam, A comparison of event models for naive bayes text classification, AAAI-98 workshop on learning for text categorization, 1998.
+**[McCallum1998]** A. McCallum and K. Nigam, A comparison of event models for naive bayes text classification, AAAI-98 workshop on learning for text categorization, 1998.
-McCullagh1989, Peter McCullagh and John Ashworth
+**[McCullagh1989]** Peter McCullagh and John Ashworth
Nelder, Generalized Linear Models, Second Edition,
Monographs on Statistics and Applied Probability, Number 37,
1989, Chapman & Hall/CRC, 532.
-Nelder1972, John Ashworth Nelder and Robert
+**[Nelder1972]** John Ashworth Nelder and Robert
William Maclagan Wedderburn, Generalized Linear Models,
Journal of the Royal Statistical Society, Series A
(General), 1972, Volume 135, Number 3,
370--384.
-Nocedal1999, J. Nocedal and S. J. Wright, Numerical Optimization, Springer-Verlag, 1999.
+**[Nocedal1999]** J. Nocedal and S. J. Wright, Numerical Optimization, Springer-Verlag, 1999.
-Nocedal2006, Optimization Numerical Optimization,
+**[Nocedal2006]** Optimization Numerical Optimization,
Jorge Nocedal and Stephen Wright, Springer Series
in Operations Research and Financial Engineering, 664,
Second Edition, Springer, 2006.
-PandaHBB2009, B. Panda, J. Herbach, S. Basu, and R. J. Bayardo. PLANET: massively parallel learning of tree ensembles with mapreduce. PVLDB, 2(2):1426– 1437, 2009.
+**[PandaHBB2009]** B. Panda, J. Herbach, S. Basu, and R. J. Bayardo. PLANET: massively parallel learning of tree ensembles with mapreduce. PVLDB, 2(2):1426– 1437, 2009.
-Russell2009, S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, Prentice Hall, 2009.
+**[Russell2009]** S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, Prentice Hall, 2009.
-Scholkopf1995, B. Scholkopf, C. Burges and V. Vapnik, Extracting Support Data for a Given Task, International Conference on Knowledge Discovery and Data Mining (ICDM), 1995.
+**[Scholkopf1995]** B. Scholkopf, C. Burges and V. Vapnik, Extracting Support Data for a Given Task, International Conference on Knowledge Discovery and Data Mining (ICDM), 1995.
-Stevens1946, Stanley Smith Stevens,
+**[Stevens1946]** Stanley Smith Stevens,
On the Theory of Scales of Measurement, Science
June 7, 1946, Volume 103, Number 2684,
677--680.
-Vetterling1992,
+**[Vetterling1992]**
W. T. Vetterling and B. P. Flannery,
Multidimensions in Numerical Recipes in C - The Art in Scientific Computing, W. H. Press and S. A. Teukolsky (eds.), Cambridge University Press, 1992.
-ZhouWSP08,
+**[ZhouWSP08]**
Y. Zhou, D. M. Wilkinson, R. Schreiber, and R. Pan. Large-scale parallel collaborative filtering for the Netflix prize.
In Algorithmic Aspects in Information and Management, 4th International Conference, AAIM 2008, Shanghai, China, June 23-25, 2008. Proceedings, pages 337–348, 2008.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b8055201/algorithms-classification.md
----------------------------------------------------------------------
diff --git a/algorithms-classification.md b/algorithms-classification.md
index d6215df..b076017 100644
--- a/algorithms-classification.md
+++ b/algorithms-classification.md
@@ -108,6 +108,8 @@ Eqs. (1) and (2).
### Usage
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f MultiLogReg.dml
-nvargs X=<file>
Y=<file>
@@ -119,7 +121,27 @@ Eqs. (1) and (2).
moi=[int]
mii=[int]
fmt=[format]
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f MultiLogReg.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ B=<file>
+ Log=[file]
+ icpt=[int]
+ reg=[double]
+ tol=[double]
+ moi=[int]
+ mii=[int]
+ fmt=[format]
+</div>
+</div>
### Arguments
@@ -173,6 +195,8 @@ SystemML Language Reference for details.
### Examples
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f MultiLogReg.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/Y.mtx
@@ -184,6 +208,27 @@ SystemML Language Reference for details.
moi=100
mii=10
Log=/user/ml/log.csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f MultiLogReg.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ fmt=csv
+ icpt=2
+ reg=1.0
+ tol=0.0001
+ moi=100
+ mii=10
+ Log=/user/ml/log.csv
+</div>
+</div>
* * *
@@ -329,6 +374,8 @@ support vector machine (`y` with domain size `2`).
**Binary-Class Support Vector Machines**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f l2-svm.dml
-nvargs X=<file>
Y=<file>
@@ -339,9 +386,31 @@ support vector machine (`y` with domain size `2`).
model=<file>
Log=<file>
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f l2-svm.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ icpt=[int]
+ tol=[double]
+ reg=[double]
+ maxiter=[int]
+ model=<file>
+ Log=<file>
+ fmt=[format]
+</div>
+</div>
**Binary-Class Support Vector Machines Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f l2-svm-predict.dml
-nvargs X=<file>
Y=[file]
@@ -351,7 +420,25 @@ support vector machine (`y` with domain size `2`).
accuracy=[file]
confusion=[file]
fmt=[format]
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f l2-svm-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=[file]
+ icpt=[int]
+ model=<file>
+ scores=[file]
+ accuracy=[file]
+ confusion=[file]
+ fmt=[format]
+</div>
+</div>
#### Arguments
@@ -403,6 +490,8 @@ using a held-out test set. Note that this is an optional argument.
**Binary-Class Support Vector Machines**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f l2-svm.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/y.mtx
@@ -413,9 +502,31 @@ using a held-out test set. Note that this is an optional argument.
maxiter=100
model=/user/ml/weights.csv
Log=/user/ml/Log.csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f l2-svm.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ icpt=0
+ tol=0.001
+ fmt=csv
+ reg=1.0
+ maxiter=100
+ model=/user/ml/weights.csv
+ Log=/user/ml/Log.csv
+</div>
+</div>
**Binary-Class Support Vector Machines Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f l2-svm-predict.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/y.mtx
@@ -425,6 +536,25 @@ using a held-out test set. Note that this is an optional argument.
scores=/user/ml/scores.csv
accuracy=/user/ml/accuracy.csv
confusion=/user/ml/confusion.csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f l2-svm-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ icpt=0
+ fmt=csv
+ model=/user/ml/weights.csv
+ scores=/user/ml/scores.csv
+ accuracy=/user/ml/accuracy.csv
+ confusion=/user/ml/confusion.csv
+</div>
+</div>
#### Details
@@ -481,6 +611,8 @@ class labels.
**Multi-Class Support Vector Machines**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f m-svm.dml
-nvargs X=<file>
Y=<file>
@@ -491,9 +623,31 @@ class labels.
model=<file>
Log=<file>
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f m-svm.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ icpt=[int]
+ tol=[double]
+ reg=[double]
+ maxiter=[int]
+ model=<file>
+ Log=<file>
+ fmt=[format]
+</div>
+</div>
**Multi-Class Support Vector Machines Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f m-svm-predict.dml
-nvargs X=<file>
Y=[file]
@@ -503,6 +657,25 @@ class labels.
accuracy=[file]
confusion=[file]
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f m-svm-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=[file]
+ icpt=[int]
+ model=<file>
+ scores=[file]
+ accuracy=[file]
+ confusion=[file]
+ fmt=[format]
+</div>
+</div>
#### Arguments
@@ -555,28 +728,71 @@ SystemML Language Reference for details.
**Multi-Class Support Vector Machines**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f m-svm.dml
-nvargs X=/user/ml/X.mtx
- Y=/user/ml/y.mtx
+ Y=/user/ml/y.mtx
icpt=0
tol=0.001
- reg=1.0
- maxiter=100
- fmt=csv
+ reg=1.0
+ maxiter=100
+ fmt=csv
model=/user/ml/weights.csv
Log=/user/ml/Log.csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f m-svm.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ icpt=0
+ tol=0.001
+ reg=1.0
+ maxiter=100
+ fmt=csv
+ model=/user/ml/weights.csv
+ Log=/user/ml/Log.csv
+</div>
+</div>
**Multi-Class Support Vector Machines Prediction**:
- hadoop jar SystemML.jar -f m-svm-predict.dml
- -nvargs X=/user/ml/X.mtx
- Y=/user/ml/y.mtx
- icpt=0
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
+ hadoop jar SystemML.jar -f m-svm-predict.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ icpt=0
fmt=csv
model=/user/ml/weights.csv
scores=/user/ml/scores.csv
accuracy=/user/ml/accuracy.csv
confusion=/user/ml/confusion.csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f m-svm-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ icpt=0
+ fmt=csv
+ model=/user/ml/weights.csv
+ scores=/user/ml/scores.csv
+ accuracy=/user/ml/accuracy.csv
+ confusion=/user/ml/confusion.csv
+</div>
+</div>
#### Details
@@ -636,6 +852,8 @@ applicable when all features are counts of categorical values.
**Naive Bayes**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f naive-bayes.dml
-nvargs X=<file>
Y=<file>
@@ -644,9 +862,29 @@ applicable when all features are counts of categorical values.
conditionals=<file>
accuracy=<file>
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f naive-bayes.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ laplace=[double]
+ prior=<file>
+ conditionals=<file>
+ accuracy=<file>
+ fmt=[format]
+</div>
+</div>
**Naive Bayes Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f naive-bayes-predict.dml
-nvargs X=<file>
Y=[file]
@@ -656,6 +894,25 @@ applicable when all features are counts of categorical values.
accuracy=[file]
confusion=[file]
probabilities=[file]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f naive-bayes-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=[file]
+ prior=<file>
+ conditionals=<file>
+ fmt=[format]
+ accuracy=[file]
+ confusion=[file]
+ probabilities=[file]
+</div>
+</div>
### Arguments
@@ -698,25 +955,67 @@ SystemML Language Reference for details.
**Naive Bayes**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f naive-bayes.dml
- -nvargs X=/user/ml/X.mtx
- Y=/user/ml/y.mtx
- laplace=1 fmt=csv
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ laplace=1
+ fmt=csv
prior=/user/ml/prior.csv
conditionals=/user/ml/conditionals.csv
accuracy=/user/ml/accuracy.csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f naive-bayes.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ laplace=1
+ fmt=csv
+ prior=/user/ml/prior.csv
+ conditionals=/user/ml/conditionals.csv
+ accuracy=/user/ml/accuracy.csv
+</div>
+</div>
**Naive Bayes Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f naive-bayes-predict.dml
- -nvargs X=/user/ml/X.mtx
- Y=/user/ml/y.mtx
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
prior=/user/ml/prior.csv
conditionals=/user/ml/conditionals.csv
fmt=csv
accuracy=/user/ml/accuracy.csv
probabilities=/user/ml/probabilities.csv
confusion=/user/ml/confusion.csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f naive-bayes-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ prior=/user/ml/prior.csv
+ conditionals=/user/ml/conditionals.csv
+ fmt=csv
+ accuracy=/user/ml/accuracy.csv
+ probabilities=/user/ml/probabilities.csv
+ confusion=/user/ml/confusion.csv
+</div>
+</div>
### Details
@@ -781,6 +1080,8 @@ implementation is well-suited to handle large-scale data and builds a
**Decision Tree**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f decision-tree.dml
-nvargs X=<file>
Y=<file>
@@ -795,9 +1096,35 @@ implementation is well-suited to handle large-scale data and builds a
S_map=[file]
C_map=[file]
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f decision-tree.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ R=[file]
+ M=<file>
+ bins=[int]
+ depth=[int]
+ num_leaf=[int]
+ num_samples=[int]
+ impurity=[Gini|entropy]
+ O=[file]
+ S_map=[file]
+ C_map=[file]
+ fmt=[format]
+</div>
+</div>
**Decision Tree Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f decision-tree-predict.dml
-nvargs X=<file>
Y=[file]
@@ -807,6 +1134,25 @@ implementation is well-suited to handle large-scale data and builds a
A=[file]
CM=[file]
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f decision-tree-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=[file]
+ R=[file]
+ M=<file>
+ P=<file>
+ A=[file]
+ CM=[file]
+ fmt=[format]
+</div>
+</div>
### Arguments
@@ -875,6 +1221,8 @@ SystemML Language Reference for details.
**Decision Tree**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f decision-tree.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/Y.mtx
@@ -886,9 +1234,32 @@ SystemML Language Reference for details.
num_samples=3000
impurity=Gini
fmt=csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f decision-tree.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ R=/user/ml/R.csv
+ M=/user/ml/model.csv
+ bins=20
+ depth=25
+ num_leaf=10
+ num_samples=3000
+ impurity=Gini
+ fmt=csv
+</div>
+</div>
**Decision Tree Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f decision-tree-predict.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/Y.mtx
@@ -898,7 +1269,25 @@ SystemML Language Reference for details.
A=/user/ml/accuracy.csv
CM=/user/ml/confusion.csv
fmt=csv
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f decision-tree-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ R=/user/ml/R.csv
+ M=/user/ml/model.csv
+ P=/user/ml/predictions.csv
+ A=/user/ml/accuracy.csv
+ CM=/user/ml/confusion.csv
+ fmt=csv
+</div>
+</div>
### Details
@@ -1096,6 +1485,8 @@ for classification in parallel.
**Random Forest**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f random-forest.dml
-nvargs X=<file>
Y=<file>
@@ -1113,9 +1504,38 @@ for classification in parallel.
S_map=[file]
C_map=[file]
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f random-forest.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ R=[file]
+ M=<file>
+ bins=[int]
+ depth=[int]
+ num_leaf=[int]
+ num_samples=[int]
+ num_trees=[int]
+ subsamp_rate=[double]
+ feature_subset=[double]
+ impurity=[Gini|entropy]
+ C=[file]
+ S_map=[file]
+ C_map=[file]
+ fmt=[format]
+</div>
+</div>
**Random Forest Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f random-forest-predict.dml
-nvargs X=<file>
Y=[file]
@@ -1127,6 +1547,27 @@ for classification in parallel.
OOB=[file]
CM=[file]
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f random-forest-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=[file]
+ R=[file]
+ M=<file>
+ C=[file]
+ P=<file>
+ A=[file]
+ OOB=[file]
+ CM=[file]
+ fmt=[format]
+</div>
+</div>
### Arguments
@@ -1215,6 +1656,8 @@ SystemML Language Reference for details.
**Random Forest**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f random-forest.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/Y.mtx
@@ -1227,11 +1670,35 @@ SystemML Language Reference for details.
num_trees=10
impurity=Gini
fmt=csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f random-forest.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ R=/user/ml/R.csv
+ M=/user/ml/model.csv
+ bins=20
+ depth=25
+ num_leaf=10
+ num_samples=3000
+ num_trees=10
+ impurity=Gini
+ fmt=csv
+</div>
+</div>
**Random Forest Prediction**:
To compute predictions:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f random-forest-predict.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/Y.mtx
@@ -1241,6 +1708,25 @@ To compute predictions:
A=/user/ml/accuracy.csv
CM=/user/ml/confusion.csv
fmt=csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f random-forest-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ R=/user/ml/R.csv
+ M=/user/ml/model.csv
+ P=/user/ml/predictions.csv
+ A=/user/ml/accuracy.csv
+ CM=/user/ml/confusion.csv
+ fmt=csv
+</div>
+</div>
### Details
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b8055201/algorithms-clustering.md
----------------------------------------------------------------------
diff --git a/algorithms-clustering.md b/algorithms-clustering.md
index a443083..31a31e2 100644
--- a/algorithms-clustering.md
+++ b/algorithms-clustering.md
@@ -95,6 +95,8 @@ apart is a “false negative” etc.
**K-Means**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Kmeans.dml
-nvargs X=<file>
C=[file]
@@ -107,9 +109,33 @@ apart is a “false negative” etc.
Y=[file]
fmt=[format]
verb=[int]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Kmeans.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ C=[file]
+ k=<int>
+ runs=[int]
+ maxi=[int]
+ tol=[double]
+ samp=[int]
+ isY=[int]
+ Y=[file]
+ fmt=[format]
+ verb=[int]
+</div>
+</div>
**K-Means Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Kmeans-predict.dml
-nvargs X=[file]
C=[file]
@@ -117,7 +143,23 @@ apart is a “false negative” etc.
prY=[file]
fmt=[format]
O=[file]
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Kmeans-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=[file]
+ C=[file]
+ spY=[file]
+ prY=[file]
+ fmt=[format]
+ O=[file]
+</div>
+</div>
### Arguments - K-Means
@@ -186,12 +228,31 @@ standard output
**K-Means**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Kmeans.dml
-nvargs X=/user/ml/X.mtx
k=5
C=/user/ml/centroids.mtx
fmt=csv
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Kmeans.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ k=5
+ C=/user/ml/centroids.mtx
+ fmt=csv
+</div>
+</div>
+
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Kmeans.dml
-nvargs X=/user/ml/X.mtx
k=5
@@ -203,33 +264,104 @@ standard output
isY=1
Y=/user/ml/Yout.mtx
verb=1
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Kmeans.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ k=5
+ runs=100
+ maxi=5000
+ tol=0.00000001
+ samp=20
+ C=/user/ml/centroids.mtx
+ isY=1
+ Y=/user/ml/Yout.mtx
+ verb=1
+</div>
+</div>
**K-Means Prediction**:
To predict Y given X and C:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Kmeans-predict.dml
-nvargs X=/user/ml/X.mtx
C=/user/ml/C.mtx
prY=/user/ml/PredY.mtx
O=/user/ml/stats.csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Kmeans-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ C=/user/ml/C.mtx
+ prY=/user/ml/PredY.mtx
+ O=/user/ml/stats.csv
+</div>
+</div>
To compare “actual” labels `spY` with “predicted” labels
given X and C:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Kmeans-predict.dml
-nvargs X=/user/ml/X.mtx
C=/user/ml/C.mtx
spY=/user/ml/Y.mtx
O=/user/ml/stats.csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Kmeans-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ C=/user/ml/C.mtx
+ spY=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+</div>
+</div>
To compare “actual” labels `spY` with given “predicted”
labels prY:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Kmeans-predict.dml
-nvargs spY=/user/ml/Y.mtx
prY=/user/ml/PredY.mtx
O=/user/ml/stats.csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Kmeans-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs spY=/user/ml/Y.mtx
+ prY=/user/ml/PredY.mtx
+ O=/user/ml/stats.csv
+</div>
+</div>
* * *
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b8055201/algorithms-descriptive-statistics.md
----------------------------------------------------------------------
diff --git a/algorithms-descriptive-statistics.md b/algorithms-descriptive-statistics.md
index eafb88e..a0dd133 100644
--- a/algorithms-descriptive-statistics.md
+++ b/algorithms-descriptive-statistics.md
@@ -99,10 +99,26 @@ to compute the mean of a categorical attribute like ‘Hair Color’.
### Usage
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Univar-Stats.dml
- -nvargs X=<file>
+ -nvargs X=<file>
TYPES=<file>
STATS=<file>
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Univar-Stats.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ TYPES=<file>
+ STATS=<file>
+</div>
+</div>
### Arguments
@@ -122,10 +138,26 @@ be stored. The format of the output matrix is defined by
### Examples
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Univar-Stats.dml
-nvargs X=/user/ml/X.mtx
TYPES=/user/ml/types.mtx
STATS=/user/ml/stats.mtx
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Univar-Stats.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ TYPES=/user/ml/types.mtx
+ STATS=/user/ml/stats.mtx
+</div>
+</div>
* * *
@@ -524,6 +556,8 @@ attributes like ‘Hair Color’.
### Usage
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f bivar-stats.dml
-nvargs X=<file>
index1=<file>
@@ -531,6 +565,23 @@ attributes like ‘Hair Color’.
types1=<file>
types2=<file>
OUTDIR=<directory>
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f bivar-stats.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ index1=<file>
+ index2=<file>
+ types1=<file>
+ types2=<file>
+ OUTDIR=<directory>
+</div>
+</div>
### Arguments
@@ -574,6 +625,8 @@ are defined in [**Table 2**](algorithms-descriptive-statistics.html#table2).
### Examples
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f bivar-stats.dml
-nvargs X=/user/ml/X.mtx
index1=/user/ml/S1.mtx
@@ -581,7 +634,24 @@ are defined in [**Table 2**](algorithms-descriptive-statistics.html#table2).
types1=/user/ml/K1.mtx
types2=/user/ml/K2.mtx
OUTDIR=/user/ml/stats.mtx
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f bivar-stats.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ index1=/user/ml/S1.mtx
+ index2=/user/ml/S2.mtx
+ types1=/user/ml/K1.mtx
+ types2=/user/ml/K2.mtx
+ OUTDIR=/user/ml/stats.mtx
+</div>
+</div>
+
* * *
@@ -1046,6 +1116,8 @@ becomes reversed and amplified (from $+0.1$ to $-0.5$) if we ignore the months.
### Usage
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f stratstats.dml
-nvargs X=<file>
Xcid=[file]
@@ -1055,6 +1127,25 @@ becomes reversed and amplified (from $+0.1$ to $-0.5$) if we ignore the months.
Scid=[int]
O=<file>
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f stratstats.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Xcid=[file]
+ Y=[file]
+ Ycid=[file]
+ S=[file]
+ Scid=[int]
+ O=<file>
+ fmt=[format]
+</div>
+</div>
### Arguments
@@ -1233,21 +1324,61 @@ SystemML Language Reference for details.
### Examples
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f stratstats.dml
-nvargs X=/user/ml/X.mtx
Xcid=/user/ml/Xcid.mtx
Y=/user/ml/Y.mtx
Ycid=/user/ml/Ycid.mtx
- S=/user/ml/S.mtx Scid=2
+ S=/user/ml/S.mtx
+ Scid=2
O=/user/ml/Out.mtx
fmt=csv
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f stratstats.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Xcid=/user/ml/Xcid.mtx
+ Y=/user/ml/Y.mtx
+ Ycid=/user/ml/Ycid.mtx
+ S=/user/ml/S.mtx
+ Scid=2
+ O=/user/ml/Out.mtx
+ fmt=csv
+</div>
+</div>
+
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f stratstats.dml
-nvargs X=/user/ml/Data.mtx
Xcid=/user/ml/Xcid.mtx
Ycid=/user/ml/Ycid.mtx
Scid=7
O=/user/ml/Out.mtx
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f stratstats.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/Data.mtx
+ Xcid=/user/ml/Xcid.mtx
+ Ycid=/user/ml/Ycid.mtx
+ Scid=7
+ O=/user/ml/Out.mtx
+</div>
+</div>
### Details
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b8055201/algorithms-matrix-factorization.md
----------------------------------------------------------------------
diff --git a/algorithms-matrix-factorization.md b/algorithms-matrix-factorization.md
index a46a2cd..94627e3 100644
--- a/algorithms-matrix-factorization.md
+++ b/algorithms-matrix-factorization.md
@@ -25,6 +25,8 @@ top-$K$ (for a given value of $K$) principle components.
### Usage
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f PCA.dml
-nvargs INPUT=<file>
K=<int>
@@ -34,6 +36,25 @@ top-$K$ (for a given value of $K$) principle components.
OFMT=[format]
MODEL=<file>
OUTPUT=<file>
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f PCA.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs INPUT=<file>
+ K=<int>
+ CENTER=[int]
+ SCALE=[int]
+ PROJDATA=<int>
+ OFMT=[format]
+ MODEL=<file>
+ OUTPUT=<file>
+</div>
+</div>
#### Arguments
@@ -66,9 +87,10 @@ SystemML Language Reference for details.
vector space.
-
#### Examples
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f PCA.dml
-nvargs INPUT=/user/ml/input.mtx
K=10
@@ -77,7 +99,27 @@ SystemML Language Reference for details.
FMT=csv
PROJDATA=1
OUTPUT=/user/ml/pca_output/
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f PCA.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs INPUT=/user/ml/input.mtx
+ K=10
+ CENTER=1
+ SCALE=1O
+ FMT=csv
+ PROJDATA=1
+ OUTPUT=/user/ml/pca_output/
+</div>
+</div>
+
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f PCA.dml
-nvargs INPUT=/user/ml/test_input.mtx
K=10
@@ -87,7 +129,25 @@ SystemML Language Reference for details.
PROJDATA=1
MODEL=/user/ml/pca_output/
OUTPUT=/user/ml/test_output.mtx
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f PCA.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs INPUT=/user/ml/test_input.mtx
+ K=10
+ CENTER=1
+ SCALE=1O
+ FMT=csv
+ PROJDATA=1
+ MODEL=/user/ml/pca_output/
+ OUTPUT=/user/ml/test_output.mtx
+</div>
+</div>
#### Details
@@ -164,6 +224,8 @@ problems.
**ALS**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f ALS.dml
-nvargs V=<file>
L=<file>
@@ -175,9 +237,32 @@ problems.
check=[boolean]
thr=[double]
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f ALS.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs V=<file>
+ L=<file>
+ R=<file>
+ rank=[int]
+ reg=[L2|wL2]
+ lambda=[double]
+ maxi=[int]
+ check=[boolean]
+ thr=[double]
+ fmt=[format]
+</div>
+</div>
**ALS Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f ALS_predict.dml
-nvargs X=<file>
Y=<file>
@@ -186,9 +271,29 @@ problems.
Vrows=<int>
Vcols=<int>
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f ALS_predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ L=<file>
+ R=<file>
+ Vrows=<int>
+ Vcols=<int>
+ fmt=[format]
+</div>
+</div>
**ALS Top-K Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f ALS_topk_predict.dml
-nvargs X=<file>
Y=<file>
@@ -197,6 +302,24 @@ problems.
V=<file>
K=[int]
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f ALS_topk_predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ L=<file>
+ R=<file>
+ V=<file>
+ K=[int]
+ fmt=[format]
+</div>
+</div>
### Arguments - ALS
@@ -275,6 +398,8 @@ SystemML Language Reference for details.
**ALS**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f ALS.dml
-nvargs V=/user/ml/V
L=/user/ml/L
@@ -286,12 +411,34 @@ SystemML Language Reference for details.
check=TRUE
thr=0.001
fmt=csv
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f ALS.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs V=/user/ml/V
+ L=/user/ml/L
+ R=/user/ml/R
+ rank=10
+ reg="wL"
+ lambda=0.0001
+ maxi=50
+ check=TRUE
+ thr=0.001
+ fmt=csv
+</div>
+</div>
**ALS Prediction**:
To compute predicted ratings for a given list of users and items:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f ALS_predict.dml
-nvargs X=/user/ml/X
Y=/user/ml/Y
@@ -300,13 +447,32 @@ To compute predicted ratings for a given list of users and items:
Vrows=100000
Vcols=10000
fmt=csv
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f ALS_predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X
+ Y=/user/ml/Y
+ L=/user/ml/L
+ R=/user/ml/R
+ Vrows=100000
+ Vcols=10000
+ fmt=csv
+</div>
+</div>
**ALS Top-K Prediction**:
To compute top-K items with highest predicted ratings together with the
predicted ratings for a given list of users:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f ALS_topk_predict.dml
-nvargs X=/user/ml/X
Y=/user/ml/Y
@@ -315,6 +481,24 @@ predicted ratings for a given list of users:
V=/user/ml/V
K=10
fmt=csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f ALS_topk_predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X
+ Y=/user/ml/Y
+ L=/user/ml/L
+ R=/user/ml/R
+ V=/user/ml/V
+ K=10
+ fmt=csv
+</div>
+</div>
### Details
[05/47] incubator-systemml git commit: Cleanup matrix block runtime
0-based indexes (slice, left index), incl updated docs
Posted by du...@apache.org.
Cleanup matrix block runtime 0-based indexes (slice, left index), incl
updated docs
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/e378de08
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/e378de08
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/e378de08
Branch: refs/heads/gh-pages
Commit: e378de0898f2be52985c65431c9e5ebb1f8af587
Parents: a891c0a
Author: mboehm7 <mb...@us.ibm.com>
Authored: Mon Aug 17 18:13:30 2015 -0700
Committer: mboehm7 <mb...@us.ibm.com>
Committed: Mon Aug 17 18:13:30 2015 -0700
----------------------------------------------------------------------
october_todo_list.txt | 62 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 62 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/e378de08/october_todo_list.txt
----------------------------------------------------------------------
diff --git a/october_todo_list.txt b/october_todo_list.txt
new file mode 100644
index 0000000..101663b
--- /dev/null
+++ b/october_todo_list.txt
@@ -0,0 +1,62 @@
+# TARGET DATE: October 1, 2015
+
+1) Major new features
+* Support for boolean matrices (data, io, operations, optimizer) 6pm
+* New input data formats (scientific data, e.g., HDF5, NetCDF) 1pm
+* New script-level algorithms (to be defined)
+* ML pipeline support (transformers and estimators) 1.5pm
+* Algorithm wrappers (ml pipelines, ml context) .5pm
+* SparkR integration (data conversion, wrappers) 1pm
+
+2) New operators / language features
+* Generalization permutation matrix mult 1pw (e.g., stratstats, cox)
+* Generalization mmchain for ncol 1..bclen 1pw (e.g., ppca)
+* Unary aggregate stddev operations .5pw
+* Remove empty w/ offset broadcasting .5pw
+* Sort w/ index vector broadcasting .5pw
+* Generalization of moments to matrices .5pw
+* Generalization of groupedagg to matrices 1pw (e.g., naive bayes)
+* Extended cumagg operator (sparse, rev) 1pw
+* Apply builtin functions 1pm
+* Cbind/rbind append generalization 2pw
+* Extended rand distribution functions 1pm
+* rev() builtin function (reorg op) 1pw
+* rep() builtin function, incl rewrites 2pw
+* Formatted print() builtin function .5pw
+
+3) Compiler features
+* Code motion (cse across loops) 2pw
+* Extended inter-procedural analysis 2pw (e.g., step_glm)
+* Sparsity-aware broadcast op selection 1pw
+* Function calls in expressions 2pw
+
+3) Parfor
+* Derived horizontal partitioning 2pw (e.g., decision tree)
+* New result agg funs (min, max, sum) 2pw
+* New result merge function 'append' 2pw (e.g., decision tree)
+* Blockwise data partitioning 2pw
+* Fused remote dpexec w/ result merge 1pw
+* Optimizer support for shared reads 1pw
+
+4) Misc features
+* Rework UDF framework (pass-through rdds) 2pw
+* Scala ML context hardening 1pw
+* Python ML context hardening 2pw
+* PyDML grammar hardening/consistency 1pw
+* Cleanup block runtime 0-based indexes 1pw .. done
+* Notebook (dml execute, stats/explain) 2pw
+* PMML support (extended pmml bigr work) 1pm [postponed]
+* Async or on-demand spark context creation 1pw .. done
+
+5) Tools
+* Stats util extension for lazy evaluation 2pw
+
+6) Nice-to-have
+* Special NaN handling (data/operations) 1-6pm
+* Tooling (eclipse tooling, etc) 1-6pm
+* Rework debugger w/ dyn recompile 2pw
+
+7) Continuous tasks
+* Documentation (user, developer)
+* Tests (unit tests, performance tests)
+
[25/47] incubator-systemml git commit: New wcemm quaternary operator
(compiler/runtime cp/mr/sp)
Posted by du...@apache.org.
New wcemm quaternary operator (compiler/runtime cp/mr/sp)
Fused operator for sum(X*log(U%*%t(V))) (introduced for kl_mf) in order
to prevent the dense intermediate (U%*%t(V)). This includes the
rewrites, 1 physical operator in cp, 2 physical operators in mr, 2
physical operators in sp, the new sequential and multi-threaded mm block
operations, tests as well as updated developer documentation.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/8fd0f746
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/8fd0f746
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/8fd0f746
Branch: refs/heads/gh-pages
Commit: 8fd0f7464c67a14e825b16730455029ff6e38108
Parents: f1eb428
Author: Matthias Boehm <mb...@us.ibm.com>
Authored: Thu Sep 10 17:36:27 2015 -0700
Committer: Matthias Boehm <mb...@us.ibm.com>
Committed: Fri Sep 11 09:48:46 2015 -0700
----------------------------------------------------------------------
devdocs/MatrixMultiplicationOperators.txt | 16 +++++++++++++---
1 file changed, 13 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/8fd0f746/devdocs/MatrixMultiplicationOperators.txt
----------------------------------------------------------------------
diff --git a/devdocs/MatrixMultiplicationOperators.txt b/devdocs/MatrixMultiplicationOperators.txt
index ffc38f2..e8a1419 100644
--- a/devdocs/MatrixMultiplicationOperators.txt
+++ b/devdocs/MatrixMultiplicationOperators.txt
@@ -1,6 +1,6 @@
#####################################################################
# TITLE: An Overview of Matrix Multiplication Operators in SystemML #
-# DATE MODIFIED: 09/07/2015 #
+# DATE MODIFIED: 09/10/2015 #
#####################################################################
In the following, we give an overview of backend-specific physical matrix multiplication operators in SystemML as well as their internally used matrix multiplication block operations.
@@ -47,6 +47,7 @@ A QuaternaryOp hop can be compiled into the following physical operators. Note t
- WSLoss (weighted squared loss) --> wsloss
- WSigmoid (weighted sigmoid) --> wsigmoid
- WDivMM (weighted divide matrix multiplication) --> wdivmm
+ - WCeMM (weighted cross entropy matrix multiplication) --> wcemm
* 2) Physical Operator in MR (distributed, mapreduce)
- MapWSLoss (map-side weighted squared loss) --> wsloss
@@ -55,6 +56,8 @@ A QuaternaryOp hop can be compiled into the following physical operators. Note t
- RedWSigmoid (reduce-side weighted sigmoid) --> wsigmoid
- MapWDivMM (map-side weighted divide matrix mult) --> wdivmm
- RedWDivMM (reduce-side weighted divide matrix mult) --> wdivmm
+ - MapWCeMM (map-side weighted cross entr. matrix mult) --> wcemm
+ - RedWCeMM (reduce-side w. cross entr. matrix mult) --> wcemm
* 3) Physical Operators in SPARK (distributed, spark)
- MapWSLoss (see MR, mappartitions + reduce) --> wsloss
@@ -66,8 +69,11 @@ A QuaternaryOp hop can be compiled into the following physical operators. Note t
- MapWDivMM (see MR, mappartitions + reducebykey ) --> wdivmm
- RedWDivMM (see MR, 1/2x flatmaptopair + 1/2x join + --> wdivmm
maptopair + reducebykey)
-
-
+ - MapWCeMM (see MR, mappartitions + reduce) --> wcemm
+ - RedWDivMM (see MR, 1/2x flatmaptopair + 1/2x join + --> wcemm
+ maptopair + reduce)
+
+
C) CORE MATRIX MULT PRIMITIVES LibMatrixMult (incl related script patterns)
-------------------------------
* 1) mm (general A %*% B)
@@ -101,5 +107,9 @@ C) CORE MATRIX MULT PRIMITIVES LibMatrixMult (incl related script patterns)
- sequential / multi-threaded (same block ops, par over rows in X)
- all dense, sparse-dense factors, sparse/dense-* x 2 patterns
+* 8) wcemm (sum(X*log(U%*%t(V))))
+ - sequential / multi-threaded (same block ops, par over rows in X)
+ - all dense, sparse-dense factors, sparse/dense-*, 1 pattern
+
\ No newline at end of file
[29/47] incubator-systemml git commit: Generalization wsloss
simplification rewrite (-X patterns), for als_cg
Posted by du...@apache.org.
Generalization wsloss simplification rewrite (-X patterns), for als_cg
Incl (1) new test cases, and (2) update documentation.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/ebb052d6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/ebb052d6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/ebb052d6
Branch: refs/heads/gh-pages
Commit: ebb052d69ba42075e227152329bbd2d68f8ab571
Parents: 0ad76f6
Author: Matthias Boehm <mb...@us.ibm.com>
Authored: Thu Sep 24 03:41:18 2015 -0700
Committer: Matthias Boehm <mb...@us.ibm.com>
Committed: Thu Sep 24 03:41:18 2015 -0700
----------------------------------------------------------------------
devdocs/MatrixMultiplicationOperators.txt | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/ebb052d6/devdocs/MatrixMultiplicationOperators.txt
----------------------------------------------------------------------
diff --git a/devdocs/MatrixMultiplicationOperators.txt b/devdocs/MatrixMultiplicationOperators.txt
index e8a1419..839a168 100644
--- a/devdocs/MatrixMultiplicationOperators.txt
+++ b/devdocs/MatrixMultiplicationOperators.txt
@@ -1,6 +1,6 @@
#####################################################################
# TITLE: An Overview of Matrix Multiplication Operators in SystemML #
-# DATE MODIFIED: 09/10/2015 #
+# DATE MODIFIED: 09/24/2015 #
#####################################################################
In the following, we give an overview of backend-specific physical matrix multiplication operators in SystemML as well as their internally used matrix multiplication block operations.
@@ -94,7 +94,8 @@ C) CORE MATRIX MULT PRIMITIVES LibMatrixMult (incl related script patterns)
- sparse-sparse, dense-dense, sparse-dense
* 5) wsloss ((a) sum(W*(X-U%*%t(V))^2), (b) sum((X-W*(U%*%t(V)))^2),
- (c) sum((X -(U%*%t(V)))^2))
+ (c) sum((X-(U%*%t(V)))^2)), (d) sum(W*(U%*%t(V)-X)^2),
+ (e) sum((W*(U%*%t(V))-X)^2), (f) sum(((U%*%t(V))-X)^2))
- sequential / multi-threaded (same block ops, par over rows in W/X)
- all dense, sparse-dense factors, sparse/dense-* x 3 patterns
@@ -103,7 +104,7 @@ C) CORE MATRIX MULT PRIMITIVES LibMatrixMult (incl related script patterns)
- sequential / multi-threaded (same block ops, par over rows in W)
- all dense, sparse-dense factors, sparse/dense-* x 4 patterns
-* 7) wdivmm ((a) t(t(U)%*%(X/(U%*%t(V)))), (b) (X/(U%*%t(V)))%*%V
+* 7) wdivmm ((a) t(t(U)%*%(X/(U%*%t(V)))), (b) (X/(U%*%t(V)))%*%V)
- sequential / multi-threaded (same block ops, par over rows in X)
- all dense, sparse-dense factors, sparse/dense-* x 2 patterns
[13/47] incubator-systemml git commit: initial port of documentation
to md, algorithms reference converted to md, template based on spark docs
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/css/bootstrap.css
----------------------------------------------------------------------
diff --git a/css/bootstrap.css b/css/bootstrap.css
new file mode 100644
index 0000000..b51ef7b
--- /dev/null
+++ b/css/bootstrap.css
@@ -0,0 +1,5624 @@
+/*!
+ * Bootstrap v2.1.0
+ *
+ * Copyright 2012 Twitter, Inc
+ * Licensed under the Apache License v2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Designed and built with all the love in the world @twitter by @mdo and @fat.
+ */
+
+article,
+aside,
+details,
+figcaption,
+figure,
+footer,
+header,
+hgroup,
+nav,
+section {
+ display: block;
+}
+
+audio,
+canvas,
+video {
+ display: inline-block;
+ *display: inline;
+ *zoom: 1;
+}
+
+audio:not([controls]) {
+ display: none;
+}
+
+html {
+ font-size: 100%;
+ -webkit-text-size-adjust: 100%;
+ -ms-text-size-adjust: 100%;
+}
+
+a:focus {
+ outline: thin dotted #333;
+ outline: 5px auto -webkit-focus-ring-color;
+ outline-offset: -2px;
+}
+
+a:hover,
+a:active {
+ outline: 0;
+}
+
+sub,
+sup {
+ position: relative;
+ font-size: 75%;
+ line-height: 0;
+ vertical-align: baseline;
+}
+
+sup {
+ top: -0.5em;
+}
+
+sub {
+ bottom: -0.25em;
+}
+
+img {
+ height: auto;
+ max-width: 100%;
+ vertical-align: middle;
+ border: 0;
+ -ms-interpolation-mode: bicubic;
+}
+
+#map_canvas img {
+ max-width: none;
+}
+
+button,
+input,
+select,
+textarea {
+ margin: 0;
+ font-size: 100%;
+ vertical-align: middle;
+}
+
+button,
+input {
+ *overflow: visible;
+ line-height: normal;
+}
+
+button::-moz-focus-inner,
+input::-moz-focus-inner {
+ padding: 0;
+ border: 0;
+}
+
+button,
+input[type="button"],
+input[type="reset"],
+input[type="submit"] {
+ cursor: pointer;
+ -webkit-appearance: button;
+}
+
+input[type="search"] {
+ -webkit-box-sizing: content-box;
+ -moz-box-sizing: content-box;
+ box-sizing: content-box;
+ -webkit-appearance: textfield;
+}
+
+input[type="search"]::-webkit-search-decoration,
+input[type="search"]::-webkit-search-cancel-button {
+ -webkit-appearance: none;
+}
+
+textarea {
+ overflow: auto;
+ vertical-align: top;
+}
+
+.clearfix {
+ *zoom: 1;
+}
+
+.clearfix:before,
+.clearfix:after {
+ display: table;
+ line-height: 0;
+ content: "";
+}
+
+.clearfix:after {
+ clear: both;
+}
+
+.hide-text {
+ font: 0/0 a;
+ color: transparent;
+ text-shadow: none;
+ background-color: transparent;
+ border: 0;
+}
+
+.input-block-level {
+ display: block;
+ width: 100%;
+ min-height: 30px;
+ -webkit-box-sizing: border-box;
+ -moz-box-sizing: border-box;
+ box-sizing: border-box;
+}
+
+body {
+ margin: 0;
+ font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
+ font-size: 14px;
+ line-height: 20px;
+ color: #333333;
+ background-color: #ffffff;
+}
+
+a {
+ color: #0088cc;
+ text-decoration: none;
+}
+
+a:hover {
+ color: #005580;
+ text-decoration: underline;
+}
+
+.img-rounded {
+ -webkit-border-radius: 6px;
+ -moz-border-radius: 6px;
+ border-radius: 6px;
+}
+
+.img-polaroid {
+ padding: 4px;
+ background-color: #fff;
+ border: 1px solid #ccc;
+ border: 1px solid rgba(0, 0, 0, 0.2);
+ -webkit-box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
+ -moz-box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
+ box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
+}
+
+.img-circle {
+ -webkit-border-radius: 500px;
+ -moz-border-radius: 500px;
+ border-radius: 500px;
+}
+
+.row {
+ margin-left: -20px;
+ *zoom: 1;
+}
+
+.row:before,
+.row:after {
+ display: table;
+ line-height: 0;
+ content: "";
+}
+
+.row:after {
+ clear: both;
+}
+
+[class*="span"] {
+ float: left;
+ margin-left: 20px;
+}
+
+.container,
+.navbar-static-top .container,
+.navbar-fixed-top .container,
+.navbar-fixed-bottom .container {
+ width: 940px;
+}
+
+.span12 {
+ width: 940px;
+}
+
+.span11 {
+ width: 860px;
+}
+
+.span10 {
+ width: 780px;
+}
+
+.span9 {
+ width: 700px;
+}
+
+.span8 {
+ width: 620px;
+}
+
+.span7 {
+ width: 540px;
+}
+
+.span6 {
+ width: 460px;
+}
+
+.span5 {
+ width: 380px;
+}
+
+.span4 {
+ width: 300px;
+}
+
+.span3 {
+ width: 220px;
+}
+
+.span2 {
+ width: 140px;
+}
+
+.span1 {
+ width: 60px;
+}
+
+.offset12 {
+ margin-left: 980px;
+}
+
+.offset11 {
+ margin-left: 900px;
+}
+
+.offset10 {
+ margin-left: 820px;
+}
+
+.offset9 {
+ margin-left: 740px;
+}
+
+.offset8 {
+ margin-left: 660px;
+}
+
+.offset7 {
+ margin-left: 580px;
+}
+
+.offset6 {
+ margin-left: 500px;
+}
+
+.offset5 {
+ margin-left: 420px;
+}
+
+.offset4 {
+ margin-left: 340px;
+}
+
+.offset3 {
+ margin-left: 260px;
+}
+
+.offset2 {
+ margin-left: 180px;
+}
+
+.offset1 {
+ margin-left: 100px;
+}
+
+.row-fluid {
+ width: 100%;
+ *zoom: 1;
+}
+
+.row-fluid:before,
+.row-fluid:after {
+ display: table;
+ line-height: 0;
+ content: "";
+}
+
+.row-fluid:after {
+ clear: both;
+}
+
+.row-fluid [class*="span"] {
+ display: block;
+ float: left;
+ width: 100%;
+ min-height: 30px;
+ margin-left: 2.127659574468085%;
+ *margin-left: 2.074468085106383%;
+ -webkit-box-sizing: border-box;
+ -moz-box-sizing: border-box;
+ box-sizing: border-box;
+}
+
+.row-fluid [class*="span"]:first-child {
+ margin-left: 0;
+}
+
+.row-fluid .span12 {
+ width: 100%;
+ *width: 99.94680851063829%;
+}
+
+.row-fluid .span11 {
+ width: 91.48936170212765%;
+ *width: 91.43617021276594%;
+}
+
+.row-fluid .span10 {
+ width: 82.97872340425532%;
+ *width: 82.92553191489361%;
+}
+
+.row-fluid .span9 {
+ width: 74.46808510638297%;
+ *width: 74.41489361702126%;
+}
+
+.row-fluid .span8 {
+ width: 65.95744680851064%;
+ *width: 65.90425531914893%;
+}
+
+.row-fluid .span7 {
+ width: 57.44680851063829%;
+ *width: 57.39361702127659%;
+}
+
+.row-fluid .span6 {
+ width: 48.93617021276595%;
+ *width: 48.88297872340425%;
+}
+
+.row-fluid .span5 {
+ width: 40.42553191489362%;
+ *width: 40.37234042553192%;
+}
+
+.row-fluid .span4 {
+ width: 31.914893617021278%;
+ *width: 31.861702127659576%;
+}
+
+.row-fluid .span3 {
+ width: 23.404255319148934%;
+ *width: 23.351063829787233%;
+}
+
+.row-fluid .span2 {
+ width: 14.893617021276595%;
+ *width: 14.840425531914894%;
+}
+
+.row-fluid .span1 {
+ width: 6.382978723404255%;
+ *width: 6.329787234042553%;
+}
+
+.row-fluid .offset12 {
+ margin-left: 104.25531914893617%;
+ *margin-left: 104.14893617021275%;
+}
+
+.row-fluid .offset12:first-child {
+ margin-left: 102.12765957446808%;
+ *margin-left: 102.02127659574467%;
+}
+
+.row-fluid .offset11 {
+ margin-left: 95.74468085106382%;
+ *margin-left: 95.6382978723404%;
+}
+
+.row-fluid .offset11:first-child {
+ margin-left: 93.61702127659574%;
+ *margin-left: 93.51063829787232%;
+}
+
+.row-fluid .offset10 {
+ margin-left: 87.23404255319149%;
+ *margin-left: 87.12765957446807%;
+}
+
+.row-fluid .offset10:first-child {
+ margin-left: 85.1063829787234%;
+ *margin-left: 84.99999999999999%;
+}
+
+.row-fluid .offset9 {
+ margin-left: 78.72340425531914%;
+ *margin-left: 78.61702127659572%;
+}
+
+.row-fluid .offset9:first-child {
+ margin-left: 76.59574468085106%;
+ *margin-left: 76.48936170212764%;
+}
+
+.row-fluid .offset8 {
+ margin-left: 70.2127659574468%;
+ *margin-left: 70.10638297872339%;
+}
+
+.row-fluid .offset8:first-child {
+ margin-left: 68.08510638297872%;
+ *margin-left: 67.9787234042553%;
+}
+
+.row-fluid .offset7 {
+ margin-left: 61.70212765957446%;
+ *margin-left: 61.59574468085106%;
+}
+
+.row-fluid .offset7:first-child {
+ margin-left: 59.574468085106375%;
+ *margin-left: 59.46808510638297%;
+}
+
+.row-fluid .offset6 {
+ margin-left: 53.191489361702125%;
+ *margin-left: 53.085106382978715%;
+}
+
+.row-fluid .offset6:first-child {
+ margin-left: 51.063829787234035%;
+ *margin-left: 50.95744680851063%;
+}
+
+.row-fluid .offset5 {
+ margin-left: 44.68085106382979%;
+ *margin-left: 44.57446808510638%;
+}
+
+.row-fluid .offset5:first-child {
+ margin-left: 42.5531914893617%;
+ *margin-left: 42.4468085106383%;
+}
+
+.row-fluid .offset4 {
+ margin-left: 36.170212765957444%;
+ *margin-left: 36.06382978723405%;
+}
+
+.row-fluid .offset4:first-child {
+ margin-left: 34.04255319148936%;
+ *margin-left: 33.93617021276596%;
+}
+
+.row-fluid .offset3 {
+ margin-left: 27.659574468085104%;
+ *margin-left: 27.5531914893617%;
+}
+
+.row-fluid .offset3:first-child {
+ margin-left: 25.53191489361702%;
+ *margin-left: 25.425531914893618%;
+}
+
+.row-fluid .offset2 {
+ margin-left: 19.148936170212764%;
+ *margin-left: 19.04255319148936%;
+}
+
+.row-fluid .offset2:first-child {
+ margin-left: 17.02127659574468%;
+ *margin-left: 16.914893617021278%;
+}
+
+.row-fluid .offset1 {
+ margin-left: 10.638297872340425%;
+ *margin-left: 10.53191489361702%;
+}
+
+.row-fluid .offset1:first-child {
+ margin-left: 8.51063829787234%;
+ *margin-left: 8.404255319148938%;
+}
+
+[class*="span"].hide,
+.row-fluid [class*="span"].hide {
+ display: none;
+}
+
+[class*="span"].pull-right,
+.row-fluid [class*="span"].pull-right {
+ float: right;
+}
+
+.container {
+ margin-right: auto;
+ margin-left: auto;
+ *zoom: 1;
+}
+
+.container:before,
+.container:after {
+ display: table;
+ line-height: 0;
+ content: "";
+}
+
+.container:after {
+ clear: both;
+}
+
+.container-fluid {
+ padding-right: 20px;
+ padding-left: 20px;
+ *zoom: 1;
+}
+
+.container-fluid:before,
+.container-fluid:after {
+ display: table;
+ line-height: 0;
+ content: "";
+}
+
+.container-fluid:after {
+ clear: both;
+}
+
+p {
+ margin: 0 0 10px;
+}
+
+.lead {
+ margin-bottom: 20px;
+ font-size: 20px;
+ font-weight: 200;
+ line-height: 30px;
+}
+
+small {
+ font-size: 85%;
+}
+
+strong {
+ font-weight: bold;
+}
+
+em {
+ font-style: italic;
+}
+
+cite {
+ font-style: normal;
+}
+
+.muted {
+ color: #999999;
+}
+
+h1,
+h2,
+h3,
+h4,
+h5,
+h6 {
+ margin: 10px 0;
+ font-family: inherit;
+ font-weight: bold;
+ line-height: 1;
+ color: inherit;
+ text-rendering: optimizelegibility;
+}
+
+h1 small,
+h2 small,
+h3 small,
+h4 small,
+h5 small,
+h6 small {
+ font-weight: normal;
+ line-height: 1;
+ color: #999999;
+}
+
+h1 {
+ font-size: 36px;
+ line-height: 40px;
+}
+
+h2 {
+ font-size: 30px;
+ line-height: 40px;
+}
+
+h3 {
+ font-size: 24px;
+ line-height: 40px;
+}
+
+h4 {
+ font-size: 18px;
+ line-height: 20px;
+}
+
+h5 {
+ font-size: 14px;
+ line-height: 20px;
+}
+
+h6 {
+ font-size: 12px;
+ line-height: 20px;
+}
+
+h1 small {
+ font-size: 24px;
+}
+
+h2 small {
+ font-size: 18px;
+}
+
+h3 small {
+ font-size: 14px;
+}
+
+h4 small {
+ font-size: 14px;
+}
+
+.page-header {
+ padding-bottom: 9px;
+ margin: 20px 0 30px;
+ border-bottom: 1px solid #eeeeee;
+}
+
+ul,
+ol {
+ padding: 0;
+ margin: 0 0 10px 25px;
+}
+
+ul ul,
+ul ol,
+ol ol,
+ol ul {
+ margin-bottom: 0;
+}
+
+li {
+ line-height: 20px;
+}
+
+ul.unstyled,
+ol.unstyled {
+ margin-left: 0;
+ list-style: none;
+}
+
+dl {
+ margin-bottom: 20px;
+}
+
+dt,
+dd {
+ line-height: 20px;
+}
+
+dt {
+ font-weight: bold;
+}
+
+dd {
+ margin-left: 10px;
+}
+
+.dl-horizontal dt {
+ float: left;
+ width: 120px;
+ overflow: hidden;
+ clear: left;
+ text-align: right;
+ text-overflow: ellipsis;
+ white-space: nowrap;
+}
+
+.dl-horizontal dd {
+ margin-left: 130px;
+}
+
+hr {
+ margin: 20px 0;
+ border: 0;
+ border-top: 1px solid #eeeeee;
+ border-bottom: 1px solid #ffffff;
+}
+
+abbr[title] {
+ cursor: help;
+ border-bottom: 1px dotted #999999;
+}
+
+abbr.initialism {
+ font-size: 90%;
+ text-transform: uppercase;
+}
+
+blockquote {
+ padding: 0 0 0 15px;
+ margin: 0 0 20px;
+ border-left: 5px solid #eeeeee;
+}
+
+blockquote p {
+ margin-bottom: 0;
+ font-size: 16px;
+ font-weight: 300;
+ line-height: 25px;
+}
+
+blockquote small {
+ display: block;
+ line-height: 20px;
+ color: #999999;
+}
+
+blockquote small:before {
+ content: '\2014 \00A0';
+}
+
+blockquote.pull-right {
+ float: right;
+ padding-right: 15px;
+ padding-left: 0;
+ border-right: 5px solid #eeeeee;
+ border-left: 0;
+}
+
+blockquote.pull-right p,
+blockquote.pull-right small {
+ text-align: right;
+}
+
+blockquote.pull-right small:before {
+ content: '';
+}
+
+blockquote.pull-right small:after {
+ content: '\00A0 \2014';
+}
+
+q:before,
+q:after,
+blockquote:before,
+blockquote:after {
+ content: "";
+}
+
+address {
+ display: block;
+ margin-bottom: 20px;
+ font-style: normal;
+ line-height: 20px;
+}
+
+code,
+pre {
+ padding: 0 3px 2px;
+ font-family: Monaco, Menlo, Consolas, "Courier New", monospace;
+ font-size: 12px;
+ color: #333333;
+ -webkit-border-radius: 3px;
+ -moz-border-radius: 3px;
+ border-radius: 3px;
+}
+
+code {
+ padding: 2px 4px;
+ color: #d14;
+ background-color: #f7f7f9;
+ border: 1px solid #e1e1e8;
+}
+
+pre {
+ display: block;
+ padding: 9.5px;
+ margin: 0 0 10px;
+ font-size: 13px;
+ line-height: 20px;
+ word-break: break-all;
+ word-wrap: break-word;
+ white-space: pre;
+ white-space: pre-wrap;
+ background-color: #f5f5f5;
+ border: 1px solid #ccc;
+ border: 1px solid rgba(0, 0, 0, 0.15);
+ -webkit-border-radius: 4px;
+ -moz-border-radius: 4px;
+ border-radius: 4px;
+}
+
+pre.prettyprint {
+ margin-bottom: 20px;
+}
+
+pre code {
+ padding: 0;
+ color: inherit;
+ background-color: transparent;
+ border: 0;
+}
+
+.pre-scrollable {
+ max-height: 340px;
+ overflow-y: scroll;
+}
+
+form {
+ margin: 0 0 20px;
+}
+
+fieldset {
+ padding: 0;
+ margin: 0;
+ border: 0;
+}
+
+legend {
+ display: block;
+ width: 100%;
+ padding: 0;
+ margin-bottom: 20px;
+ font-size: 21px;
+ line-height: 40px;
+ color: #333333;
+ border: 0;
+ border-bottom: 1px solid #e5e5e5;
+}
+
+legend small {
+ font-size: 15px;
+ color: #999999;
+}
+
+label,
+input,
+button,
+select,
+textarea {
+ font-size: 14px;
+ font-weight: normal;
+ line-height: 20px;
+}
+
+input,
+button,
+select,
+textarea {
+ font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
+}
+
+label {
+ display: block;
+ margin-bottom: 5px;
+}
+
+select,
+textarea,
+input[type="text"],
+input[type="password"],
+input[type="datetime"],
+input[type="datetime-local"],
+input[type="date"],
+input[type="month"],
+input[type="time"],
+input[type="week"],
+input[type="number"],
+input[type="email"],
+input[type="url"],
+input[type="search"],
+input[type="tel"],
+input[type="color"],
+.uneditable-input {
+ display: inline-block;
+ height: 20px;
+ padding: 4px 6px;
+ margin-bottom: 9px;
+ font-size: 14px;
+ line-height: 20px;
+ color: #555555;
+ -webkit-border-radius: 3px;
+ -moz-border-radius: 3px;
+ border-radius: 3px;
+}
+
+input,
+textarea {
+ width: 210px;
+}
+
+textarea {
+ height: auto;
+}
+
+textarea,
+input[type="text"],
+input[type="password"],
+input[type="datetime"],
+input[type="datetime-local"],
+input[type="date"],
+input[type="month"],
+input[type="time"],
+input[type="week"],
+input[type="number"],
+input[type="email"],
+input[type="url"],
+input[type="search"],
+input[type="tel"],
+input[type="color"],
+.uneditable-input {
+ background-color: #ffffff;
+ border: 1px solid #cccccc;
+ -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+ -moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+ box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+ -webkit-transition: border linear 0.2s, box-shadow linear 0.2s;
+ -moz-transition: border linear 0.2s, box-shadow linear 0.2s;
+ -o-transition: border linear 0.2s, box-shadow linear 0.2s;
+ transition: border linear 0.2s, box-shadow linear 0.2s;
+}
+
+textarea:focus,
+input[type="text"]:focus,
+input[type="password"]:focus,
+input[type="datetime"]:focus,
+input[type="datetime-local"]:focus,
+input[type="date"]:focus,
+input[type="month"]:focus,
+input[type="time"]:focus,
+input[type="week"]:focus,
+input[type="number"]:focus,
+input[type="email"]:focus,
+input[type="url"]:focus,
+input[type="search"]:focus,
+input[type="tel"]:focus,
+input[type="color"]:focus,
+.uneditable-input:focus {
+ border-color: rgba(82, 168, 236, 0.8);
+ outline: 0;
+ outline: thin dotted \9;
+ /* IE6-9 */
+
+ -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(82, 168, 236, 0.6);
+ -moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(82, 168, 236, 0.6);
+ box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(82, 168, 236, 0.6);
+}
+
+input[type="radio"],
+input[type="checkbox"] {
+ margin: 4px 0 0;
+ margin-top: 1px \9;
+ *margin-top: 0;
+ line-height: normal;
+ cursor: pointer;
+}
+
+input[type="file"],
+input[type="image"],
+input[type="submit"],
+input[type="reset"],
+input[type="button"],
+input[type="radio"],
+input[type="checkbox"] {
+ width: auto;
+}
+
+select,
+input[type="file"] {
+ height: 30px;
+ /* In IE7, the height of the select element cannot be changed by height, only font-size */
+
+ *margin-top: 4px;
+ /* For IE7, add top margin to align select with labels */
+
+ line-height: 30px;
+}
+
+select {
+ width: 220px;
+ background-color: #ffffff;
+ border: 1px solid #bbb;
+}
+
+select[multiple],
+select[size] {
+ height: auto;
+}
+
+select:focus,
+input[type="file"]:focus,
+input[type="radio"]:focus,
+input[type="checkbox"]:focus {
+ outline: thin dotted #333;
+ outline: 5px auto -webkit-focus-ring-color;
+ outline-offset: -2px;
+}
+
+.uneditable-input,
+.uneditable-textarea {
+ color: #999999;
+ cursor: not-allowed;
+ background-color: #fcfcfc;
+ border-color: #cccccc;
+ -webkit-box-shadow: inset 0 1px 2px rgba(0, 0, 0, 0.025);
+ -moz-box-shadow: inset 0 1px 2px rgba(0, 0, 0, 0.025);
+ box-shadow: inset 0 1px 2px rgba(0, 0, 0, 0.025);
+}
+
+.uneditable-input {
+ overflow: hidden;
+ white-space: nowrap;
+}
+
+.uneditable-textarea {
+ width: auto;
+ height: auto;
+}
+
+input:-moz-placeholder,
+textarea:-moz-placeholder {
+ color: #999999;
+}
+
+input:-ms-input-placeholder,
+textarea:-ms-input-placeholder {
+ color: #999999;
+}
+
+input::-webkit-input-placeholder,
+textarea::-webkit-input-placeholder {
+ color: #999999;
+}
+
+.radio,
+.checkbox {
+ min-height: 18px;
+ padding-left: 18px;
+}
+
+.radio input[type="radio"],
+.checkbox input[type="checkbox"] {
+ float: left;
+ margin-left: -18px;
+}
+
+.controls > .radio:first-child,
+.controls > .checkbox:first-child {
+ padding-top: 5px;
+}
+
+.radio.inline,
+.checkbox.inline {
+ display: inline-block;
+ padding-top: 5px;
+ margin-bottom: 0;
+ vertical-align: middle;
+}
+
+.radio.inline + .radio.inline,
+.checkbox.inline + .checkbox.inline {
+ margin-left: 10px;
+}
+
+.input-mini {
+ width: 60px;
+}
+
+.input-small {
+ width: 90px;
+}
+
+.input-medium {
+ width: 150px;
+}
+
+.input-large {
+ width: 210px;
+}
+
+.input-xlarge {
+ width: 270px;
+}
+
+.input-xxlarge {
+ width: 530px;
+}
+
+input[class*="span"],
+select[class*="span"],
+textarea[class*="span"],
+.uneditable-input[class*="span"],
+.row-fluid input[class*="span"],
+.row-fluid select[class*="span"],
+.row-fluid textarea[class*="span"],
+.row-fluid .uneditable-input[class*="span"] {
+ float: none;
+ margin-left: 0;
+}
+
+.input-append input[class*="span"],
+.input-append .uneditable-input[class*="span"],
+.input-prepend input[class*="span"],
+.input-prepend .uneditable-input[class*="span"],
+.row-fluid input[class*="span"],
+.row-fluid select[class*="span"],
+.row-fluid textarea[class*="span"],
+.row-fluid .uneditable-input[class*="span"],
+.row-fluid .input-prepend [class*="span"],
+.row-fluid .input-append [class*="span"] {
+ display: inline-block;
+}
+
+input,
+textarea,
+.uneditable-input {
+ margin-left: 0;
+}
+
+.controls-row [class*="span"] + [class*="span"] {
+ margin-left: 20px;
+}
+
+input.span12,
+textarea.span12,
+.uneditable-input.span12 {
+ width: 926px;
+}
+
+input.span11,
+textarea.span11,
+.uneditable-input.span11 {
+ width: 846px;
+}
+
+input.span10,
+textarea.span10,
+.uneditable-input.span10 {
+ width: 766px;
+}
+
+input.span9,
+textarea.span9,
+.uneditable-input.span9 {
+ width: 686px;
+}
+
+input.span8,
+textarea.span8,
+.uneditable-input.span8 {
+ width: 606px;
+}
+
+input.span7,
+textarea.span7,
+.uneditable-input.span7 {
+ width: 526px;
+}
+
+input.span6,
+textarea.span6,
+.uneditable-input.span6 {
+ width: 446px;
+}
+
+input.span5,
+textarea.span5,
+.uneditable-input.span5 {
+ width: 366px;
+}
+
+input.span4,
+textarea.span4,
+.uneditable-input.span4 {
+ width: 286px;
+}
+
+input.span3,
+textarea.span3,
+.uneditable-input.span3 {
+ width: 206px;
+}
+
+input.span2,
+textarea.span2,
+.uneditable-input.span2 {
+ width: 126px;
+}
+
+input.span1,
+textarea.span1,
+.uneditable-input.span1 {
+ width: 46px;
+}
+
+.controls-row {
+ *zoom: 1;
+}
+
+.controls-row:before,
+.controls-row:after {
+ display: table;
+ line-height: 0;
+ content: "";
+}
+
+.controls-row:after {
+ clear: both;
+}
+
+.controls-row [class*="span"] {
+ float: left;
+}
+
+input[disabled],
+select[disabled],
+textarea[disabled],
+input[readonly],
+select[readonly],
+textarea[readonly] {
+ cursor: not-allowed;
+ background-color: #eeeeee;
+}
+
+input[type="radio"][disabled],
+input[type="checkbox"][disabled],
+input[type="radio"][readonly],
+input[type="checkbox"][readonly] {
+ background-color: transparent;
+}
+
+.control-group.warning > label,
+.control-group.warning .help-block,
+.control-group.warning .help-inline {
+ color: #c09853;
+}
+
+.control-group.warning .checkbox,
+.control-group.warning .radio,
+.control-group.warning input,
+.control-group.warning select,
+.control-group.warning textarea {
+ color: #c09853;
+ border-color: #c09853;
+ -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+ -moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+ box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+}
+
+.control-group.warning .checkbox:focus,
+.control-group.warning .radio:focus,
+.control-group.warning input:focus,
+.control-group.warning select:focus,
+.control-group.warning textarea:focus {
+ border-color: #a47e3c;
+ -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #dbc59e;
+ -moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #dbc59e;
+ box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #dbc59e;
+}
+
+.control-group.warning .input-prepend .add-on,
+.control-group.warning .input-append .add-on {
+ color: #c09853;
+ background-color: #fcf8e3;
+ border-color: #c09853;
+}
+
+.control-group.error > label,
+.control-group.error .help-block,
+.control-group.error .help-inline {
+ color: #b94a48;
+}
+
+.control-group.error .checkbox,
+.control-group.error .radio,
+.control-group.error input,
+.control-group.error select,
+.control-group.error textarea {
+ color: #b94a48;
+ border-color: #b94a48;
+ -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+ -moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+ box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+}
+
+.control-group.error .checkbox:focus,
+.control-group.error .radio:focus,
+.control-group.error input:focus,
+.control-group.error select:focus,
+.control-group.error textarea:focus {
+ border-color: #953b39;
+ -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #d59392;
+ -moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #d59392;
+ box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #d59392;
+}
+
+.control-group.error .input-prepend .add-on,
+.control-group.error .input-append .add-on {
+ color: #b94a48;
+ background-color: #f2dede;
+ border-color: #b94a48;
+}
+
+.control-group.success > label,
+.control-group.success .help-block,
+.control-group.success .help-inline {
+ color: #468847;
+}
+
+.control-group.success .checkbox,
+.control-group.success .radio,
+.control-group.success input,
+.control-group.success select,
+.control-group.success textarea {
+ color: #468847;
+ border-color: #468847;
+ -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+ -moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+ box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+}
+
+.control-group.success .checkbox:focus,
+.control-group.success .radio:focus,
+.control-group.success input:focus,
+.control-group.success select:focus,
+.control-group.success textarea:focus {
+ border-color: #356635;
+ -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #7aba7b;
+ -moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #7aba7b;
+ box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #7aba7b;
+}
+
+.control-group.success .input-prepend .add-on,
+.control-group.success .input-append .add-on {
+ color: #468847;
+ background-color: #dff0d8;
+ border-color: #468847;
+}
+
+input:focus:required:invalid,
+textarea:focus:required:invalid,
+select:focus:required:invalid {
+ color: #b94a48;
+ border-color: #ee5f5b;
+}
+
+input:focus:required:invalid:focus,
+textarea:focus:required:invalid:focus,
+select:focus:required:invalid:focus {
+ border-color: #e9322d;
+ -webkit-box-shadow: 0 0 6px #f8b9b7;
+ -moz-box-shadow: 0 0 6px #f8b9b7;
+ box-shadow: 0 0 6px #f8b9b7;
+}
+
+.form-actions {
+ padding: 19px 20px 20px;
+ margin-top: 20px;
+ margin-bottom: 20px;
+ background-color: #f5f5f5;
+ border-top: 1px solid #e5e5e5;
+ *zoom: 1;
+}
+
+.form-actions:before,
+.form-actions:after {
+ display: table;
+ line-height: 0;
+ content: "";
+}
+
+.form-actions:after {
+ clear: both;
+}
+
+.help-block,
+.help-inline {
+ color: #595959;
+}
+
+.help-block {
+ display: block;
+ margin-bottom: 10px;
+}
+
+.help-inline {
+ display: inline-block;
+ *display: inline;
+ padding-left: 5px;
+ vertical-align: middle;
+ *zoom: 1;
+}
+
+.input-append,
+.input-prepend {
+ margin-bottom: 5px;
+ font-size: 0;
+ white-space: nowrap;
+}
+
+.input-append input,
+.input-prepend input,
+.input-append select,
+.input-prepend select,
+.input-append .uneditable-input,
+.input-prepend .uneditable-input {
+ position: relative;
+ margin-bottom: 0;
+ *margin-left: 0;
+ font-size: 14px;
+ vertical-align: top;
+ -webkit-border-radius: 0 3px 3px 0;
+ -moz-border-radius: 0 3px 3px 0;
+ border-radius: 0 3px 3px 0;
+}
+
+.input-append input:focus,
+.input-prepend input:focus,
+.input-append select:focus,
+.input-prepend select:focus,
+.input-append .uneditable-input:focus,
+.input-prepend .uneditable-input:focus {
+ z-index: 2;
+}
+
+.input-append .add-on,
+.input-prepend .add-on {
+ display: inline-block;
+ width: auto;
+ height: 20px;
+ min-width: 16px;
+ padding: 4px 5px;
+ font-size: 14px;
+ font-weight: normal;
+ line-height: 20px;
+ text-align: center;
+ text-shadow: 0 1px 0 #ffffff;
+ background-color: #eeeeee;
+ border: 1px solid #ccc;
+}
+
+.input-append .add-on,
+.input-prepend .add-on,
+.input-append .btn,
+.input-prepend .btn {
+ margin-left: -1px;
+ vertical-align: top;
+ -webkit-border-radius: 0;
+ -moz-border-radius: 0;
+ border-radius: 0;
+}
+
+.input-append .active,
+.input-prepend .active {
+ background-color: #a9dba9;
+ border-color: #46a546;
+}
+
+.input-prepend .add-on,
+.input-prepend .btn {
+ margin-right: -1px;
+}
+
+.input-prepend .add-on:first-child,
+.input-prepend .btn:first-child {
+ -webkit-border-radius: 3px 0 0 3px;
+ -moz-border-radius: 3px 0 0 3px;
+ border-radius: 3px 0 0 3px;
+}
+
+.input-append input,
+.input-append select,
+.input-append .uneditable-input {
+ -webkit-border-radius: 3px 0 0 3px;
+ -moz-border-radius: 3px 0 0 3px;
+ border-radius: 3px 0 0 3px;
+}
+
+.input-append .add-on:last-child,
+.input-append .btn:last-child {
+ -webkit-border-radius: 0 3px 3px 0;
+ -moz-border-radius: 0 3px 3px 0;
+ border-radius: 0 3px 3px 0;
+}
+
+.input-prepend.input-append input,
+.input-prepend.input-append select,
+.input-prepend.input-append .uneditable-input {
+ -webkit-border-radius: 0;
+ -moz-border-radius: 0;
+ border-radius: 0;
+}
+
+.input-prepend.input-append .add-on:first-child,
+.input-prepend.input-append .btn:first-child {
+ margin-right: -1px;
+ -webkit-border-radius: 3px 0 0 3px;
+ -moz-border-radius: 3px 0 0 3px;
+ border-radius: 3px 0 0 3px;
+}
+
+.input-prepend.input-append .add-on:last-child,
+.input-prepend.input-append .btn:last-child {
+ margin-left: -1px;
+ -webkit-border-radius: 0 3px 3px 0;
+ -moz-border-radius: 0 3px 3px 0;
+ border-radius: 0 3px 3px 0;
+}
+
+input.search-query {
+ padding-right: 14px;
+ padding-right: 4px \9;
+ padding-left: 14px;
+ padding-left: 4px \9;
+ /* IE7-8 doesn't have border-radius, so don't indent the padding */
+
+ margin-bottom: 0;
+ -webkit-border-radius: 15px;
+ -moz-border-radius: 15px;
+ border-radius: 15px;
+}
+
+/* Allow for input prepend/append in search forms */
+
+.form-search .input-append .search-query,
+.form-search .input-prepend .search-query {
+ -webkit-border-radius: 0;
+ -moz-border-radius: 0;
+ border-radius: 0;
+}
+
+.form-search .input-append .search-query {
+ -webkit-border-radius: 14px 0 0 14px;
+ -moz-border-radius: 14px 0 0 14px;
+ border-radius: 14px 0 0 14px;
+}
+
+.form-search .input-append .btn {
+ -webkit-border-radius: 0 14px 14px 0;
+ -moz-border-radius: 0 14px 14px 0;
+ border-radius: 0 14px 14px 0;
+}
+
+.form-search .input-prepend .search-query {
+ -webkit-border-radius: 0 14px 14px 0;
+ -moz-border-radius: 0 14px 14px 0;
+ border-radius: 0 14px 14px 0;
+}
+
+.form-search .input-prepend .btn {
+ -webkit-border-radius: 14px 0 0 14px;
+ -moz-border-radius: 14px 0 0 14px;
+ border-radius: 14px 0 0 14px;
+}
+
+.form-search input,
+.form-inline input,
+.form-horizontal input,
+.form-search textarea,
+.form-inline textarea,
+.form-horizontal textarea,
+.form-search select,
+.form-inline select,
+.form-horizontal select,
+.form-search .help-inline,
+.form-inline .help-inline,
+.form-horizontal .help-inline,
+.form-search .uneditable-input,
+.form-inline .uneditable-input,
+.form-horizontal .uneditable-input,
+.form-search .input-prepend,
+.form-inline .input-prepend,
+.form-horizontal .input-prepend,
+.form-search .input-append,
+.form-inline .input-append,
+.form-horizontal .input-append {
+ display: inline-block;
+ *display: inline;
+ margin-bottom: 0;
+ vertical-align: middle;
+ *zoom: 1;
+}
+
+.form-search .hide,
+.form-inline .hide,
+.form-horizontal .hide {
+ display: none;
+}
+
+.form-search label,
+.form-inline label,
+.form-search .btn-group,
+.form-inline .btn-group {
+ display: inline-block;
+}
+
+.form-search .input-append,
+.form-inline .input-append,
+.form-search .input-prepend,
+.form-inline .input-prepend {
+ margin-bottom: 0;
+}
+
+.form-search .radio,
+.form-search .checkbox,
+.form-inline .radio,
+.form-inline .checkbox {
+ padding-left: 0;
+ margin-bottom: 0;
+ vertical-align: middle;
+}
+
+.form-search .radio input[type="radio"],
+.form-search .checkbox input[type="checkbox"],
+.form-inline .radio input[type="radio"],
+.form-inline .checkbox input[type="checkbox"] {
+ float: left;
+ margin-right: 3px;
+ margin-left: 0;
+}
+
+.control-group {
+ margin-bottom: 10px;
+}
+
+legend + .control-group {
+ margin-top: 20px;
+ -webkit-margin-top-collapse: separate;
+}
+
+.form-horizontal .control-group {
+ margin-bottom: 20px;
+ *zoom: 1;
+}
+
+.form-horizontal .control-group:before,
+.form-horizontal .control-group:after {
+ display: table;
+ line-height: 0;
+ content: "";
+}
+
+.form-horizontal .control-group:after {
+ clear: both;
+}
+
+.form-horizontal .control-label {
+ float: left;
+ width: 140px;
+ padding-top: 5px;
+ text-align: right;
+}
+
+.form-horizontal .controls {
+ *display: inline-block;
+ *padding-left: 20px;
+ margin-left: 160px;
+ *margin-left: 0;
+}
+
+.form-horizontal .controls:first-child {
+ *padding-left: 160px;
+}
+
+.form-horizontal .help-block {
+ margin-top: 10px;
+ margin-bottom: 0;
+}
+
+.form-horizontal .form-actions {
+ padding-left: 160px;
+}
+
+table {
+ max-width: 100%;
+ background-color: transparent;
+ border-collapse: collapse;
+ border-spacing: 0;
+}
+
+.table {
+ width: 100%;
+ margin-bottom: 20px;
+}
+
+.table th,
+.table td {
+ padding: 8px;
+ line-height: 20px;
+ text-align: left;
+ vertical-align: top;
+ border-top: 1px solid #dddddd;
+}
+
+.table th {
+ font-weight: bold;
+}
+
+.table thead th {
+ vertical-align: bottom;
+}
+
+.table caption + thead tr:first-child th,
+.table caption + thead tr:first-child td,
+.table colgroup + thead tr:first-child th,
+.table colgroup + thead tr:first-child td,
+.table thead:first-child tr:first-child th,
+.table thead:first-child tr:first-child td {
+ border-top: 0;
+}
+
+.table tbody + tbody {
+ border-top: 2px solid #dddddd;
+}
+
+.table-condensed th,
+.table-condensed td {
+ padding: 4px 5px;
+}
+
+.table-bordered {
+ border: 1px solid #dddddd;
+ border-collapse: separate;
+ *border-collapse: collapse;
+ border-left: 0;
+ -webkit-border-radius: 4px;
+ -moz-border-radius: 4px;
+ border-radius: 4px;
+}
+
+.table-bordered th,
+.table-bordered td {
+ border-left: 1px solid #dddddd;
+}
+
+.table-bordered caption + thead tr:first-child th,
+.table-bordered caption + tbody tr:first-child th,
+.table-bordered caption + tbody tr:first-child td,
+.table-bordered colgroup + thead tr:first-child th,
+.table-bordered colgroup + tbody tr:first-child th,
+.table-bordered colgroup + tbody tr:first-child td,
+.table-bordered thead:first-child tr:first-child th,
+.table-bordered tbody:first-child tr:first-child th,
+.table-bordered tbody:first-child tr:first-child td {
+ border-top: 0;
+}
+
+.table-bordered thead:first-child tr:first-child th:first-child,
+.table-bordered tbody:first-child tr:first-child td:first-child {
+ -webkit-border-top-left-radius: 4px;
+ border-top-left-radius: 4px;
+ -moz-border-radius-topleft: 4px;
+}
+
+.table-bordered thead:first-child tr:first-child th:last-child,
+.table-bordered tbody:first-child tr:first-child td:last-child {
+ -webkit-border-top-right-radius: 4px;
+ border-top-right-radius: 4px;
+ -moz-border-radius-topright: 4px;
+}
+
+.table-bordered thead:last-child tr:last-child th:first-child,
+.table-bordered tbody:last-child tr:last-child td:first-child,
+.table-bordered tfoot:last-child tr:last-child td:first-child {
+ -webkit-border-radius: 0 0 0 4px;
+ -moz-border-radius: 0 0 0 4px;
+ border-radius: 0 0 0 4px;
+ -webkit-border-bottom-left-radius: 4px;
+ border-bottom-left-radius: 4px;
+ -moz-border-radius-bottomleft: 4px;
+}
+
+.table-bordered thead:last-child tr:last-child th:last-child,
+.table-bordered tbody:last-child tr:last-child td:last-child,
+.table-bordered tfoot:last-child tr:last-child td:last-child {
+ -webkit-border-bottom-right-radius: 4px;
+ border-bottom-right-radius: 4px;
+ -moz-border-radius-bottomright: 4px;
+}
+
+.table-bordered caption + thead tr:first-child th:first-child,
+.table-bordered caption + tbody tr:first-child td:first-child,
+.table-bordered colgroup + thead tr:first-child th:first-child,
+.table-bordered colgroup + tbody tr:first-child td:first-child {
+ -webkit-border-top-left-radius: 4px;
+ border-top-left-radius: 4px;
+ -moz-border-radius-topleft: 4px;
+}
+
+.table-bordered caption + thead tr:first-child th:last-child,
+.table-bordered caption + tbody tr:first-child td:last-child,
+.table-bordered colgroup + thead tr:first-child th:last-child,
+.table-bordered colgroup + tbody tr:first-child td:last-child {
+ -webkit-border-top-right-radius: 4px;
+ border-top-right-radius: 4px;
+ -moz-border-right-topleft: 4px;
+}
+
+.table-striped tbody tr:nth-child(odd) td,
+.table-striped tbody tr:nth-child(odd) th {
+ background-color: #f9f9f9;
+}
+
+.table-hover tbody tr:hover td,
+.table-hover tbody tr:hover th {
+ background-color: #f5f5f5;
+}
+
+table [class*=span],
+.row-fluid table [class*=span] {
+ display: table-cell;
+ float: none;
+ margin-left: 0;
+}
+
+table .span1 {
+ float: none;
+ width: 44px;
+ margin-left: 0;
+}
+
+table .span2 {
+ float: none;
+ width: 124px;
+ margin-left: 0;
+}
+
+table .span3 {
+ float: none;
+ width: 204px;
+ margin-left: 0;
+}
+
+table .span4 {
+ float: none;
+ width: 284px;
+ margin-left: 0;
+}
+
+table .span5 {
+ float: none;
+ width: 364px;
+ margin-left: 0;
+}
+
+table .span6 {
+ float: none;
+ width: 444px;
+ margin-left: 0;
+}
+
+table .span7 {
+ float: none;
+ width: 524px;
+ margin-left: 0;
+}
+
+table .span8 {
+ float: none;
+ width: 604px;
+ margin-left: 0;
+}
+
+table .span9 {
+ float: none;
+ width: 684px;
+ margin-left: 0;
+}
+
+table .span10 {
+ float: none;
+ width: 764px;
+ margin-left: 0;
+}
+
+table .span11 {
+ float: none;
+ width: 844px;
+ margin-left: 0;
+}
+
+table .span12 {
+ float: none;
+ width: 924px;
+ margin-left: 0;
+}
+
+table .span13 {
+ float: none;
+ width: 1004px;
+ margin-left: 0;
+}
+
+table .span14 {
+ float: none;
+ width: 1084px;
+ margin-left: 0;
+}
+
+table .span15 {
+ float: none;
+ width: 1164px;
+ margin-left: 0;
+}
+
+table .span16 {
+ float: none;
+ width: 1244px;
+ margin-left: 0;
+}
+
+table .span17 {
+ float: none;
+ width: 1324px;
+ margin-left: 0;
+}
+
+table .span18 {
+ float: none;
+ width: 1404px;
+ margin-left: 0;
+}
+
+table .span19 {
+ float: none;
+ width: 1484px;
+ margin-left: 0;
+}
+
+table .span20 {
+ float: none;
+ width: 1564px;
+ margin-left: 0;
+}
+
+table .span21 {
+ float: none;
+ width: 1644px;
+ margin-left: 0;
+}
+
+table .span22 {
+ float: none;
+ width: 1724px;
+ margin-left: 0;
+}
+
+table .span23 {
+ float: none;
+ width: 1804px;
+ margin-left: 0;
+}
+
+table .span24 {
+ float: none;
+ width: 1884px;
+ margin-left: 0;
+}
+
+.table tbody tr.success td {
+ background-color: #dff0d8;
+}
+
+.table tbody tr.error td {
+ background-color: #f2dede;
+}
+
+.table tbody tr.info td {
+ background-color: #d9edf7;
+}
+
+[class^="icon-"],
+[class*=" icon-"] {
+ display: inline-block;
+ width: 14px;
+ height: 14px;
+ margin-top: 1px;
+ *margin-right: .3em;
+ line-height: 14px;
+ vertical-align: text-top;
+ background-image: url("../img/glyphicons-halflings.png");
+ background-position: 14px 14px;
+ background-repeat: no-repeat;
+}
+
+/* White icons with optional class, or on hover/active states of certain elements */
+
+.icon-white,
+.nav > .active > a > [class^="icon-"],
+.nav > .active > a > [class*=" icon-"],
+.dropdown-menu > li > a:hover > [class^="icon-"],
+.dropdown-menu > li > a:hover > [class*=" icon-"],
+.dropdown-menu > .active > a > [class^="icon-"],
+.dropdown-menu > .active > a > [class*=" icon-"] {
+ background-image: url("../img/glyphicons-halflings-white.png");
+}
+
+.icon-glass {
+ background-position: 0 0;
+}
+
+.icon-music {
+ background-position: -24px 0;
+}
+
+.icon-search {
+ background-position: -48px 0;
+}
+
+.icon-envelope {
+ background-position: -72px 0;
+}
+
+.icon-heart {
+ background-position: -96px 0;
+}
+
+.icon-star {
+ background-position: -120px 0;
+}
+
+.icon-star-empty {
+ background-position: -144px 0;
+}
+
+.icon-user {
+ background-position: -168px 0;
+}
+
+.icon-film {
+ background-position: -192px 0;
+}
+
+.icon-th-large {
+ background-position: -216px 0;
+}
+
+.icon-th {
+ background-position: -240px 0;
+}
+
+.icon-th-list {
+ background-position: -264px 0;
+}
+
+.icon-ok {
+ background-position: -288px 0;
+}
+
+.icon-remove {
+ background-position: -312px 0;
+}
+
+.icon-zoom-in {
+ background-position: -336px 0;
+}
+
+.icon-zoom-out {
+ background-position: -360px 0;
+}
+
+.icon-off {
+ background-position: -384px 0;
+}
+
+.icon-signal {
+ background-position: -408px 0;
+}
+
+.icon-cog {
+ background-position: -432px 0;
+}
+
+.icon-trash {
+ background-position: -456px 0;
+}
+
+.icon-home {
+ background-position: 0 -24px;
+}
+
+.icon-file {
+ background-position: -24px -24px;
+}
+
+.icon-time {
+ background-position: -48px -24px;
+}
+
+.icon-road {
+ background-position: -72px -24px;
+}
+
+.icon-download-alt {
+ background-position: -96px -24px;
+}
+
+.icon-download {
+ background-position: -120px -24px;
+}
+
+.icon-upload {
+ background-position: -144px -24px;
+}
+
+.icon-inbox {
+ background-position: -168px -24px;
+}
+
+.icon-play-circle {
+ background-position: -192px -24px;
+}
+
+.icon-repeat {
+ background-position: -216px -24px;
+}
+
+.icon-refresh {
+ background-position: -240px -24px;
+}
+
+.icon-list-alt {
+ background-position: -264px -24px;
+}
+
+.icon-lock {
+ background-position: -287px -24px;
+}
+
+.icon-flag {
+ background-position: -312px -24px;
+}
+
+.icon-headphones {
+ background-position: -336px -24px;
+}
+
+.icon-volume-off {
+ background-position: -360px -24px;
+}
+
+.icon-volume-down {
+ background-position: -384px -24px;
+}
+
+.icon-volume-up {
+ background-position: -408px -24px;
+}
+
+.icon-qrcode {
+ background-position: -432px -24px;
+}
+
+.icon-barcode {
+ background-position: -456px -24px;
+}
+
+.icon-tag {
+ background-position: 0 -48px;
+}
+
+.icon-tags {
+ background-position: -25px -48px;
+}
+
+.icon-book {
+ background-position: -48px -48px;
+}
+
+.icon-bookmark {
+ background-position: -72px -48px;
+}
+
+.icon-print {
+ background-position: -96px -48px;
+}
+
+.icon-camera {
+ background-position: -120px -48px;
+}
+
+.icon-font {
+ background-position: -144px -48px;
+}
+
+.icon-bold {
+ background-position: -167px -48px;
+}
+
+.icon-italic {
+ background-position: -192px -48px;
+}
+
+.icon-text-height {
+ background-position: -216px -48px;
+}
+
+.icon-text-width {
+ background-position: -240px -48px;
+}
+
+.icon-align-left {
+ background-position: -264px -48px;
+}
+
+.icon-align-center {
+ background-position: -288px -48px;
+}
+
+.icon-align-right {
+ background-position: -312px -48px;
+}
+
+.icon-align-justify {
+ background-position: -336px -48px;
+}
+
+.icon-list {
+ background-position: -360px -48px;
+}
+
+.icon-indent-left {
+ background-position: -384px -48px;
+}
+
+.icon-indent-right {
+ background-position: -408px -48px;
+}
+
+.icon-facetime-video {
+ background-position: -432px -48px;
+}
+
+.icon-picture {
+ background-position: -456px -48px;
+}
+
+.icon-pencil {
+ background-position: 0 -72px;
+}
+
+.icon-map-marker {
+ background-position: -24px -72px;
+}
+
+.icon-adjust {
+ background-position: -48px -72px;
+}
+
+.icon-tint {
+ background-position: -72px -72px;
+}
+
+.icon-edit {
+ background-position: -96px -72px;
+}
+
+.icon-share {
+ background-position: -120px -72px;
+}
+
+.icon-check {
+ background-position: -144px -72px;
+}
+
+.icon-move {
+ background-position: -168px -72px;
+}
+
+.icon-step-backward {
+ background-position: -192px -72px;
+}
+
+.icon-fast-backward {
+ background-position: -216px -72px;
+}
+
+.icon-backward {
+ background-position: -240px -72px;
+}
+
+.icon-play {
+ background-position: -264px -72px;
+}
+
+.icon-pause {
+ background-position: -288px -72px;
+}
+
+.icon-stop {
+ background-position: -312px -72px;
+}
+
+.icon-forward {
+ background-position: -336px -72px;
+}
+
+.icon-fast-forward {
+ background-position: -360px -72px;
+}
+
+.icon-step-forward {
+ background-position: -384px -72px;
+}
+
+.icon-eject {
+ background-position: -408px -72px;
+}
+
+.icon-chevron-left {
+ background-position: -432px -72px;
+}
+
+.icon-chevron-right {
+ background-position: -456px -72px;
+}
+
+.icon-plus-sign {
+ background-position: 0 -96px;
+}
+
+.icon-minus-sign {
+ background-position: -24px -96px;
+}
+
+.icon-remove-sign {
+ background-position: -48px -96px;
+}
+
+.icon-ok-sign {
+ background-position: -72px -96px;
+}
+
+.icon-question-sign {
+ background-position: -96px -96px;
+}
+
+.icon-info-sign {
+ background-position: -120px -96px;
+}
+
+.icon-screenshot {
+ background-position: -144px -96px;
+}
+
+.icon-remove-circle {
+ background-position: -168px -96px;
+}
+
+.icon-ok-circle {
+ background-position: -192px -96px;
+}
+
+.icon-ban-circle {
+ background-position: -216px -96px;
+}
+
+.icon-arrow-left {
+ background-position: -240px -96px;
+}
+
+.icon-arrow-right {
+ background-position: -264px -96px;
+}
+
+.icon-arrow-up {
+ background-position: -289px -96px;
+}
+
+.icon-arrow-down {
+ background-position: -312px -96px;
+}
+
+.icon-share-alt {
+ background-position: -336px -96px;
+}
+
+.icon-resize-full {
+ background-position: -360px -96px;
+}
+
+.icon-resize-small {
+ background-position: -384px -96px;
+}
+
+.icon-plus {
+ background-position: -408px -96px;
+}
+
+.icon-minus {
+ background-position: -433px -96px;
+}
+
+.icon-asterisk {
+ background-position: -456px -96px;
+}
+
+.icon-exclamation-sign {
+ background-position: 0 -120px;
+}
+
+.icon-gift {
+ background-position: -24px -120px;
+}
+
+.icon-leaf {
+ background-position: -48px -120px;
+}
+
+.icon-fire {
+ background-position: -72px -120px;
+}
+
+.icon-eye-open {
+ background-position: -96px -120px;
+}
+
+.icon-eye-close {
+ background-position: -120px -120px;
+}
+
+.icon-warning-sign {
+ background-position: -144px -120px;
+}
+
+.icon-plane {
+ background-position: -168px -120px;
+}
+
+.icon-calendar {
+ background-position: -192px -120px;
+}
+
+.icon-random {
+ width: 16px;
+ background-position: -216px -120px;
+}
+
+.icon-comment {
+ background-position: -240px -120px;
+}
+
+.icon-magnet {
+ background-position: -264px -120px;
+}
+
+.icon-chevron-up {
+ background-position: -288px -120px;
+}
+
+.icon-chevron-down {
+ background-position: -313px -119px;
+}
+
+.icon-retweet {
+ background-position: -336px -120px;
+}
+
+.icon-shopping-cart {
+ background-position: -360px -120px;
+}
+
+.icon-folder-close {
+ background-position: -384px -120px;
+}
+
+.icon-folder-open {
+ width: 16px;
+ background-position: -408px -120px;
+}
+
+.icon-resize-vertical {
+ background-position: -432px -119px;
+}
+
+.icon-resize-horizontal {
+ background-position: -456px -118px;
+}
+
+.icon-hdd {
+ background-position: 0 -144px;
+}
+
+.icon-bullhorn {
+ background-position: -24px -144px;
+}
+
+.icon-bell {
+ background-position: -48px -144px;
+}
+
+.icon-certificate {
+ background-position: -72px -144px;
+}
+
+.icon-thumbs-up {
+ background-position: -96px -144px;
+}
+
+.icon-thumbs-down {
+ background-position: -120px -144px;
+}
+
+.icon-hand-right {
+ background-position: -144px -144px;
+}
+
+.icon-hand-left {
+ background-position: -168px -144px;
+}
+
+.icon-hand-up {
+ background-position: -192px -144px;
+}
+
+.icon-hand-down {
+ background-position: -216px -144px;
+}
+
+.icon-circle-arrow-right {
+ background-position: -240px -144px;
+}
+
+.icon-circle-arrow-left {
+ background-position: -264px -144px;
+}
+
+.icon-circle-arrow-up {
+ background-position: -288px -144px;
+}
+
+.icon-circle-arrow-down {
+ background-position: -312px -144px;
+}
+
+.icon-globe {
+ background-position: -336px -144px;
+}
+
+.icon-wrench {
+ background-position: -360px -144px;
+}
+
+.icon-tasks {
+ background-position: -384px -144px;
+}
+
+.icon-filter {
+ background-position: -408px -144px;
+}
+
+.icon-briefcase {
+ background-position: -432px -144px;
+}
+
+.icon-fullscreen {
+ background-position: -456px -144px;
+}
+
+.dropup,
+.dropdown {
+ position: relative;
+}
+
+.dropdown-toggle {
+ *margin-bottom: -3px;
+}
+
+.dropdown-toggle:active,
+.open .dropdown-toggle {
+ outline: 0;
+}
+
+.caret {
+ display: inline-block;
+ width: 0;
+ height: 0;
+ vertical-align: top;
+ border-top: 4px solid #000000;
+ border-right: 4px solid transparent;
+ border-left: 4px solid transparent;
+ content: "";
+}
+
+.dropdown .caret {
+ margin-top: 8px;
+ margin-left: 2px;
+}
+
+.dropdown-menu {
+ position: absolute;
+ top: 100%;
+ left: 0;
+ z-index: 1000;
+ display: none;
+ float: left;
+ min-width: 160px;
+ padding: 5px 0;
+ margin: 2px 0 0;
+ list-style: none;
+ background-color: #ffffff;
+ border: 1px solid #ccc;
+ border: 1px solid rgba(0, 0, 0, 0.2);
+ *border-right-width: 2px;
+ *border-bottom-width: 2px;
+ -webkit-border-radius: 6px;
+ -moz-border-radius: 6px;
+ border-radius: 6px;
+ -webkit-box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2);
+ -moz-box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2);
+ box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2);
+ -webkit-background-clip: padding-box;
+ -moz-background-clip: padding;
+ background-clip: padding-box;
+}
+
+.dropdown-menu.pull-right {
+ right: 0;
+ left: auto;
+}
+
+.dropdown-menu .divider {
+ *width: 100%;
+ height: 1px;
+ margin: 9px 1px;
+ *margin: -5px 0 5px;
+ overflow: hidden;
+ background-color: #e5e5e5;
+ border-bottom: 1px solid #ffffff;
+}
+
+.dropdown-menu a {
+ display: block;
+ padding: 3px 20px;
+ clear: both;
+ font-weight: normal;
+ line-height: 20px;
+ color: #333333;
+ white-space: nowrap;
+}
+
+.dropdown-menu li > a:hover,
+.dropdown-menu li > a:focus,
+.dropdown-submenu:hover > a {
+ color: #ffffff;
+ text-decoration: none;
+ background-color: #0088cc;
+ background-color: #0081c2;
+ background-image: -moz-linear-gradient(top, #0088cc, #0077b3);
+ background-image: -webkit-gradient(linear, 0 0, 0 100%, from(#0088cc), to(#0077b3));
+ background-image: -webkit-linear-gradient(top, #0088cc, #0077b3);
+ background-image: -o-linear-gradient(top, #0088cc, #0077b3);
+ background-image: linear-gradient(to bottom, #0088cc, #0077b3);
+ background-repeat: repeat-x;
+ filter: progid:dximagetransform.microsoft.gradient(startColorstr='#ff0088cc', endColorstr='#ff0077b3', GradientType=0);
+}
+
+.dropdown-menu .active > a,
+.dropdown-menu .active > a:hover {
+ color: #ffffff;
+ text-decoration: none;
+ background-color: #0098cc;
+ background-color: #0098cc;
+ background-image: linear-gradient(to bottom, #0098cc, #0087b3);
+ background-image: -moz-linear-gradient(top, #0098cc, #0087b3);
+ background-image: -webkit-gradient(linear, 0 0, 0 100%, from(#0098cc), to(#0087b3));
+ background-image: -webkit-linear-gradient(top, #0098cc, #0087b3);
+ background-image: -o-linear-gradient(top, #0098cc, #0087b3);
+ background-repeat: repeat-x;
+ outline: 0;
+ filter: progid:dximagetransform.microsoft.gradient(startColorstr='#ff0098cc', endColorstr='#ff0087b3', GradientType=0);
+}
+
+.dropdown-menu .disabled > a,
+.dropdown-menu .disabled > a:hover {
+ color: #999999;
+}
+
+.dropdown-menu .disabled > a:hover {
+ text-decoration: none;
+ cursor: default;
+ background-color: transparent;
+}
+
+.open {
+ *z-index: 1000;
+}
+
+.open > .dropdown-menu {
+ display: block;
+}
+
+.pull-right > .dropdown-menu {
+ right: 0;
+ left: auto;
+}
+
+.dropup .caret,
+.navbar-fixed-bottom .dropdown .caret {
+ border-top: 0;
+ border-bottom: 4px solid #000000;
+ content: "\2191";
+}
+
+.dropup .dropdown-menu,
+.navbar-fixed-bottom .dropdown .dropdown-menu {
+ top: auto;
+ bottom: 100%;
+ margin-bottom: 1px;
+}
+
+.dropdown-submenu {
+ position: relative;
+}
+
+.dropdown-submenu > .dropdown-menu {
+ top: 0;
+ left: 100%;
+ margin-top: -6px;
+ margin-left: -1px;
+ -webkit-border-radius: 0 6px 6px 6px;
+ -moz-border-radius: 0 6px 6px 6px;
+ border-radius: 0 6px 6px 6px;
+}
+
+.dropdown-submenu:hover .dropdown-menu {
+ display: block;
+}
+
+.dropdown-submenu > a:after {
+ display: block;
+ float: right;
+ width: 0;
+ height: 0;
+ margin-top: 5px;
+ margin-right: -10px;
+ border-color: transparent;
+ border-left-color: #cccccc;
+ border-style: solid;
+ border-width: 5px 0 5px 5px;
+ content: " ";
+}
+
+.dropdown-submenu:hover > a:after {
+ border-left-color: #ffffff;
+}
+
+.dropdown .dropdown-menu .nav-header {
+ padding-right: 20px;
+ padding-left: 20px;
+}
+
+.typeahead {
+ margin-top: 2px;
+ -webkit-border-radius: 4px;
+ -moz-border-radius: 4px;
+ border-radius: 4px;
+}
+
+.well {
+ min-height: 20px;
+ padding: 19px;
+ margin-bottom: 20px;
+ background-color: #f5f5f5;
+ border: 1px solid #e3e3e3;
+ -webkit-border-radius: 4px;
+ -moz-border-radius: 4px;
+ border-radius: 4px;
+ -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05);
+ -moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05);
+ box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05);
+}
+
+.well blockquote {
+ border-color: #ddd;
+ border-color: rgba(0, 0, 0, 0.15);
+}
+
+.well-large {
+ padding: 24px;
+ -webkit-border-radius: 6px;
+ -moz-border-radius: 6px;
+ border-radius: 6px;
+}
+
+.well-small {
+ padding: 9px;
+ -webkit-border-radius: 3px;
+ -moz-border-radius: 3px;
+ border-radius: 3px;
+}
+
+.fade {
+ opacity: 0;
+ -webkit-transition: opacity 0.15s linear;
+ -moz-transition: opacity 0.15s linear;
+ -o-transition: opacity 0.15s linear;
+ transition: opacity 0.15s linear;
+}
+
+.fade.in {
+ opacity: 1;
+}
+
+.collapse {
+ position: relative;
+ height: 0;
+ overflow: hidden;
+ overflow: visible \9;
+ -webkit-transition: height 0.35s ease;
+ -moz-transition: height 0.35s ease;
+ -o-transition: height 0.35s ease;
+ transition: height 0.35s ease;
+}
+
+.collapse.in {
+ height: auto;
+}
+
+.close {
+ float: right;
+ font-size: 20px;
+ font-weight: bold;
+ line-height: 20px;
+ color: #000000;
+ text-shadow: 0 1px 0 #ffffff;
+ opacity: 0.2;
+ filter: alpha(opacity=20);
+}
+
+.close:hover {
+ color: #000000;
+ text-decoration: none;
+ cursor: pointer;
+ opacity: 0.4;
+ filter: alpha(opacity=40);
+}
+
+button.close {
+ padding: 0;
+ cursor: pointer;
+ background: transparent;
+ border: 0;
+ -webkit-appearance: none;
+}
+
+.btn {
+ display: inline-block;
+ *display: inline;
+ padding: 4px 14px;
+ margin-bottom: 0;
+ *margin-left: .3em;
+ font-size: 14px;
+ line-height: 20px;
+ *line-height: 20px;
+ color: #333333;
+ text-align: center;
+ text-shadow: 0 1px 1px rgba(255, 255, 255, 0.75);
+ vertical-align: middle;
+ cursor: pointer;
+ background-color: #f5f5f5;
+ *background-color: #e6e6e6;
+ background-image: -webkit-gradient(linear, 0 0, 0 100%, from(#ffffff), to(#e6e6e6));
+ background-image: -webkit-linear-gradient(top, #ffffff, #e6e6e6);
+ background-image: -o-linear-gradient(top, #ffffff, #e6e6e6);
+ background-image: linear-gradient(to bottom, #ffffff, #e6e6e6);
+ background-image: -moz-linear-gradient(top, #ffffff, #e6e6e6);
+ background-repeat: repeat-x;
+ border: 1px solid #bbbbbb;
+ *border: 0;
+ border-color: rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.25);
+ border-color: #e6e6e6 #e6e6e6 #bfbfbf;
+ border-bottom-color: #a2a2a2;
+ -webkit-border-radius: 4px;
+ -moz-border-radius: 4px;
+ border-radius: 4px;
+ filter: progid:dximagetransform.microsoft.gradient(startColorstr='#ffffffff', endColorstr='#ffe6e6e6', GradientType=0);
+ filter: progid:dximagetransform.microsoft.gradient(enabled=false);
+ *zoom: 1;
+ -webkit-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.2), 0 1px 2px rgba(0, 0, 0, 0.05);
+ -moz-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.2), 0 1px 2px rgba(0, 0, 0, 0.05);
+ box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.2), 0 1px 2px rgba(0, 0, 0, 0.05);
+}
+
+.btn:hover,
+.btn:active,
+.btn.active,
+.btn.disabled,
+.btn[disabled] {
+ color: #333333;
+ background-color: #e6e6e6;
+ *background-color: #d9d9d9;
+}
+
+.btn:active,
+.btn.active {
+ background-color: #cccccc \9;
+}
+
+.btn:first-child {
+ *margin-left: 0;
+}
+
+.btn:hover {
+ color: #333333;
+ text-decoration: none;
+ background-color: #e6e6e6;
+ *background-color: #d9d9d9;
+ /* Buttons in IE7 don't get borders, so darken on hover */
+
+ background-position: 0 -15px;
+ -webkit-transition: background-position 0.1s linear;
+ -moz-transition: background-position 0.1s linear;
+ -o-transition: background-position 0.1s linear;
+ transition: background-position 0.1s linear;
+}
+
+.btn:focus {
+ outline: thin dotted #333;
+ outline: 5px auto -webkit-focus-ring-color;
+ outline-offset: -2px;
+}
+
+.btn.active,
+.btn:active {
+ background-color: #e6e6e6;
+ background-color: #d9d9d9 \9;
+ background-image: none;
+ outline: 0;
+ -webkit-box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
+ -moz-box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
+ box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
+}
+
+.btn.disabled,
+.btn[disabled] {
+ cursor: default;
+ background-color: #e6e6e6;
+ background-image: none;
+ opacity: 0.65;
+ filter: alpha(opacity=65);
+ -webkit-box-shadow: none;
+ -moz-box-shadow: none;
+ box-shadow: none;
+}
+
+.btn-large {
+ padding: 9px 14px;
+ font-size: 16px;
+ line-height: normal;
+ -webkit-border-radius: 5px;
+ -moz-border-radius: 5px;
+ border-radius: 5px;
+}
+
+.btn-large [class^="icon-"] {
+ margin-top: 2px;
+}
+
+.btn-small {
+ padding: 3px 9px;
+ font-size: 12px;
+ line-height: 18px;
+}
+
+.btn-small [class^="icon-"] {
+ margin-top: 0;
+}
+
+.btn-mini {
+ padding: 2px 6px;
+ font-size: 11px;
+ line-height: 16px;
+}
+
+.btn-block {
+ display: block;
+ width: 100%;
+ padding-right: 0;
+ padding-left: 0;
+ -webkit-box-sizing: border-box;
+ -moz-box-sizing: border-box;
+ box-sizing: border-box;
+}
+
+.btn-block + .btn-block {
+ margin-top: 5px;
+}
+
+.btn-primary.active,
+.btn-warning.active,
+.btn-danger.active,
+.btn-success.active,
+.btn-info.active,
+.btn-inverse.active {
+ color: rgba(255, 255, 255, 0.75);
+}
+
+.btn {
+ border-color: #c5c5c5;
+ border-color: rgba(0, 0, 0, 0.15) rgba(0, 0, 0, 0.15) rgba(0, 0, 0, 0.25);
+}
+
+.btn-primary {
+ color: #ffffff;
+ text-shadow: 0 -1px 0 rgba(0, 0, 0, 0.25);
+ background-color: #006dcc;
+ *background-color: #0044cc;
+ background-image: -webkit-gradient(linear, 0 0, 0 100%, from(#0088cc), to(#0044cc));
+ background-image: -webkit-linear-gradient(top, #0088cc, #0044cc);
+ background-image: -o-linear-gradient(top, #0088cc, #0044cc);
+ background-image: linear-gradient(to bottom, #0088cc, #0044cc);
+ background-image: -moz-linear-gradient(top, #0088cc, #0044cc);
+ background-repeat: repeat-x;
+ border-color: #0044cc #0044cc #002a80;
+ border-color: rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.25);
+ filter: progid:dximagetransform.microsoft.gradient(startColorstr='#ff0088cc', endColorstr='#ff0044cc', GradientType=0);
+ filter: progid:dximagetransform.microsoft.gradient(enabled=false);
+}
+
+.btn-primary:hover,
+.btn-primary:active,
+.btn-primary.active,
+.btn-primary.disabled,
+.btn-primary[disabled] {
+ color: #ffffff;
+ background-color: #0044cc;
+ *background-color: #003bb3;
+}
+
+.btn-primary:active,
+.btn-primary.active {
+ background-color: #003399 \9;
+}
+
+.btn-warning {
+ color: #ffffff;
+ text-shadow: 0 -1px 0 rgba(0, 0, 0, 0.25);
+ background-color: #faa732;
+ *background-color: #f89406;
+ background-image: -webkit-gradient(linear, 0 0, 0 100%, from(#fbb450), to(#f89406));
+ background-image: -webkit-linear-gradient(top, #fbb450, #f89406);
+ background-image: -o-linear-gradient(top, #fbb450, #f89406);
+ background-image: linear-gradient(to bottom, #fbb450, #f89406);
+ background-image: -moz-linear-gradient(top, #fbb450, #f89406);
+ background-repeat: repeat-x;
+ border-color: #f89406 #f89406 #ad6704;
+ border-color: rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.25);
+ filter: progid:dximagetransform.microsoft.gradient(startColorstr='#fffbb450', endColorstr='#fff89406', GradientType=0);
+ filter: progid:dximagetransform.microsoft.gradient(enabled=false);
+}
+
+.btn-warning:hover,
+.btn-warning:active,
+.btn-warning.active,
+.btn-warning.disabled,
+.btn-warning[disabled] {
+ color: #ffffff;
+ background-color: #f89406;
+ *background-color: #df8505;
+}
+
+.btn-warning:active,
+.btn-warning.active {
+ background-color: #c67605 \9;
+}
+
+.btn-danger {
+ color: #ffffff;
+ text-shadow: 0 -1px 0 rgba(0, 0, 0, 0.25);
+ background-color: #da4f49;
+ *background-color: #bd362f;
+ background-image: -webkit-gradient(linear, 0 0, 0 100%, from(#ee5f5b), to(#bd362f));
+ background-image: -webkit-linear-gradient(top, #ee5f5b, #bd362f);
+ background-image: -o-linear-gradient(top, #ee5f5b, #bd362f);
+ background-image: linear-gradient(to bottom, #ee5f5b, #bd362f);
+ background-image: -moz-linear-gradient(top, #ee5f5b, #bd362f);
+ background-repeat: repeat-x;
+ border-color: #bd362f #bd362f #802420;
+ border-color: rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.25);
+ filter: progid:dximagetransform.microsoft.gradient(startColorstr='#ffee5f5b', endColorstr='#ffbd362f', GradientType=0);
+ filter: progid:dximagetransform.microsoft.gradient(enabled=false);
+}
+
+.btn-danger:hover,
+.btn-danger:active,
+.btn-danger.active,
+.btn-danger.disabled,
+.btn-danger[disabled] {
+ color: #ffffff;
+ background-color: #bd362f;
+ *background-color: #a9302a;
+}
+
+.btn-danger:active,
+.btn-danger.active {
+ background-color: #942a25 \9;
+}
+
+.btn-success {
+ color: #ffffff;
+ text-shadow: 0 -1px 0 rgba(0, 0, 0, 0.25);
+ background-color: #5bb75b;
+ *background-color: #51a351;
+ background-image: -webkit-gradient(linear, 0 0, 0 100%, from(#62c462), to(#51a351));
+ background-image: -webkit-linear-gradient(top, #62c462, #51a351);
+ background-image: -o-linear-gradient(top, #62c462, #51a351);
+ background-image: linear-gradient(to bottom, #62c462, #51a351);
+ background-image: -moz-linear-gradient(top, #62c462, #51a351);
+ background-repeat: repeat-x;
+ border-color: #51a351 #51a351 #387038;
+ border-color: rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.25);
+ filter: progid:dximagetransform.microsoft.gradient(startColorstr='#ff62c462', endColorstr='#ff51a351', GradientType=0);
+ filter: progid:dximagetransform.microsoft.gradient(enabled=false);
+}
+
+.btn-success:hover,
+.btn-success:active,
+.btn-success.active,
+.btn-success.disabled,
+.btn-success[disabled] {
+ color: #ffffff;
+ background-color: #51a351;
+ *background-color: #499249;
+}
+
+.btn-success:active,
+.btn-success.active {
+ background-color: #408140 \9;
+}
+
+.btn-info {
+ color: #ffffff;
+ text-shadow: 0 -1px 0 rgba(0, 0, 0, 0.25);
+ background-color: #49afcd;
+ *background-color: #2f96b4;
+ background-image: -webkit-gradient(linear, 0 0, 0 100%, from(#5bc0de), to(#2f96b4));
+ background-image: -webkit-linear-gradient(top, #5bc0de, #2f96b4);
+ background-image: -o-linear-gradient(top, #5bc0de, #2f96b4);
+ background-image: linear-gradient(to bottom, #5bc0de, #2f96b4);
+ background-image: -moz-linear-gradient(top, #5bc0de, #2f96b4);
+ background-repeat: repeat-x;
+ border-color: #2f96b4 #2f96b4 #1f6377;
+ border-color: rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.25);
+ filter: progid:dximagetransform.microsoft.gradient(startColorstr='#ff5bc0de', endColorstr='#ff2f96b4', GradientType=0);
+ filter: progid:dximagetransform.microsoft.gradient(enabled=false);
+}
+
+.btn-info:hover,
+.btn-info:active,
+.btn-info.active,
+.btn-info.disabled,
+.btn-info[disabled] {
+ color: #ffffff;
+ background-color: #2f96b4;
+ *background-color: #2a85a0;
+}
+
+.btn-info:active,
+.btn-info.active {
+ background-color: #24748c \9;
+}
+
+.btn-inverse {
+ color: #ffffff;
+ text-shadow: 0 -1px 0 rgba(0, 0, 0, 0.25);
+ background-color: #363636;
+ *background-color: #222222;
+ background-image: -webkit-gradient(linear, 0 0, 0 100%, from(#444444), to(#222222));
+ background-image: -webkit-linear-gradient(top, #444444, #222222);
+ background-image: -o-linear-gradient(top, #444444, #222222);
+ background-image: linear-gradient(to bottom, #444444, #222222);
+ background-image: -moz-linear-gradient(top, #444444, #222222);
+ background-repeat: repeat-x;
+ border-color: #222222 #222222 #000000;
+ border-color: rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.25);
+ filter: progid:dximagetransform.microsoft.gradient(startColorstr='#ff444444', endColorstr='#ff222222', GradientType=0);
+ filter: progid:dximagetransform.microsoft.gradient(enabled=false);
+}
+
+.btn-inverse:hover,
+.btn-inverse:active,
+.btn-inverse.active,
+.btn-inverse.disabled,
+.btn-inverse[disabled] {
+ color: #ffffff;
+ background-color: #222222;
+ *background-color: #151515;
+}
+
+.btn-inverse:active,
+.btn-inverse.active {
+ background-color: #080808 \9;
+}
+
+button.btn,
+input[type="submit"].btn {
+ *padding-top: 3px;
+ *padding-bottom: 3px;
+}
+
+button.btn::-moz-focus-inner,
+input[type="submit"].btn::-moz-focus-inner {
+ padding: 0;
+ border: 0;
+}
+
+button.btn.btn-large,
+input[type="submit"].btn.btn-large {
+ *padding-top: 7px;
+ *padding-bottom: 7px;
+}
+
+button.btn.btn-small,
+input[type="submit"].btn.btn-small {
+ *padding-top: 3px;
+ *padding-bottom: 3px;
+}
+
+button.btn.btn-mini,
+input[type="submit"].btn.btn-mini {
+ *padding-top: 1px;
+ *padding-bottom: 1px;
+}
+
+.btn-link,
+.btn-link:active {
+ background-color: transparent;
+ background-image: none;
+ -webkit-box-shadow: none;
+ -moz-box-shadow: none;
+ box-shadow: none;
+}
+
+.btn-link {
+ color: #0088cc;
+ cursor: pointer;
+ border-color: transparent;
+ -webkit-border-radius: 0;
+ -moz-border-radius: 0;
+ border-radius: 0;
+}
+
+.btn-link:hover {
+ color: #005580;
+ text-decoration: underline;
+ background-color: transparent;
+}
+
+.btn-group {
+ position: relative;
+ *margin-left: .3em;
+ font-size: 0;
+ white-space: nowrap;
+}
+
+.btn-group:first-child {
+ *margin-left: 0;
+}
+
+.btn-group + .btn-group {
+ margin-left: 5px;
+}
+
+.btn-toolbar {
+ margin-top: 10px;
+ margin-bottom: 10px;
+ font-size: 0;
+}
+
+.btn-toolbar .btn-group {
+ display: inline-block;
+ *display: inline;
+ /* IE7 inline-block hack */
+
+ *zoom: 1;
+}
+
+.btn-toolbar .btn + .btn,
+.btn-toolbar .btn-group + .btn,
+.btn-toolbar .btn + .btn-group {
+ margin-left: 5px;
+}
+
+.btn-group > .btn {
+ position: relative;
+ -webkit-border-radius: 0;
+ -moz-border-radius: 0;
+ border-radius: 0;
+}
+
+.btn-group > .btn + .btn {
+ margin-left: -1px;
+}
+
+.btn-group > .btn,
+.btn-group > .dropdown-menu {
+ font-size: 14px;
+}
+
+.btn-group > .btn-mini {
+ font-size: 11px;
+}
+
+.btn-group > .btn-small {
+ font-size: 12px;
+}
+
+.btn-group > .btn-large {
+ font-size: 16px;
+}
+
+.btn-group > .btn:first-child {
+ margin-left: 0;
+ -webkit-border-bottom-left-radius: 4px;
+ border-bottom-left-radius: 4px;
+ -webkit-border-top-left-radius: 4px;
+ border-top-left-radius: 4px;
+ -moz-border-radius-bottomleft: 4px;
+ -moz-border-radius-topleft: 4px;
+}
+
+.btn-group > .btn:last-child,
+.btn-group > .dropdown-toggle {
+ -webkit-border-top-right-radius: 4px;
+ border-top-right-radius: 4px;
+ -webkit-border-bottom-right-radius: 4px;
+ border-bottom-right-radius: 4px;
+ -moz-border-radius-topright: 4px;
+ -moz-border-radius-bottomright: 4px;
+}
+
+.btn-group > .btn.large:first-child {
+ margin-left: 0;
+ -webkit-border-bottom-left-radius: 6px;
+ border-bottom-left-radius: 6px;
+ -webkit-border-top-left-radius: 6px;
+ border-top-left-radius: 6px;
+ -moz-border-radius-bottomleft: 6px;
+ -moz-border-radius-topleft: 6px;
+}
+
+.btn-group > .btn.large:last-child,
+.btn-group > .large.dropdown-toggle {
+ -webkit-border-top-right-radius: 6px;
+ border-top-right-radius: 6px;
+ -webkit-border-bottom-right-radius: 6px;
+ border-bottom-right-radius: 6px;
+ -moz-border-radius-topright: 6px;
+ -moz-border-radius-bottomright: 6px;
+}
+
+.btn-group > .btn:hover,
+.btn-group > .btn:focus,
+.btn-group > .btn:active,
+.btn-group > .btn.active {
+ z-index: 2;
+}
+
+.btn-group .dropdown-toggle:active,
+.btn-group.open .dropdown-toggle {
+ outline: 0;
+}
+
+.btn-group > .btn + .dropdown-toggle {
+ *padding-top: 5px;
+ padding-right: 8px;
+ *padding-bottom: 5px;
+ padding-left: 8px;
+ -webkit-box-shadow: inset 1px 0 0 rgba(255, 255, 255, 0.125), inset 0 1px 0 rgba(255, 255, 255, 0.2), 0 1px 2px rgba(0, 0, 0, 0.05);
+ -moz-box-shadow: inset 1px 0 0 rgba(255, 255, 255, 0.125), inset 0 1px 0 rgba(255, 255, 255, 0.2), 0 1px 2px rgba(0, 0, 0, 0.05);
+ box-shadow: inset 1px 0 0 rgba(255, 255, 255, 0.125), inset 0 1px 0 rgba(255, 255, 255, 0.2), 0 1px 2px rgba(0, 0, 0, 0.05);
+}
+
+.btn-group > .btn-mini + .dropdown-toggle {
+ *padding-top: 2px;
+ padding-right: 5px;
+ *padding-bottom: 2px;
+ padding-left: 5px;
+}
+
+.btn-group > .btn-small + .dropdown-toggle {
+ *padding-top: 5px;
+ *padding-bottom: 4px;
+}
+
+.btn-group > .btn-large + .dropdown-toggle {
+ *padding-top: 7px;
+ padding-right: 12px;
+ *padding-bottom: 7px;
+ padding-left: 12px;
+}
+
+.btn-group.open .dropdown-toggle {
+ background-image: none;
+ -webkit-box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
+ -moz-box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
+ box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
+}
+
+.btn-group.open .btn.dropdown-toggle {
+ background-color: #e6e6e6;
+}
+
+.btn-group.open .btn-primary.dropdown-toggle {
+ background-color: #0044cc;
+}
+
+.btn-group.open .btn-warning.dropdown-toggle {
+ background-color: #f89406;
+}
+
+.btn-group.open .btn-danger.dropdown-toggle {
+ background-color: #bd362f;
+}
+
+.btn-group.open .btn-success.dropdown-toggle {
+ background-color: #51a351;
+}
+
+.btn-group.open .btn-info.dropdown-toggle {
+ background-color: #2f96b4;
+}
+
+.btn-group.open .btn-inverse.dropdown-toggle {
+ background-color: #222222;
+}
+
+.btn .caret {
+ margin-top: 8px;
+ margin-left: 0;
+}
+
+.btn-mini .caret,
+.btn-small .caret,
+.btn-large .caret {
+ margin-top: 6px;
+}
+
+.btn-large .caret {
+ border-top-width: 5px;
+ border-right-width: 5px;
+ border-left-width: 5px;
+}
+
+.dropup .btn-large .caret {
+ border-top: 0;
+ border-bottom: 5px solid #000000;
+}
+
+.btn-primary .caret,
+.btn-warning .caret,
+.btn-danger .caret,
+.btn-info .caret,
+.btn-success .caret,
+.btn-inverse .caret {
+ border-top-color: #ffffff;
+ border-bottom-color: #ffffff;
+}
+
+.btn-group-vertical {
+ display: inline-block;
+ *display: inline;
+ /* IE7 inline-block hack */
+
+ *zoom: 1;
+}
+
+.btn-group-vertical .btn {
+ display: block;
+ float: none;
+ width: 100%;
+ -webkit-border-radius: 0;
+ -moz-border-radius: 0;
+ border-radius: 0;
+}
+
+.btn-group-vertical .btn + .btn {
+ margin-top: -1px;
+ margin-left: 0;
+}
+
+.btn-group-vertical .btn:first-child {
+ -webkit-border-radius: 4px 4px 0 0;
+ -moz-border-radius: 4px 4px 0 0;
+ border-radius: 4px 4px 0 0;
+}
+
+.btn-group-vertical .btn:last-child {
+ -webkit-border-radius: 0 0 4px 4px;
+ -moz-border-radius: 0 0 4px 4px;
+ border-radius: 0 0 4px 4px;
+}
+
+.btn-group-vertical .btn-large:first-child {
+ -webkit-border-radius: 6px 6px 0 0;
+ -moz-border-radius: 6px 6px 0 0;
+ border-radius: 6px 6px 0 0;
+}
+
+.btn-group-vertical .btn-large:last-child {
+ -webkit-border-radius: 0 0 6px 6px;
+ -moz-border-radius: 0 0 6px 6px;
+ border-radius: 0 0 6px 6px;
+}
+
+.alert {
+ padding: 8px 35px 8px 14px;
+ margin-bottom: 20px;
+ color: #c09853;
+ text-shadow: 0 1px 0 rgba(255, 255, 255, 0.5);
+ background-color: #fcf8e3;
+ border: 1px solid #fbeed5;
+ -webkit-border-radius: 4px;
+ -moz-border-radius: 4px;
+ border-radius: 4px;
+}
+
+.alert h4 {
+ margin: 0;
+}
+
+.alert .close {
+ position: relative;
+ top: -2px;
+ right: -21px;
+ line-height: 20px;
+}
+
+.alert-success {
+ color: #468847;
+ background-color: #dff0d8;
+ border-color: #d6e9c6;
+}
+
+.alert-danger,
+.alert-error {
+ color: #b94a48;
+ background-color: #f2dede;
+ border-color: #eed3d7;
+}
+
+.alert-info {
+ color: #3a87ad;
+ background-color: #d9edf7;
+ border-color: #bce8f1;
+}
+
+.alert-block {
+ padding-top: 14px;
+ padding-bottom: 14px;
+}
+
+.alert-block > p,
+.alert-block > ul {
+ margin-bottom: 0;
+}
+
+.alert-block p + p {
+ margin-top: 5px;
+}
+
+.nav {
+ margin-bottom: 20px;
+ margin-left: 0;
+ list-style: none;
+}
+
+.nav > li > a {
+ display: block;
+}
+
+.nav > li > a:hover {
+ text-decoration: none;
+ background-color: #eeeeee;
+}
+
+.nav > .pull-right {
+ float: right;
+}
+
+.nav-header {
+ display: block;
+ padding: 3px 15px;
+ font-size: 11px;
+ font-weight: bold;
+ line-height: 20px;
+ color: #999999;
+ text-shadow: 0 1px 0 rgba(255, 255, 255, 0.5);
+ text-transform: uppercase;
+}
+
+.nav li + .nav-header {
+ margin-top: 9px;
+}
+
+.nav-list {
+ padding-right: 15px;
+ padding-left: 15px;
+ margin-bottom: 0;
+}
+
+.nav-list > li > a,
+.nav-list .nav-header {
+ margin-right: -15px;
+ margin-left: -15px;
+ text-shadow: 0 1px 0 rgba(255, 255, 255, 0.5);
+}
+
+.nav-list > li > a {
+ padding: 3px 15px;
+}
+
+.nav-list > .active > a,
+.nav-list > .active > a:hover {
+ color: #ffffff;
+ text-shadow: 0 -1px 0 rgba(0, 0, 0, 0.2);
+ background-color: #0088cc;
+}
+
+.nav-list [class^="icon-"] {
+ margin-right: 2px;
+}
+
+.nav-list .divider {
+ *width: 100%;
+ height: 1px;
+ margin: 9px 1px;
+ *margin: -5px 0 5px;
+ overflow: hidden;
+ background-color: #e5e5e5;
+ border-bottom: 1px solid #ffffff;
+}
+
+.nav-tabs,
+.nav-pills {
+ *zoom: 1;
+}
+
+.nav-tabs:before,
+.nav-pills:before,
+.nav-tabs:after,
+.nav-pills:after {
+ display: table;
+ line-height: 0;
+ content: "";
+}
+
+.nav-tabs:after,
+.nav-pills:after {
+ clear: both;
+}
+
+.nav-tabs > li,
+.nav-pills > li {
+ float: left;
+}
+
+.nav-tabs > li > a,
+.nav-pills > li > a {
+ padding-right: 12px;
+ padding-left: 12px;
+ margin-right: 2px;
+ line-height: 14px;
+}
+
+.nav-tabs {
+ border-bottom: 1px solid #ddd;
+}
+
+.nav-tabs > li {
+ margin-bottom: -1px;
+}
+
+.nav-tabs > li > a {
+ padding-top: 8px;
+ padding-bottom: 8px;
+ line-height: 20px;
+ border: 1px solid transparent;
+ -webkit-border-radius: 4px 4px 0 0;
+ -moz-border-radius: 4px 4px 0 0;
+ border-radius: 4px 4px 0 0;
+}
+
+.nav-tabs > li > a:hover {
+ border-color: #eeeeee #eeeeee #dddddd;
+}
+
+.nav-tabs > .active > a,
+.nav-tabs > .active > a:hover {
+ color: #555555;
+ cursor: default;
+ background-color: #ffffff;
+ border: 1px solid #ddd;
+ border-bottom-color: transparent;
+}
+
+.nav-pills > li > a {
+ padding-top: 8px;
+ padding-bottom: 8px;
+ margin-top: 2px;
+ margin-bottom: 2px;
+ -webkit-border-radius: 5px;
+ -moz-border-radius: 5px;
+ border-radius: 5px;
+}
+
+.nav-pills > .active > a,
+.nav-pills > .active > a:hover {
+ color: #ffffff;
+ background-color: #0088cc;
+}
+
+.nav-stacked > li {
+ float: none;
+}
+
+.nav-stacked > li > a {
+ margin-right: 0;
+}
+
+.nav-tabs.nav-stacked {
+ border-bottom: 0;
+}
+
+.nav-tabs.nav-stacked > li > a {
+ border: 1px solid #ddd;
+ -webkit-border-radius: 0;
+ -moz-border-radius: 0;
+ border-radius: 0;
+}
+
+.nav-tabs.nav-stacked > li:first-child > a {
+ -webkit-border-top-right-radius: 4px;
+ border-top-right-radius: 4px;
+ -webkit-border-top-left-radius: 4px;
+ border-top-left-radius: 4px;
+ -moz-border-radius-topright: 4px;
+ -moz-border-radius-topleft: 4px;
+}
+
+.nav-tabs.nav-stacked > li:last-child > a {
+ -webkit-border-bottom-right-radius: 4px;
+ border-bottom-right-radius: 4px;
+ -webkit-border-bottom-left-radius: 4px;
+ border-bottom-left-radius: 4px;
+ -moz-border-radius-bottomright: 4px;
+ -moz-border-radius-bottomleft: 4px;
+}
+
+.nav-tabs.nav-stacked > li > a:hover {
+ z-index: 2;
+ border-color: #ddd;
+}
+
+.nav-pills.nav-stacked > li > a {
+ margin-bottom: 3px;
+}
+
+.nav-pills.nav-stacked > li:last-child > a {
+ margin-bottom: 1px;
+}
+
+.nav-tabs .dropdown-menu {
+ -webkit-border-radius: 0 0 6px 6px;
+ -moz-border-radius: 0 0 6px 6px;
+ border-radius: 0 0 6px 6px;
+}
+
+.nav-pills .dropdown-menu {
+ -webkit-border-radius: 6px;
+ -moz-border-radius: 6px;
+ border-radius: 6px;
+}
+
+.nav .dropdown-toggle .caret {
+ margin-top: 6px;
+ border-top-color: #0088cc;
+ border-bottom-color: #0088cc;
+}
+
+.nav .dropdown-toggle:hover .caret {
+ border-top-color: #005580;
+ border-bottom-color: #005580;
+}
+
+/* move down carets for tabs */
+
+.nav-tabs .dropdown-toggle .caret {
+ margin-top: 8px;
+}
+
+.nav .active .dropdown-toggle .caret {
+ border-top-color: #fff;
+ border-bottom-color: #fff;
+}
+
+.nav-tabs .active .dropdown-toggle .caret {
+ border-top-color: #555555;
+ border-bottom-color: #555555;
+}
+
+.nav > .dropdown.active > a:hover {
+ cursor: pointer;
+}
+
+.nav-tabs .open .dropdown-toggle,
+.nav-pills .open .dropdown-toggle,
+.nav > li.dropdown.open.active > a:hover {
+ color: #ffffff;
+ background-color: #999999;
+ border-color: #999999;
+}
+
+.nav li.dropdown.open .caret,
+.nav li.dropdown.open.active .caret,
+.nav li.dropdown.open a:hover .caret {
+ border-top-color: #ffffff;
+ border-bottom-color: #ffffff;
+ opacity: 1;
+ filter: alpha(opacity=100);
+}
+
+.tabs-stacked .open > a:hover {
+ border-color: #999999;
+}
+
+.tabbable {
+ *zoom: 1;
+}
+
+.tabbable:before,
+.tabbable:after {
+ display: table;
+ line-height: 0;
+ content: "";
+}
+
+.tabbable:after {
+ clear: both;
+}
+
+.tab-content {
+ overflow: auto;
+}
+
+.tabs-below > .nav-tabs,
+.tabs-right > .nav-tabs,
+.tabs-left > .nav-tabs {
+ border-bottom: 0;
+}
+
+.tab-content > .tab-pane,
+.pill-content > .pill-pane {
+ display: none;
+}
+
+.tab-content > .active,
+.pill-content > .active {
+ display: block;
+}
+
+.tabs-below > .nav-tabs {
+ border-top: 1px solid #ddd;
+}
+
+.tabs-below > .nav-tabs > li {
+ margin-top: -1px;
+ margin-bottom: 0;
+}
+
+.tabs-below > .nav-tabs > li > a {
+ -webkit-border-radius: 0 0 4px 4px;
+ -moz-border-radius: 0 0 4px 4px;
+ border-radius: 0 0 4px 4px;
+}
+
+.tabs-below > .nav-tabs > li > a:hover {
+ border-top-color: #ddd;
+ border-bottom-color: transparent;
+}
+
+.tabs-below > .nav-tabs > .active > a,
+.tabs-below > .nav-tabs > .active > a:hover {
+ border-color: transparent #ddd #ddd #ddd;
+}
+
+.tabs-left > .nav-tabs > li,
+.tabs-right > .nav-tabs > li {
+ float: none;
+}
+
+.tabs-left > .nav-tabs > li > a,
+.tabs-right > .nav-tabs > li > a {
+ min-width: 74px;
+ margin-right: 0;
+ margin-bottom: 3px;
+}
+
+.tabs-left > .nav-tabs {
+ float: left;
+ margin-right: 19px;
+ border-right: 1px solid #ddd;
+}
+
+.tabs-left > .nav-tabs > li > a {
+ margin-right: -1px;
+ -webkit-border-radius: 4px 0 0 4px;
+ -moz-border-radius: 4px 0 0 4px;
+ border-radius: 4px 0 0 4px;
+}
+
+.tabs-left > .nav-tabs > li > a:hover {
+ border-color: #eeeeee #dddddd #eeeeee #eeeeee;
+}
+
+.tabs-left > .nav-tabs .active > a,
+.tabs-left > .nav-tabs .active > a:hover {
+ border-color: #ddd transparent #ddd #ddd;
+ *border-right-color: #ffffff;
+}
+
+.tabs-right > .nav-tabs {
+ float: right;
+ margin-left: 19px;
+ border-left: 1px solid #ddd;
+}
+
+.tabs-right > .nav-tabs > li > a {
+ margin-left: -1px;
+ -webkit-border-radius: 0 4px 4px 0;
+ -moz-border-radius: 0 4px 4px 0;
+ border-radius: 0 4px 4px 0;
+}
+
+.tabs-right > .nav-tabs > li > a:hover {
+ border-color: #eeeeee #eeeeee #eeeeee #dddddd;
+}
+
+.tabs-right > .nav-tabs .active > a,
+.tabs-right > .nav-tabs .active > a:hover {
+ border-color: #ddd #ddd #ddd transparent;
+ *border-left-color: #ffffff;
+}
+
+.nav > .disabled > a {
+ color: #999999;
+}
+
+.nav > .disabled > a:hover {
+ text-decoration: none;
+ cursor: default;
+ background-color: transparent;
+}
+
+.navbar {
+ *position: relative;
+ *z-index: 2;
+ margin-bottom: 20px;
+ overflow: visible;
+ color: #555555;
+}
+
+.navbar-inner {
+ min-height: 40px;
+ padding-right: 20px;
+ padding-left: 20px;
+ background-color: #fadafa;
+ background-image: -moz-linear-gradient(top, #ffddff, #f2d2f2);
+ background-image: -webkit-gradient(linear, 0 0, 0 100%, from(#ffddff), to(#f2d2f2));
+ background-image: -webkit-linear-gradient(top, #ffddff, #f2d2f2);
+ background-image: -o-linear-gradient(top, #ffddff, #f2d2f2);
+ background-image: linear-gradient(to bottom, #ffddff, #f2d2f2);
+ background-repeat: repeat-x;
+ border: 1px solid #d4d4d4;
+ -webkit-border-radius: 4px;
+ -moz-border-radius: 4px;
+ border-radius: 4px;
+ filter: progid:dximagetransform.microsoft.gradient(startColorstr='#ffffddff', endColorstr='#fff2d2f2', GradientType=0);
+ -webkit-box-shadow: 0 1px 4px rgba(0, 0, 0, 0.065);
+ -moz-box-shadow: 0 1px 4px rgba(0, 0, 0, 0.065);
+ box-shadow: 0 1px 4px rgba(0, 0, 0, 0.065);
+}
+
+.navbar .container {
+ width: auto;
+}
+
+.nav-collapse.collapse {
+ height: auto;
+}
+
+.navbar .brand {
+ display: block;
+ float: left;
+ padding: 10px 20px 10px;
+ margin-left: -20px;
+ font-size: 20px;
+ font-weight: 200;
+ color: #555555;
+ text-shadow: 0 1px 0 #ffffff;
+}
+
+.navbar .brand:hover {
+ text-decoration: none;
+}
+
+.navbar-text {
+ margin-bottom: 0;
+ line-height: 40px;
+}
+
+.navbar-link {
+ color: #555555;
+}
+
+.navbar-link:hover {
+ color: #333333;
+}
+
+.navbar .divider-vertical {
+ height: 40px;
+ margin: 0 9px;
+ border-right: 1px solid #ffffff;
+ border-left: 1px solid #f2f2f2;
+}
+
+.navbar .btn,
+.navbar .btn-group {
+ margin-top: 6px;
+}
+
+.navbar .btn-group .btn {
+ margin: 0;
+}
+
+.navbar-form {
+ margin-bottom: 0;
+ *zoom: 1;
+}
+
+.navbar-form:before,
+.navbar-form:after {
+ display: table;
+ line-height: 0;
+ content: "";
+}
+
+.navbar-form:after {
+ clear: both;
+}
+
+.navbar-form input,
+.navbar-form select,
+.navbar-form .radio,
+.navbar-form .checkbox {
+ margin-top: 5px;
+}
+
+.navbar-form input,
+.navbar-form select,
+.navbar-form .btn {
+ display: inline-block;
+ margin-bottom: 0;
+}
+
+.navbar-form input[type="image"],
+.navbar-form input[type="checkbox"],
+.navbar-form input[type="radio"] {
+ margin-top: 3px;
+}
+
+.navbar-form .input-append,
+.navbar-form .input-prepend {
+ margin-top: 6px;
+ white-space: nowrap;
+}
+
+.navbar-form .input-append input,
+.navbar-form .input-prepend input {
+ margin-top: 0;
+}
+
+.navbar-search {
+ position: relative;
+ float: left;
+ margin-top: 5px;
+ margin-bottom: 0;
+}
+
+.navbar-search .search-query {
+ padding: 4px 14px;
+ margin-bottom: 0;
+ font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
+ font-size: 13px;
+ font-weight: normal;
+ line-height: 1;
+ -webkit-border-radius: 15px;
+ -moz-border-radius: 15px;
+ border-radius: 15px;
+}
+
+.navbar-static-top {
+ position: static;
+ width: 100%;
+ margin-bottom: 0;
+}
+
+.navbar-static-top .navbar-inner {
+ -webkit-border-radius: 0;
+ -moz-border-radius: 0;
+ border-radius: 0;
+}
+
+.navbar-fixed-top,
+.navbar-fixed-bottom {
+ position: fixed;
+ right: 0;
+ left: 0;
+ z-index: 1030;
+ margin-bottom: 0;
+}
+
+.navbar-fixed-top .navbar-inner,
+.navbar-fixed-bottom .navbar-inner,
+.navbar-static-top .navbar-inner {
+ border: 0;
+}
+
+.navbar-fixed-top .navbar-inner,
+.navbar-fixed-bottom .navbar-inner {
+ padding-right: 0;
+ padding-left: 0;
+ -webkit-border-radius: 0;
+ -moz-border-radius: 0;
+ border-radius: 0;
+}
+
+.navbar-static-top .container,
+.navbar-fixed-top .container,
+.navbar-fixed-bottom .container {
+ width: 940px;
+}
+
+.navbar-fixed-top {
+ top: 0;
+}
+
+.navbar-fixed-top .navbar-inner,
+.navbar-static-top .navbar-inner {
+ -webkit-box-shadow: inset 0 -1px 0 rgba(0, 0, 0, 0.1), 0 1px 10px rgba(0, 0, 0, 0.1);
+ -moz-box-shadow: inset 0 -1px 0 rgba(0, 0, 0, 0.1), 0 1px 10px rgba(0, 0, 0, 0.1);
+ box-shadow: inset 0 -1px 0 rgba(0, 0, 0, 0.1), 0 1px 10px rgba(0, 0, 0, 0.1);
+}
+
+.navbar-fixed-bottom {
+ bottom: 0;
+}
+
+.navbar-fixed-bottom .navbar-inner {
+ -webkit-box-shadow: inset 0 1px 0 rgba(0, 0, 0, 0.1), 0 -1px 10px rgba(0, 0, 0, 0.1);
+ -moz-box-shadow: inset 0 1px 0 rgba(0, 0, 0, 0.1), 0 -1px 10px rgba(0, 0, 0, 0.1);
+ box-shadow: inset 0 1px 0 rgba(0, 0, 0, 0.1), 0 -1px 10px rgba(0, 0, 0, 0.1);
+}
+
+.navbar .nav {
+ position: relative;
+ left: 0;
+ display: block;
+ float: left;
+ margin: 0 10px 0 0;
+}
+
+.navbar .nav.pull-right {
+ float: right;
+}
+
+.navbar .nav > li {
+ float: left;
+}
+
+.navbar .nav > li > a {
+ float: none;
+ padding: 10px 15px 10px;
+ color: #555555;
+ text-decoration: none;
+ text-shadow: 0 1px 0 #ffffff;
+}
+
+.navbar .nav .dropdown-toggle .caret {
+ margin-top: 8px;
+}
+
+.navbar .nav > li > a:focus,
+.navbar .nav > li > a:hover {
+ color: #333333;
+ text-decoration: none;
+ background-color: transparent;
+}
+
+.navbar .nav > .active > a,
+.navbar .nav > .active > a:hover,
+.navbar .nav > .active > a:focus {
+ color: #555555;
+ text-decoration: none;
+ background-color: #e5e5e5;
+ -webkit-box-shadow: inset 0 3px 8px rgba(0, 0, 0, 0.125);
+ -moz-box-shadow: inset 0 3px 8px rgba(0, 0, 0, 0.125);
+ box-shadow: inset 0 3px 8px rgba(0, 0, 0, 0.125);
+}
+
+.navbar .btn-navbar {
+ display: none;
+ float: right;
+ padding: 7px 10px;
+ margin-right: 5px;
+ margin-left: 5px;
+ color: #ffffff;
+ text-shadow: 0 -1px 0 rgba(0, 0, 0, 0.25);
+ background-color: #ededed;
+ *background-color: #e5e5e5;
+ background-image: -webkit-gradient(linear, 0 0, 0 100%, from(#f2f2f2), to(#e5e5e5));
+ background-image: -webkit-linear-gradient(top, #f2f2f2, #e5e5e5);
+ background-image: -o-linear-gradient(top, #f2f2f2, #e5e5e5);
+ background-image: linear-gradient(to bottom, #f2f2f2, #e5e5e5);
+ background-image: -moz-linear-gradient(top, #f2f2f2, #e5e5e5);
+ background-repeat: repeat-x;
+ border-color: #e5e5e5 #e5e5e5 #bfbfbf;
+ border-color: rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.1) rgba(0, 0, 0, 0.25);
+ filter: progid:dximagetransform.microsoft.gradient(startColorstr='#fff2f2f2', endColorstr='#ffe5e5e5', GradientType=0);
+ filter: progid:dximagetransform.microsoft.gradient(enabled=false);
+ -webkit-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1), 0 1px 0 rgba(255, 255, 255, 0.075);
+ -moz-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1), 0 1px 0 rgba(255, 255, 255, 0.075);
+ box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1), 0 1px 0 rgba(255, 255, 255, 0.075);
+}
+
+.navbar .btn-navbar:hover,
+.navbar .btn-navbar:active,
+.navbar .btn-navbar.active,
+.navbar .btn-navbar.disabled,
+.navbar .btn-navbar[disabled] {
+ color: #ffffff;
+ background-color: #e5e5e5;
+ *background-color: #d9d9d9;
+}
+
+.navbar .btn-navbar:active,
+.navbar .btn-navbar.active {
+ background-color: #cccccc \9;
+}
+
+.navbar .btn-navbar .icon-bar {
+ display: block;
+ width: 18px;
+ height: 2px;
+ background-color: #f5f5f5;
+ -webkit-border-radius: 1px;
+ -moz-border-radius: 1px;
+ border-radius: 1px;
+ -webkit-box-shadow: 0 1px 0 rgba(0, 0, 0, 0.25);
+ -moz-box-shadow: 0 1px 0 rgba(0, 0, 0, 0.25);
+ box-shadow: 0 1px 0 rgba(0, 0, 0, 0.25);
+}
+
+.btn-navbar .icon-bar + .icon-bar {
+ margin-top: 3px;
+}
+
+.navbar .nav > li > .dropdown-menu:before {
+ position: absolute;
+ top: -7px;
+ left: 9px;
+ display: inline-block;
+ border-right: 7px solid transparent;
+ border-bottom: 7px solid #ccc;
+ border-left: 7px solid transparent;
+ border-bottom-color: rgba(0, 0, 0, 0.2);
+ content: '';
+}
+
+.navbar .nav > li > .dropdown-menu:after {
+ position: absolute;
+ top: -6px;
+ left: 10px;
+ display: inline-block;
+ border-right: 6px solid transparent;
+ border-bottom: 6px solid #ffffff;
+ border-left: 6px solid transparent;
+ content: '';
+}
+
+.navbar-fixed-bottom .nav > li > .dropdown-menu:before {
+ top: auto;
+ bottom: -7px;
+ border-top: 7px solid #ccc;
+ border-bottom: 0;
+ border-top-color: rgba(0, 0, 0, 0.2);
+}
+
+.navbar-fixed-bottom .nav > li > .dropdown-menu:after {
+ top: auto;
+ bottom: -6px;
+ border-top: 6px solid #ffffff;
+ border-bottom: 0;
+}
+
+.navbar .nav li.dropdown.open > .dropdown-toggle,
+.navbar .nav li.dropdown.active > .dropdown-toggle,
+.navbar .nav li.dropdown.open.active > .dropdown-toggle {
+ color: #555555;
+ background-color: #e5e5e5;
+}
+
+.navbar .nav li.dropdown > .dropdown-toggle .caret {
+ border-top-color: #555555;
+ border-bottom-color: #555555;
+}
+
+.navbar .nav li.dropdown.open > .dropdown-toggle .caret,
+.navbar .nav li.dropdown.active > .dropdown-toggle .caret,
+.navbar .nav li.dropdown.open.active > .dropdown-toggle .caret {
+ border-top-color: #555555;
+ border-bottom-color: #555555;
+}
+
+.navbar .pull-right > li > .dropdown-menu,
+.navbar .nav > li > .dropdown-menu.pull-right {
+ right: 0;
+ left: auto;
+}
+
+.navbar .pull-right > li > .dropdown-menu:before,
+.navbar .nav > li > .dropdown-menu.pull-right:before {
+ right: 12px;
+ left: auto;
+}
+
+.navbar .pull-right > li > .dropdown-menu:after,
+.navbar .nav > li > .dropdown-menu.pull-right:after {
+ right: 13px;
+ left: auto;
+}
+
+.navbar .pull-right > li > .dropdown-menu .dropdown-menu,
+.navbar .nav > li > .dropdown-menu.pull-right .dropdown-menu {
+ right: 100%;
+ left: auto;
+ margin-right: -1px;
+ margin-left: 0;
+ -webkit-border-radius: 6px 0 6px 6px;
+ -moz-border-radius: 6px 0 6px 6px;
+ border-radius: 6px 0 6px 6px;
+}
+
+.navbar-inverse {
+ color: #999999;
+}
+
+.navbar-inverse .navbar-inner {
+ background-color
<TRUNCATED>
[04/47] incubator-systemml git commit: Add Algorithms and Language
References
Posted by du...@apache.org.
Add Algorithms and Language References
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/a891c0a1
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/a891c0a1
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/a891c0a1
Branch: refs/heads/gh-pages
Commit: a891c0a1b36d62834ef4fd3da4b77c4a93ebacba
Parents:
Author: Luciano Resende <lr...@apache.org>
Authored: Sun Aug 16 23:48:13 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Sun Aug 16 23:48:13 2015 -0700
----------------------------------------------------------------------
Algorithms Reference/.jazzignore | 35 +
Algorithms Reference/BinarySVM.tex | 154 +
Algorithms Reference/DecisionTrees.tex | 241 +
Algorithms Reference/DescriptiveBivarStats.tex | 417 +
Algorithms Reference/DescriptiveStats.tex | 94 +
Algorithms Reference/DescriptiveStratStats.tex | 285 +
Algorithms Reference/DescriptiveUnivarStats.tex | 582 ++
Algorithms Reference/GLM.tex | 409 +
Algorithms Reference/GLMpredict.tex | 453 +
Algorithms Reference/Kmeans.tex | 350 +
Algorithms Reference/LinReg.tex | 306 +
Algorithms Reference/LogReg.tex | 266 +
Algorithms Reference/MultiSVM.tex | 153 +
Algorithms Reference/NaiveBayes.tex | 134 +
Algorithms Reference/PCA.tex | 121 +
.../SystemML_Algorithms_Reference.bib | 138 +
.../SystemML_Algorithms_Reference.pdf | Bin 0 -> 620293 bytes
.../SystemML_Algorithms_Reference.tex | 142 +
Language Reference/PyDML Language Reference.doc | Bin 0 -> 209408 bytes
Language Reference/Python syntax for DML.doc | Bin 0 -> 207360 bytes
Language Reference/README.txt | 87 +
.../SystemML_Language_Reference.html | 9801 ++++++++++++++++++
22 files changed, 14168 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/.jazzignore
----------------------------------------------------------------------
diff --git a/Algorithms Reference/.jazzignore b/Algorithms Reference/.jazzignore
new file mode 100644
index 0000000..d782c85
--- /dev/null
+++ b/Algorithms Reference/.jazzignore
@@ -0,0 +1,35 @@
+### Jazz Ignore 0
+# Ignored files and folders will not be committed, but may be modified during
+# accept or update.
+# - Ignore properties should contain a space separated list of filename patterns.
+# - Each pattern is case sensitive and surrounded by braces ('{' and '}').
+# - "*" matches zero or more characters.
+# - "?" matches a single character.
+# - The pattern list may be split across lines by ending the line with a
+# backslash and starting the next line with a tab.
+# - Patterns in core.ignore prevent matching resources in the same
+# directory from being committed.
+# - Patterns in core.ignore.recursive matching resources in the current
+# directory and all subdirectories from being committed.
+# - The default value of core.ignore.recursive is *.class
+# - The default value for core.ignore is bin
+#
+# To ignore shell scripts and hidden files in this subtree:
+# e.g: core.ignore.recursive = {*.sh} {\.*}
+#
+# To ignore resources named 'bin' in the current directory (but allow
+# them in any sub directorybelow):
+# e.g: core.ignore.recursive = {*.sh} {\.*}
+#
+# NOTE: modifying ignore files will not change the ignore status of
+# Eclipse derived resources.
+
+core.ignore.recursive=
+
+core.ignore= \
+ {SystemML_Algorithms_Reference.aux} \
+ {SystemML_Algorithms_Reference.bbl} \
+ {SystemML_Algorithms_Reference.blg} \
+ {SystemML_Algorithms_Reference.dvi} \
+ {SystemML_Algorithms_Reference.out} \
+ {SystemML_Algorithms_Reference.synctex.gz}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/BinarySVM.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/BinarySVM.tex b/Algorithms Reference/BinarySVM.tex
new file mode 100644
index 0000000..5d59adc
--- /dev/null
+++ b/Algorithms Reference/BinarySVM.tex
@@ -0,0 +1,154 @@
+\subsubsection{Binary-class Support Vector Machines}
+\label{l2svm}
+
+\noindent{\bf Description}
+
+Support Vector Machines are used to model the relationship between a categorical
+dependent variable y and one or more explanatory variables denoted X. This
+implementation learns (and predicts with) a binary class support vector machine
+(y with domain size 2).
+\\
+
+\noindent{\bf Usage}
+
+\begin{tabbing}
+\texttt{-f} \textit{path}/\texttt{l2-svm.dml -nvargs}
+\=\texttt{X=}\textit{path}/\textit{file}
+ \texttt{Y=}\textit{path}/\textit{file}
+ \texttt{icpt=}\textit{int}
+ \texttt{tol=}\textit{double}\\
+\>\texttt{reg=}\textit{double}
+ \texttt{maxiter=}\textit{int}
+ \texttt{model=}\textit{path}/\textit{file}\\
+\>\texttt{Log=}\textit{path}/\textit{file}
+ \texttt{fmt=}\textit{csv}$\vert$\textit{text}
+\end{tabbing}
+
+\begin{tabbing}
+\texttt{-f} \textit{path}/\texttt{l2-svm-predict.dml -nvargs}
+\=\texttt{X=}\textit{path}/\textit{file}
+ \texttt{Y=}\textit{path}/\textit{file}
+ \texttt{icpt=}\textit{int}
+ \texttt{model=}\textit{path}/\textit{file}\\
+\>\texttt{scores=}\textit{path}/\textit{file}
+ \texttt{accuracy=}\textit{path}/\textit{file}\\
+\>\texttt{confusion=}\textit{path}/\textit{file}
+ \texttt{fmt=}\textit{csv}$\vert$\textit{text}
+\end{tabbing}
+
+%%\begin{verbatim}
+%%-f path/l2-svm.dml -nvargs X=path/file Y=path/file icpt=int tol=double
+%% reg=double maxiter=int model=path/file
+%%\end{verbatim}
+
+\noindent{\bf Arguments}
+
+\begin{itemize}
+\item X: Location (on HDFS) to read the matrix of feature vectors;
+each row constitutes one feature vector.
+\item Y: Location to read the one-column matrix of (categorical)
+labels that correspond to feature vectors in X. Binary class labels
+can be expressed in one of two choices: $\pm 1$ or $1/2$. Note that,
+this argument is optional for prediction.
+\item icpt (default: {\tt 0}): If set to 1 then a constant bias column is
+added to X.
+\item tol (default: {\tt 0.001}): Procedure terminates early if the reduction
+in objective function value is less than tolerance times the initial objective
+function value.
+\item reg (default: {\tt 1}): Regularization constant. See details to find
+out where lambda appears in the objective function. If one were interested
+in drawing an analogy with the C parameter in C-SVM, then C = 2/lambda.
+Usually, cross validation is employed to determine the optimum value of
+lambda.
+\item maxiter (default: {\tt 100}): The maximum number of iterations.
+\item model: Location (on HDFS) that contains the learnt weights.
+\item Log: Location (on HDFS) to collect various metrics (e.g., objective
+function value etc.) that depict progress across iterations while training.
+\item fmt (default: {\tt text}): Specifies the output format. Choice of
+comma-separated values (csv) or as a sparse-matrix (text).
+\item scores: Location (on HDFS) to store scores for a held-out test set.
+Note that, this is an optional argument.
+\item accuracy: Location (on HDFS) to store the accuracy computed on a
+held-out test set. Note that, this is an optional argument.
+\item confusion: Location (on HDFS) to store the confusion matrix
+computed using a held-out test set. Note that, this is an optional
+argument.
+\end{itemize}
+
+\noindent{\bf Details}
+
+Support vector machines learn a classification function by solving the
+following optimization problem ($L_2$-SVM):
+\begin{eqnarray*}
+&\textrm{argmin}_w& \frac{\lambda}{2} ||w||_2^2 + \sum_i \xi_i^2\\
+&\textrm{subject to:}& y_i w^{\top} x_i \geq 1 - \xi_i ~ \forall i
+\end{eqnarray*}
+where $x_i$ is an example from the training set with its label given by $y_i$,
+$w$ is the vector of parameters and $\lambda$ is the regularization constant
+specified by the user.
+
+To account for the missing bias term, one may augment the data with a column
+of constants which is achieved by setting intercept argument to 1 (C-J Hsieh
+et al, 2008).
+
+This implementation optimizes the primal directly (Chapelle, 2007). It uses
+nonlinear conjugate gradient descent to minimize the objective function
+coupled with choosing step-sizes by performing one-dimensional Newton
+minimization in the direction of the gradient.
+\\
+
+\noindent{\bf Returns}
+
+The learnt weights produced by l2-svm.dml are populated into a single column matrix
+and written to file on HDFS (see model in section Arguments). The number of rows in
+this matrix is ncol(X) if intercept was set to 0 during invocation and ncol(X) + 1
+otherwise. The bias term, if used, is placed in the last row. Depending on what arguments
+are provided during invocation, l2-svm-predict.dml may compute one or more of scores,
+accuracy and confusion matrix in the output format specified.
+\\
+
+%%\noindent{\bf See Also}
+%%
+%%In case of multi-class classification problems (y with domain size greater than 2),
+%%please consider using a multi-class classifier learning algorithm, e.g., multi-class
+%%support vector machines (see Section \ref{msvm}). To model the relationship between
+%%a scalar dependent variable y and one or more explanatory variables X, consider
+%%Linear Regression instead (see Section \ref{linreg-solver} or Section
+%%\ref{linreg-iterative}).
+%%\\
+%%
+\noindent{\bf Examples}
+
+\begin{verbatim}
+hadoop jar SystemML.jar -f l2-svm.dml -nvargs X=/user/biadmin/X.mtx
+ Y=/user/biadmin/y.mtx
+ icpt=0 tol=0.001 fmt=csv
+ reg=1.0 maxiter=100
+ model=/user/biadmin/weights.csv
+ Log=/user/biadmin/Log.csv
+\end{verbatim}
+
+\begin{verbatim}
+hadoop jar SystemML.jar -f l2-svm-predict.dml -nvargs X=/user/biadmin/X.mtx
+ Y=/user/biadmin/y.mtx
+ icpt=0 fmt=csv
+ model=/user/biadmin/weights.csv
+ scores=/user/biadmin/scores.csv
+ accuracy=/user/biadmin/accuracy.csv
+ confusion=/user/biadmin/confusion.csv
+\end{verbatim}
+
+\noindent{\bf References}
+
+\begin{itemize}
+\item W. T. Vetterling and B. P. Flannery. \newblock{\em Conjugate Gradient Methods in Multidimensions in
+Numerical Recipes in C - The Art in Scientific Computing}. \newblock W. H. Press and S. A. Teukolsky
+(eds.), Cambridge University Press, 1992.
+\item J. Nocedal and S. J. Wright. Numerical Optimization, Springer-Verlag, 1999.
+\item C-J Hsieh, K-W Chang, C-J Lin, S. S. Keerthi and S. Sundararajan. \newblock{\em A Dual Coordinate
+Descent Method for Large-scale Linear SVM.} \newblock International Conference of Machine Learning
+(ICML), 2008.
+\item Olivier Chapelle. \newblock{\em Training a Support Vector Machine in the Primal}. \newblock Neural
+Computation, 2007.
+\end{itemize}
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/DecisionTrees.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/DecisionTrees.tex b/Algorithms Reference/DecisionTrees.tex
new file mode 100644
index 0000000..f404dfc
--- /dev/null
+++ b/Algorithms Reference/DecisionTrees.tex
@@ -0,0 +1,241 @@
+\subsection{Decision Trees}
+\label{decision_trees}
+
+\noindent{\bf Description}
+Decision trees (for classification) is a classifier that is considered
+more interpretable than other statistical classifiers. This implementation
+is well-suited to handle large-scale data and builds a (binary) decision
+tree in parallel.\\
+
+\noindent{\bf Usage}
+\begin{tabbing}
+\texttt{-f} \textit{path}/\texttt{decision-tree.dml -nvargs}
+\=\texttt{X=}\textit{path}/\textit{file}
+ \texttt{Y=}\textit{path}/\textit{file}
+ \texttt{types=}\textit{path}/\textit{file}\\
+\>\texttt{model=}\textit{path}/\textit{file}
+ \texttt{bins=}\textit{int}
+ \texttt{depth=}\textit{int}\\
+\>\texttt{num\_leaf=}\textit{int}
+ \texttt{num\_samples=}\textit{int}\\
+\>\texttt{Log=}\textit{path}/\textit{file}
+ \texttt{fmt=}\textit{csv}$\vert$\textit{text}
+\end{tabbing}
+
+\begin{tabbing}
+\texttt{-f} \textit{path}/\texttt{decision-tree-predict.dml -nvargs}
+\=\texttt{X=}\textit{path}/\textit{file}
+ \texttt{Y=}\textit{path}/\textit{file}
+ \texttt{model=}\textit{path}/\textit{file}\\
+\>\texttt{fmt=}\textit{csv}$\vert$\textit{text}
+ \texttt{accuracy=}\textit{path}/\textit{file}\\
+\>\texttt{confusion=}\textit{path}/\textit{file}
+ \texttt{predictions=}\textit{path}/\textit{file}
+\end{tabbing}
+
+\noindent{\bf Arguments}
+
+\begin{itemize}
+\item X: Location (on HDFS) to read the matrix of feature vectors;
+each row constitutes one feature vector.
+\item Y: Location (on HDFS) to read the one-column matrix of (categorical)
+labels that correspond to feature vectors in X. Classes are assumed to be
+contiguously labeled beginning from 1. Note that, this argument is optional
+for prediction.
+\item model: Location (on HDFS) that contains the learnt decision tree.
+\item types: Location (on HDFS) that contains each feature's type. 1 denotes
+continuous-valued (scale) and 2 denotes categorical.
+\item bins (default: {\tt 50}): Number of thresholds to choose for each
+continuous-valued feature (deterimined by equi-height binning).
+\item depth (default: {\tt 10}): Maximum depth of the learnt decision tree.
+\item num\_leaf (default: {\tt 1}): Parameter that controls pruning. The tree
+is not expanded if a node receives less than num\_leaf training examples.
+\item num\_samples (default: {\tt 10}): Parameter that decides when to switch
+to in-memory building of subtrees. If a node $v$ receives less than num\_samples
+training examples then this implementation switches to an in-memory subtree
+building procedure to build the subtree under $v$ in its entirety.
+\item Log: Location (on HDFS) that collects various useful metrics that indicate
+training progress.
+\item predictions: Location (on HDFS) to store predictions for a held-out test set.
+Note that, this is an optional argument.
+\item fmt (default: {\tt text}): Specifies the output format. Choice of
+comma-separated values (csv) or as a sparse-matrix (text).
+\item accuracy: Location (on HDFS) to store the testing accuracy from a
+held-out test set during prediction. Note that, this is an optional argument.
+\item confusion: Location (on HDFS) to store the confusion matrix
+computed using a held-out test set. Note that, this is an optional
+argument.
+\end{itemize}
+
+\noindent{\bf Details}
+
+Decision trees (Breiman et al, 1984) are simple models of
+classification that, due to their structure, are easy to
+interpret. Given an example feature vector, each node in the learnt
+tree runs a simple test on it. Based on the result of the test, the
+example is either diverted to the left subtree or to the right
+subtree. Once the example reaches a leaf, then the label stored at the
+leaf is returned as the prediction for the example.
+
+\par
+
+Building a decision tree from a fully labeled training set entails
+choosing appropriate tests for each internal node in the tree and this
+is usually performed in a top-down manner. Choosing a test requires
+first choosing a feature $j$ and depending on the type of $j$, either
+a threshold $\sigma$, in case $j$ is continuous-valued, or a subset of
+values $S \subseteq \text{Dom}(j)$ where $\text{Dom}(j)$ denotes
+domain of $j$, in case it is categorical. For continuous-valued
+features the test is thus of form $x_j < \sigma$ and for categorical
+features it is of form $x_j \in S$, where $x_j$ denotes the $j^{th}$
+feature value of feature vector $x$. One way to determine which test
+to include, is to compare impurities of the subtrees induced by the
+test and this implementation uses {\it Gini impurity}.
+
+\par
+
+The current implementation allows the user to specify the maximum
+depth of the learnt tree using the argument {\it depth}. It also
+allows for some automated pruning via the argument {\it num\_leaf}. If
+a node receives $\leq$ {\it num\_leaf} training examples, then a leaf
+is built in its place. Furthermore, for a continuous-valued feature
+$j$ the number of candidate thresholds $\sigma$ to choose from is of
+the order of the number of examples present in the training set. Since
+for large-scale data this can result in a large number of candidate
+thresholds, the user can limit this number via the arguments {\it
+ bins} which controls the number of candidate thresholds considered
+for each continuous-valued feature. For each continuous-valued
+feature, the implementation computes an equi-height histogram to
+generate one candidate threshold per equi-height bin. To determine the
+best value subset to split on in the case of categorical features,
+this implementation greedily includes values from the feature's domain
+until the sum of the impurities of the subtrees induced stops
+improving.
+
+\par
+
+Learning a decision tree on large-scale data has received some
+attention in the literature. The current implementation includes logic
+for choosing tests for multiple nodes that belong to the same level in
+the decision tree in parallel (breadth-first expansion) and for
+building entire subtrees under multiple nodes in parallel (depth-first
+subtree building). Empirically it has been demonstrated that it is
+advantageous to perform breadth-first expansion for the nodes
+belonging to the top levels of the tree and to perform depth-first
+subtree building for nodes belonging to the lower levels of the tree
+(Panda et al, 2009). The parameter {\it num\_samples} controls when we
+switch to depth-first subtree building. Any node in the decision tree
+that receives $\leq$ {\it num\_samples} training examples, the subtree
+under it is built in its entirety in one shot.
+
+\par
+
+{\it Description of the model}: The learnt decision tree is represented using a matrix that
+contains at least 3 rows. Each column in the matrix contains the parameters relevant
+to a single node in the tree. The $i^{th}$ column denotes the parameters for the $i^{th}$ node
+whose left child is stored in the $2i^{th}$ column and right child is stored in the $2i+1^{th}$
+column. Here is a brief description of what each row in the matrix contains:
+\begin{itemize}
+\item $1^{st}$ row: Contains the feature index of the feature that this node looks at if
+the node is an internal node, otherwise -1.
+\item $2^{nd}$ row: Contains the type of the feature that this node looks at if the node is
+an internal node, otherwise the label this leaf node is supposed to predict.
+1 denotes continuous-valued feature and 2 denotes categorical.
+\item $3^{rd}$: Only applicable for internal nodes. Contains the threshold the example's
+feature value is compared to if the feature chosen for this node is a continuous-valued feature.
+If on the other hand, the feature chosen for this node is categorical then the size of the
+subset of values is stored here.
+\item $4^{th}$ row onwards: Only applicable in the case of internal nodes where the feature
+chosen is a categorical feature. Rows $4, 5 \ldots$ depict the value subset
+chosen for this node.
+\end{itemize}
+As an example, Figure \ref{dtree} shows a decision tree with $5$ nodes and its matrix
+representation.
+
+\begin{figure}
+\begin{minipage}{0.3\linewidth}
+\begin{center}
+\begin{tikzpicture}
+\node (labelleft) [draw,shape=circle,minimum size=16pt] at (0,0) {$2$};
+\node (labelright) [draw,shape=circle,minimum size=16pt] at (1,0) {$1$};
+\node (rootleft) [draw,shape=rectangle,minimum size=16pt] at (0.5,1) {$x_5 \in \{2,3\}$};
+\node (rootlabel) [draw,shape=circle,minimum size=16pt] at (2.5,1) {$1$};
+\node (root) [draw,shape=rectangle,minimum size=16pt] at (1.75,2) {$x_3 < 0.45$};
+
+\draw[-latex] (root) -- (rootleft);
+\draw[-latex] (root) -- (rootlabel);
+\draw[-latex] (rootleft) -- (labelleft);
+\draw[-latex] (rootleft) -- (labelright);
+
+\end{tikzpicture}
+\end{center}
+\begin{center}
+(a)
+\end{center}
+\end{minipage}
+\hfill
+\begin{minipage}{0.65\linewidth}
+\begin{center}
+\begin{tabular}{c|c|c|c|c|c|}
+& Col 1 & Col 2 & Col 3 & Col 4 & Col 5\\
+\hline
+Row 1 & 3 & 5 & -1 & -1 & -1\\
+\hline
+Row 2 & 1 & 2 & 1 & 2 & 1\\
+\hline
+Row 3 & 0.45 & 2 & & & \\
+\hline
+Row 4 & & 2 & & & \\
+\hline
+Row 5 & & 3 & & & \\
+\hline
+\end{tabular}
+\end{center}
+\begin{center}
+(b)
+\end{center}
+\end{minipage}
+\caption{(a) An example tree and its (b) matrix representation. $x$ denotes an example and $x_j$ the $j^{th}$ feature's value in it.}
+\label{dtree}
+\end{figure}
+
+\vspace{16pt}
+
+\noindent{\bf Returns}
+
+The matrix corresponding to the learnt model is written to a file in the format requested. See
+details where the structure of the model matrix is
+described. Depending on what arguments are provided during
+invocation, decision-tree-predict.dml may compute one or more of
+predictions, accuracy and confusion matrix in the requested output format.
+\\
+
+\noindent{\bf Examples}
+\begin{verbatim}
+hadoop jar SystemML.jar -f decision-tree.dml -nvargs
+ X=/user/biadmin/X.mtx
+ Y=/user/biadmin/y.mtx
+ types=/user/biadmin/types.mtx
+ model=/user/biadmin/model.mtx
+ bins=50 depth=10 num_leaf=1
+ num_samples=250 fmt=csv
+ Log=/user/biadmin/accuracy.csv
+\end{verbatim}
+
+\begin{verbatim}
+hadoop jar SystemML.jar -f decision-tree-predict.dml -nvargs
+ X=/user/biadmin/X.mtx
+ Y=/user/biadmin/y.mtx
+ model=/user/biadmin/model.mtx
+ fmt=csv
+ predictions=/user/biadmin/probabilities.csv
+ accuracy=/user/biadmin/accuracy.csv
+ confusion=/user/biadmin/confusion.csv
+\end{verbatim}
+
+\noindent{\bf References}
+
+\begin{itemize}
+\item B. Panda, J. Herbach, S. Basu, and R. Bayardo. \newblock{PLANET: massively parallel learning of tree ensembles with MapReduce}. In Proceedings of the VLDB Endowment, 2009.
+\item L. Breiman, J. Friedman, R. Olshen, and C. Stone. \newblock{Classification and Regression Trees}. Wadsworth and Brooks, 1984.
+\end{itemize}
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/DescriptiveBivarStats.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/DescriptiveBivarStats.tex b/Algorithms Reference/DescriptiveBivarStats.tex
new file mode 100644
index 0000000..2fe872d
--- /dev/null
+++ b/Algorithms Reference/DescriptiveBivarStats.tex
@@ -0,0 +1,417 @@
+\subsection{Bivariate Statistics}
+
+\noindent{\bf Description}
+\smallskip
+
+Bivariate statistics are used to quantitatively describe the association between
+two features, such as test their statistical (in-)dependence or measure
+the accuracy of one data feature predicting the other feature, in a sample.
+The \BivarScriptName{} script computes common bivariate statistics,
+such as \NameStatR{} and \NameStatChi{}, in parallel for many pairs
+of data features. For a given dataset matrix, script \BivarScriptName{} computes
+certain bivariate statistics for the given feature (column) pairs in the
+matrix. The feature types govern the exact set of statistics computed for that pair.
+For example, \NameStatR{} can only be computed on two quantitative (scale)
+features like `Height' and `Temperature'.
+It does not make sense to compute the linear correlation of two categorical attributes
+like `Hair Color'.
+
+
+\smallskip
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%\tolerance=0
+{\tt{}-f }path/\/\BivarScriptName{}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} index1=}path/file
+{\tt{} index2=}path/file
+{\tt{} types1=}path/file
+{\tt{} types2=}path/file
+{\tt{} OUTDIR=}path
+% {\tt{} fmt=}format
+
+}
+
+
+\smallskip
+\noindent{\bf Arguments}
+\begin{Description}
+\item[{\tt X}:]
+Location (on HDFS) to read the data matrix $X$ whose columns are the features
+that we want to compare and correlate with bivariate statistics.
+\item[{\tt index1}:] % (default:\mbox{ }{\tt " "})
+Location (on HDFS) to read the single-row matrix that lists the column indices
+of the \emph{first-argument} features in pairwise statistics.
+Its $i^{\textrm{th}}$ entry (i.e.\ $i^{\textrm{th}}$ column-cell) contains the
+index $k$ of column \texttt{X[,$\,k$]} in the data matrix
+whose bivariate statistics need to be computed.
+% The default value means ``use all $X$-columns from the first to the last.''
+\item[{\tt index2}:] % (default:\mbox{ }{\tt " "})
+Location (on HDFS) to read the single-row matrix that lists the column indices
+of the \emph{second-argument} features in pairwise statistics.
+Its $j^{\textrm{th}}$ entry (i.e.\ $j^{\textrm{th}}$ column-cell) contains the
+index $l$ of column \texttt{X[,$\,l$]} in the data matrix
+whose bivariate statistics need to be computed.
+% The default value means ``use all $X$-columns from the first to the last.''
+\item[{\tt types1}:] % (default:\mbox{ }{\tt " "})
+Location (on HDFS) to read the single-row matrix that lists the \emph{types}
+of the \emph{first-argument} features in pairwise statistics.
+Its $i^{\textrm{th}}$ entry (i.e.\ $i^{\textrm{th}}$ column-cell) contains the type
+of column \texttt{X[,$\,k$]} in the data matrix, where $k$ is the $i^{\textrm{th}}$
+entry in the {\tt index1} matrix. Feature types must be encoded by
+integer numbers: $1 = {}$scale, $2 = {}$nominal, $3 = {}$ordinal.
+% The default value means ``treat all referenced $X$-columns as scale.''
+\item[{\tt types2}:] % (default:\mbox{ }{\tt " "})
+Location (on HDFS) to read the single-row matrix that lists the \emph{types}
+of the \emph{second-argument} features in pairwise statistics.
+Its $j^{\textrm{th}}$ entry (i.e.\ $j^{\textrm{th}}$ column-cell) contains the type
+of column \texttt{X[,$\,l$]} in the data matrix, where $l$ is the $j^{\textrm{th}}$
+entry in the {\tt index2} matrix. Feature types must be encoded by
+integer numbers: $1 = {}$scale, $2 = {}$nominal, $3 = {}$ordinal.
+% The default value means ``treat all referenced $X$-columns as scale.''
+\item[{\tt OUTDIR}:]
+Location path (on HDFS) where the output matrices with computed bivariate
+statistics will be stored. The matrices' file names and format are defined
+in Table~\ref{table:bivars}.
+% \item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+% Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+% see read/write functions in SystemML Language Reference for details.
+\end{Description}
+
+\begin{table}[t]\hfil
+\begin{tabular}{|lll|}
+\hline\rule{0pt}{12pt}%
+Ouput File / Matrix & Row$\,$\# & Name of Statistic \\[2pt]
+\hline\hline\rule{0pt}{12pt}%
+\emph{All Files} & 1 & 1-st feature column \\
+\rule{1em}{0pt}" & 2 & 2-nd feature column \\[2pt]
+\hline\rule{0pt}{12pt}%
+bivar.scale.scale.stats & 3 & \NameStatR \\[2pt]
+\hline\rule{0pt}{12pt}%
+bivar.nominal.nominal.stats & 3 & \NameStatChi \\
+\rule{1em}{0pt}" & 4 & Degrees of freedom \\
+\rule{1em}{0pt}" & 5 & \NameStatPChi \\
+\rule{1em}{0pt}" & 6 & \NameStatV \\[2pt]
+\hline\rule{0pt}{12pt}%
+bivar.nominal.scale.stats & 3 & \NameStatEta \\
+\rule{1em}{0pt}" & 4 & \NameStatF \\[2pt]
+\hline\rule{0pt}{12pt}%
+bivar.ordinal.ordinal.stats & 3 & \NameStatRho \\[2pt]
+\hline
+\end{tabular}\hfil
+\caption{%
+The output matrices of \BivarScriptName{} have one row per one bivariate
+statistic and one column per one pair of input features. This table lists
+the meaning of each matrix and each row.%
+% Signs ``+'' show applicability to scale or/and to categorical features.
+}
+\label{table:bivars}
+\end{table}
+
+
+
+\pagebreak[2]
+
+\noindent{\bf Details}
+\smallskip
+
+Script \BivarScriptName{} takes an input matrix \texttt{X} whose columns represent
+the features and whose rows represent the records of a data sample.
+Given \texttt{X}, the script computes certain relevant bivariate statistics
+for specified pairs of feature columns \texttt{X[,$\,i$]} and \texttt{X[,$\,j$]}.
+Command-line parameters \texttt{index1} and \texttt{index2} specify the files with
+column pairs of interest to the user. Namely, the file given by \texttt{index1}
+contains the vector of the 1st-attribute column indices and the file given
+by \texttt{index2} has the vector of the 2nd-attribute column indices, with
+``1st'' and ``2nd'' referring to their places in bivariate statistics.
+Note that both \texttt{index1} and \texttt{index2} files should contain a 1-row matrix
+of positive integers.
+
+The bivariate statistics to be computed depend on the \emph{types}, or
+\emph{measurement levels}, of the two columns.
+The types for each pair are provided in the files whose locations are specified by
+\texttt{types1} and \texttt{types2} command-line parameters.
+These files are also 1-row matrices, i.e.\ vectors, that list the 1st-attribute and
+the 2nd-attribute column types in the same order as their indices in the
+\texttt{index1} and \texttt{index2} files. The types must be provided as per
+the following convention: $1 = {}$scale, $2 = {}$nominal, $3 = {}$ordinal.
+
+The script orgainizes its results into (potentially) four output matrices, one per
+each type combination. The types of bivariate statistics are defined using the types
+of the columns that were used for their arguments, with ``ordinal'' sometimes
+retrogressing to ``nominal.'' Table~\ref{table:bivars} describes what each column
+in each output matrix contains. In particular, the script includes the following
+statistics:
+\begin{Itemize}
+\item For a pair of scale (quantitative) columns, \NameStatR;
+\item For a pair of nominal columns (with finite-sized, fixed, unordered domains),
+the \NameStatChi{} and its p-value;
+\item For a pair of one scale column and one nominal column, \NameStatF{};
+\item For a pair of ordinal columns (ordered domains depicting ranks), \NameStatRho.
+\end{Itemize}
+Note that, as shown in Table~\ref{table:bivars}, the output matrices contain the
+column indices of the features involved in each statistic.
+Moreover, if the output matrix does not contain
+a value in a certain cell then it should be interpreted as a~$0$
+(sparse matrix representation).
+
+Below we list all bivariate statistics computed by script \BivarScriptName.
+The statistics are collected into several groups by the type of their input
+features. We refer to the two input features as $v_1$ and $v_2$ unless
+specified otherwise; the value pairs are $(v_{1,i}, v_{2,i})$ for $i=1,\ldots,n$,
+where $n$ is the number of rows in \texttt{X}, i.e.\ the sample size.
+
+
+\paragraph{Scale-vs-scale statistics.}
+Sample statistics that describe association between two quantitative (scale) features.
+A scale feature has numerical values, with the natural ordering relation.
+\begin{Description}
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it\NameStatR]:
+A measure of linear dependence between two numerical features:
+\begin{equation*}
+r \,\,=\,\, \frac{\Cov(v_1, v_2)}{\sqrt{\Var v_1 \Var v_2}}
+\,\,=\,\, \frac{\sum_{i=1}^n (v_{1,i} - \bar{v}_1) (v_{2,i} - \bar{v}_2)}%
+{\sqrt{\sum_{i=1}^n (v_{1,i} - \bar{v}_1)^{2\mathstrut} \cdot \sum_{i=1}^n (v_{2,i} - \bar{v}_2)^{2\mathstrut}}}
+\end{equation*}
+Commonly denoted by~$r$, correlation ranges between $-1$ and $+1$, reaching ${\pm}1$ when all value
+pairs $(v_{1,i}, v_{2,i})$ lie on the same line. Correlation near~0 means that a line is not a good
+way to represent the dependence between the two features; however, this does not imply independence.
+The sign indicates direction of the linear association: $r > 0$ ($r < 0$) if one feature tends to
+linearly increase (decrease) when the other feature increases. Nonlinear association, if present,
+may disobey this sign.
+\NameStatR{} is symmetric: $r(v_1, v_2) = r(v_2, v_1)$; it does not change if we transform $v_1$ and $v_2$
+to $a + b v_1$ and $c + d v_2$ where $a, b, c, d$ are constants and $b, d > 0$.
+
+Suppose that we use simple linear regression to represent one feature given the other, say
+represent $v_{2,i} \approx \alpha + \beta v_{1,i}$ by selecting $\alpha$ and $\beta$
+to minimize the least-squares error $\sum_{i=1}^n (v_{2,i} - \alpha - \beta v_{1,i})^2$.
+Then the best error equals
+\begin{equation*}
+\min_{\alpha, \beta} \,\,\sum_{i=1}^n \big(v_{2,i} - \alpha - \beta v_{1,i}\big)^2 \,\,=\,\,
+(1 - r^2) \,\sum_{i=1}^n \big(v_{2,i} - \bar{v}_2\big)^2
+\end{equation*}
+In other words, $1\,{-}\,r^2$ is the ratio of the residual sum of squares to
+the total sum of squares. Hence, $r^2$ is an accuracy measure of the linear regression.
+\end{Description}
+
+
+\paragraph{Nominal-vs-nominal statistics.}
+Sample statistics that describe association between two nominal categorical features.
+Both features' value domains are encoded with positive integers in arbitrary order:
+nominal features do not order their value domains.
+\begin{Description}
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it\NameStatChi]:
+A measure of how much the frequencies of value pairs of two categorical features deviate from
+statistical independence. Under independence, the probability of every value pair must equal
+the product of probabilities of each value in the pair:
+$\Prob[a, b] - \Prob[a]\,\Prob[b] = 0$. But we do not know these (hypothesized) probabilities;
+we only know the sample frequency counts. Let $n_{a,b}$ be the frequency count of pair
+$(a, b)$, let $n_a$ and $n_b$ be the frequency counts of $a$~alone and of $b$~alone. Under
+independence, difference $n_{a,b}{/}n - (n_a{/}n)(n_b{/}n)$ is unlikely to be exactly~0 due
+to sample randomness, yet it is unlikely to be too far from~0. For some pairs $(a,b)$ it may
+deviate from~0 farther than for other pairs. \NameStatChi{}~is an aggregate measure that
+combines squares of these differences across all value pairs:
+\begin{equation*}
+\chi^2 \,\,=\,\, \sum_{a,\,b} \Big(\frac{n_a n_b}{n}\Big)^{-1} \Big(n_{a,b} - \frac{n_a n_b}{n}\Big)^2
+\,=\,\, \sum_{a,\,b} \frac{(O_{a,b} - E_{a,b})^2}{E_{a,b}}
+\end{equation*}
+where $O_{a,b} = n_{a,b}$ are the \emph{observed} frequencies and $E_{a,b} = (n_a n_b){/}n$ are
+the \emph{expected} frequencies for all pairs~$(a,b)$. Under independence (plus other standard
+assumptions) the sample~$\chi^2$ closely follows a well-known distribution, making it a basis for
+statistical tests for independence, see~\emph{\NameStatPChi} for details. Note that \NameStatChi{}
+does \emph{not} measure the strength of dependence: even very weak dependence may result in a
+significant deviation from independence if the counts are large enough. Use~\NameStatV{} instead
+to measure the strength of dependence.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Degrees of freedom]:
+An integer parameter required for the interpretation of~\NameStatChi{} measure. Under independence
+(plus other standard assumptions) the sample~$\chi^2$ statistic is approximately distributed as the
+sum of $d$~squares of independent normal random variables with mean~0 and variance~1, where $d$ is
+this integer parameter. For a pair of categorical features such that the $1^{\textrm{st}}$~feature
+has $k_1$ categories and the $2^{\textrm{nd}}$~feature has $k_2$ categories, the number of degrees
+of freedom is $d = (k_1 - 1)(k_2 - 1)$.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it\NameStatPChi]:
+A measure of how likely we would observe the current frequencies of value pairs of two categorical
+features assuming their statistical independence. More precisely, it computes the probability that
+the sum of $d$~squares of independent normal random variables with mean~0 and variance~1
+(called the $\chi^2$~distribution with $d$ degrees of freedom) generates a value at least as large
+as the current sample \NameStatChi. The $d$ parameter is \emph{degrees of freedom}, see above.
+Under independence (plus other standard assumptions) the sample \NameStatChi{} closely follows the
+$\chi^2$~distribution and is unlikely to land very far into its tail. On the other hand, if the
+two features are dependent, their sample \NameStatChi{} becomes arbitrarily large as $n\to\infty$
+and lands extremely far into the tail of the $\chi^2$~distribution given a large enough data sample.
+\NameStatPChi{} returns the tail ``weight'' on the right-hand side of \NameStatChi:
+\begin{equation*}
+P\,\,=\,\, \Prob\big[r \geq \textrm{\NameStatChi} \,\,\big|\,\, r \sim \textrm{the $\chi^2$ distribution}\big]
+\end{equation*}
+As any probability, $P$ ranges between 0 and~1. If $P\leq 0.05$, the dependence between the two
+features may be considered statistically significant (i.e.\ their independence is considered
+statistically ruled out). For highly dependent features, it is not unusual to have $P\leq 10^{-20}$
+or less, in which case our script will simply return $P = 0$. Independent features should have
+their $P\geq 0.05$ in about 95\% cases.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it\NameStatV]:
+A measure for the strength of association, i.e.\ of statistical dependence, between two categorical
+features, conceptually similar to \NameStatR. It divides the observed~\NameStatChi{} by the maximum
+possible~$\chi^2_{\textrm{max}}$ given $n$ and the number $k_1, k_2$~of categories in each feature,
+then takes the square root. Thus, \NameStatV{} ranges from 0 to~1,
+where 0 implies no association and 1 implies the maximum possible association (one-to-one
+correspondence) between the two features. See \emph{\NameStatChi} for the computation of~$\chi^2$;
+its maximum${} = {}$%
+$n\cdot\min\{k_1\,{-}\,1, k_2\,{-}\,1\}$ where the $1^{\textrm{st}}$~feature
+has $k_1$ categories and the $2^{\textrm{nd}}$~feature has $k_2$ categories~\cite{AcockStavig1979:CramersV},
+so
+\begin{equation*}
+\textrm{\NameStatV} \,\,=\,\, \sqrt{\frac{\textrm{\NameStatChi}}{n\cdot\min\{k_1\,{-}\,1, k_2\,{-}\,1\}}}
+\end{equation*}
+As opposed to \NameStatPChi, which goes to~0 (rapidly) as the features' dependence increases,
+\NameStatV{} goes towards~1 (slowly) as the dependence increases. Both \NameStatChi{} and
+\NameStatPChi{} are very sensitive to~$n$, but in \NameStatV{} this is mitigated by taking the
+ratio.
+\end{Description}
+
+
+\paragraph{Nominal-vs-scale statistics.}
+Sample statistics that describe association between a categorical feature
+(order ignored) and a quantitative (scale) feature.
+The values of the categorical feature must be coded as positive integers.
+\begin{Description}
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it\NameStatEta]:
+A measure for the strength of association (statistical dependence) between a nominal feature
+and a scale feature, conceptually similar to \NameStatR. Ranges from 0 to~1, approaching 0
+when there is no association and approaching 1 when there is a strong association.
+The nominal feature, treated as the independent variable, is assumed to have relatively few
+possible values, all with large frequency counts. The scale feature is treated as the dependent
+variable. Denoting the nominal feature by~$x$ and the scale feature by~$y$, we have:
+\begin{equation*}
+\eta^2 \,=\, 1 - \frac{\sum_{i=1}^{n} \big(y_i - \hat{y}[x_i]\big)^2}{\sum_{i=1}^{n} (y_i - \bar{y})^2},
+\,\,\,\,\textrm{where}\,\,\,\,
+\hat{y}[x] = \frac{1}{\mathop{\mathrm{freq}}(x)}\sum_{i=1}^n
+\,\left\{\!\!\begin{array}{rl} y_i & \textrm{if $x_i = x$}\\ 0 & \textrm{otherwise}\end{array}\right.\!\!\!
+\end{equation*}
+and $\bar{y} = (1{/}n)\sum_{i=1}^n y_i$ is the mean. Value $\hat{y}[x]$ is the average
+of~$y_i$ among all records where $x_i = x$; it can also be viewed as the ``predictor''
+of $y$ given~$x$. Then $\sum_{i=1}^{n} (y_i - \hat{y}[x_i])^2$ is the residual error
+sum-of-squares and $\sum_{i=1}^{n} (y_i - \bar{y})^2$ is the total sum-of-squares for~$y$.
+Hence, $\eta^2$ measures the accuracy of predicting $y$ with~$x$, just like the
+``R-squared'' statistic measures the accuracy of linear regression. Our output $\eta$
+is the square root of~$\eta^2$.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it\NameStatF]:
+A measure of how much the values of the scale feature, denoted here by~$y$,
+deviate from statistical independence on the nominal feature, denoted by~$x$.
+The same measure appears in the one-way analysis of vari\-ance (ANOVA).
+Like \NameStatChi, \NameStatF{} is used to test the hypothesis that
+$y$~is independent from~$x$, given the following assumptions:
+\begin{Itemize}
+\item The scale feature $y$ has approximately normal distribution whose mean
+may depend only on~$x$ and variance is the same for all~$x$;
+\item The nominal feature $x$ has relatively small value domain with large
+frequency counts, the $x_i$-values are treated as fixed (non-random);
+\item All records are sampled independently of each other.
+\end{Itemize}
+To compute \NameStatF{}, we first compute $\hat{y}[x]$ as the average of~$y_i$
+among all records where $x_i = x$. These $\hat{y}[x]$ can be viewed as
+``predictors'' of $y$ given~$x$; if $y$ is independent on~$x$, they should
+``predict'' only the global mean~$\bar{y}$. Then we form two sums-of-squares:
+\begin{Itemize}
+\item \emph{Residual} sum-of-squares of the ``predictor'' accuracy: $y_i - \hat{y}[x_i]$;
+\item \emph{Explained} sum-of-squares of the ``predictor'' variability: $\hat{y}[x_i] - \bar{y}$.
+\end{Itemize}
+\NameStatF{} is the ratio of the explained sum-of-squares to
+the residual sum-of-squares, each divided by their corresponding degrees
+of freedom:
+\begin{equation*}
+F \,\,=\,\,
+\frac{\sum_{x}\, \mathop{\mathrm{freq}}(x) \, \big(\hat{y}[x] - \bar{y}\big)^2 \,\big/\,\, (k\,{-}\,1)}%
+{\sum_{i=1}^{n} \big(y_i - \hat{y}[x_i]\big)^2 \,\big/\,\, (n\,{-}\,k)} \,\,=\,\,
+\frac{n\,{-}\,k}{k\,{-}\,1} \cdot \frac{\eta^2}{1 - \eta^2}
+\end{equation*}
+Here $k$ is the domain size of the nominal feature~$x$. The $k$ ``predictors''
+lose 1~freedom due to their linear dependence with~$\bar{y}$; similarly,
+the $n$~$y_i$-s lose $k$~freedoms due to the ``predictors''.
+
+The statistic can test if the independence hypothesis of $y$ from $x$ is reasonable;
+more generally (with relaxed normality assumptions) it can test the hypothesis that
+\emph{the mean} of $y$ among records with a given~$x$ is the same for all~$x$.
+Under this hypothesis \NameStatF{} has, or approximates, the $F(k\,{-}\,1, n\,{-}\,k)$-distribution.
+But if the mean of $y$ given $x$ depends on~$x$, \NameStatF{}
+becomes arbitrarily large as $n\to\infty$ (with $k$~fixed) and lands extremely far
+into the tail of the $F(k\,{-}\,1, n\,{-}\,k)$-distribution given a large enough data sample.
+\end{Description}
+
+
+\paragraph{Ordinal-vs-ordinal statistics.}
+Sample statistics that describe association between two ordinal categorical features.
+Both features' value domains are encoded with positive integers, so that the natural
+order of the integers coincides with the order in each value domain.
+\begin{Description}
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it\NameStatRho]:
+A measure for the strength of association (statistical dependence) between
+two ordinal features, conceptually similar to \NameStatR. Specifically, it is \NameStatR{}
+applied to the feature vectors in which all values are replaced by their ranks, i.e.\
+their positions if the vector is sorted. The ranks of identical (duplicate) values
+are replaced with their average rank. For example, in vector
+$(15, 11, 26, 15, 8)$ the value ``15'' occurs twice with ranks 3 and~4 per the sorted
+order $(8_1, 11_2, 15_3, 15_4, 26_5)$; so, both values are assigned their average
+rank of $3.5 = (3\,{+}\,4)\,{/}\,2$ and the vector is replaced by~$(3.5,\, 2,\, 5,\, 3.5,\, 1)$.
+
+Our implementation of \NameStatRho{} is geared towards features having small value domains
+and large counts for the values. Given the two input vectors, we form a contingency table $T$
+of pairwise frequency counts, as well as a vector of frequency counts for each feature: $f_1$
+and~$f_2$. Here in $T_{i,j}$, $f_{1,i}$, $f_{2,j}$ indices $i$ and~$j$ refer to the
+order-preserving integer encoding of the feature values.
+We use prefix sums over $f_1$ and~$f_2$ to compute the values' average ranks:
+$r_{1,i} = \sum_{j=1}^{i-1} f_{1,j} + (f_{1,i}\,{+}\,1){/}2$, and analogously for~$r_2$.
+Finally, we compute rank variances for $r_1, r_2$ weighted by counts $f_1, f_2$ and their
+covariance weighted by~$T$, before applying the standard formula for \NameStatR:
+\begin{equation*}
+\rho \,\,=\,\, \frac{\Cov_T(r_1, r_2)}{\sqrt{\Var_{f_1}(r_1)\Var_{f_2}(r_2)}}
+\,\,=\,\, \frac{\sum_{i,j} T_{i,j} (r_{1,i} - \bar{r}_1) (r_{2,j} - \bar{r}_2)}%
+{\sqrt{\sum_i f_{1,i} (r_{1,i} - \bar{r}_1)^{2\mathstrut} \cdot \sum_j f_{2,j} (r_{2,j} - \bar{r}_2)^{2\mathstrut}}}
+\end{equation*}
+where $\bar{r}_1 = \sum_i r_{1,i} f_{1,i}{/}n$, analogously for~$\bar{r}_2$.
+The value of $\rho$ lies between $-1$ and $+1$, with sign indicating the prevalent direction
+of the association: $\rho > 0$ ($\rho < 0$) means that one feature tends to increase (decrease)
+when the other feature increases. The correlation becomes~1 when the two features are
+monotonically related.
+\end{Description}
+
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+A collection of (potentially) 4 matrices. Each matrix contains bivariate statistics that
+resulted from a different combination of feature types. There is one matrix for scale-scale
+statistics (which includes \NameStatR), one for nominal-nominal statistics (includes \NameStatChi{}),
+one for nominal-scale statistics (includes \NameStatF) and one for ordinal-ordinal statistics
+(includes \NameStatRho). If any of these matrices is not produced, then no pair of columns required
+the corresponding type combination. See Table~\ref{table:bivars} for the matrix naming and
+format details.
+
+
+\smallskip
+\pagebreak[2]
+
+\noindent{\bf Examples}
+\smallskip
+
+{\hangindent=\parindent\noindent\tt
+\hml -f \BivarScriptName{} -nvargs
+X=/user/biadmin/X.mtx
+index1=/user/biadmin/S1.mtx
+index2=/user/biadmin/S2.mtx
+types1=/user/biadmin/K1.mtx
+types2=/user/biadmin/K2.mtx
+OUTDIR=/user/biadmin/stats.mtx
+
+}
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/DescriptiveStats.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/DescriptiveStats.tex b/Algorithms Reference/DescriptiveStats.tex
new file mode 100644
index 0000000..d471b50
--- /dev/null
+++ b/Algorithms Reference/DescriptiveStats.tex
@@ -0,0 +1,94 @@
+\newcommand{\UnivarScriptName}{\texttt{\tt Univar-Stats.dml}}
+\newcommand{\BivarScriptName}{\texttt{\tt bivar-stats.dml}}
+
+\newcommand{\OutputRowIDMinimum}{1}
+\newcommand{\OutputRowIDMaximum}{2}
+\newcommand{\OutputRowIDRange}{3}
+\newcommand{\OutputRowIDMean}{4}
+\newcommand{\OutputRowIDVariance}{5}
+\newcommand{\OutputRowIDStDeviation}{6}
+\newcommand{\OutputRowIDStErrorMean}{7}
+\newcommand{\OutputRowIDCoeffVar}{8}
+\newcommand{\OutputRowIDQuartiles}{?, 13, ?}
+\newcommand{\OutputRowIDMedian}{13}
+\newcommand{\OutputRowIDIQMean}{14}
+\newcommand{\OutputRowIDSkewness}{9}
+\newcommand{\OutputRowIDKurtosis}{10}
+\newcommand{\OutputRowIDStErrorSkewness}{11}
+\newcommand{\OutputRowIDStErrorCurtosis}{12}
+\newcommand{\OutputRowIDNumCategories}{15}
+\newcommand{\OutputRowIDMode}{16}
+\newcommand{\OutputRowIDNumModes}{17}
+\newcommand{\OutputRowText}[1]{\mbox{(output row~{#1})\hspace{0.5pt}:}}
+
+\newcommand{\NameStatR}{Pearson's correlation coefficient}
+\newcommand{\NameStatChi}{Pearson's~$\chi^2$}
+\newcommand{\NameStatPChi}{$P\textrm{-}$value of Pearson's~$\chi^2$}
+\newcommand{\NameStatV}{Cram\'er's~$V$}
+\newcommand{\NameStatEta}{Eta statistic}
+\newcommand{\NameStatF}{$F$~statistic}
+\newcommand{\NameStatRho}{Spearman's rank correlation coefficient}
+
+Descriptive statistics are used to quantitatively describe the main characteristics of the data.
+They provide meaningful summaries computed over different observations or data records
+collected in a study. These summaries typically form the basis of the initial data exploration
+as part of a more extensive statistical analysis. Such a quantitative analysis assumes that
+every variable (also known as, attribute, feature, or column) in the data has a specific
+\emph{level of measurement}~\cite{Stevens1946:scales}.
+
+The measurement level of a variable, often called as {\bf variable type}, can either be
+\emph{scale} or \emph{categorical}. A \emph{scale} variable represents the data measured on
+an interval scale or ratio scale. Examples of scale variables include `Height', `Weight',
+`Salary', and `Temperature'. Scale variables are also referred to as \emph{quantitative}
+or \emph{continuous} variables. In contrast, a \emph{categorical} variable has a fixed
+limited number of distinct values or categories. Examples of categorical variables
+include `Gender', `Region', `Hair color', `Zipcode', and `Level of Satisfaction'.
+Categorical variables can further be classified into two types, \emph{nominal} and
+\emph{ordinal}, depending on whether the categories in the variable can be ordered via an
+intrinsic ranking. For example, there is no meaningful ranking among distinct values in
+`Hair color' variable, while the categories in `Level of Satisfaction' can be ranked from
+highly dissatisfied to highly satisfied.
+
+The input dataset for descriptive statistics is provided in the form of a matrix, whose
+rows are the records (data points) and whose columns are the features (i.e.~variables).
+Some scripts allow this matrix to be vertically split into two or three matrices. Descriptive
+statistics are computed over the specified features (columns) in the matrix. Which
+statistics are computed depends on the types of the features. It is important to keep
+in mind the following caveats and restrictions:
+\begin{Enumerate}
+\item Given a finite set of data records, i.e.~a \emph{sample}, we take their feature
+values and compute their \emph{sample statistics}. These statistics
+will vary from sample to sample even if the underlying distribution of feature values
+remains the same. Sample statistics are accurate for the given sample only.
+If the goal is to estimate the \emph{distribution statistics} that are parameters of
+the (hypothesized) underlying distribution of the features, the corresponding sample
+statistics may sometimes be used as approximations, but their accuracy will vary.
+\item In particular, the accuracy of the estimated distribution statistics will be low
+if the number of values in the sample is small. That is, for small samples, the computed
+statistics may depend on the randomness of the individual sample values more than on
+the underlying distribution of the features.
+\item The accuracy will also be low if the sample records cannot be assumed mutually
+independent and identically distributed (i.i.d.), that is, sampled at random from the
+same underlying distribution. In practice, feature values in one record often depend
+on other features and other records, including unknown ones.
+\item Most of the computed statistics will have low estimation accuracy in the presence of
+extreme values (outliers) or if the underlying distribution has heavy tails, for example
+obeys a power law. However, a few of the computed statistics, such as the median and
+\NameStatRho{}, are \emph{robust} to outliers.
+\item Some sample statistics are reported with their \emph{sample standard errors}
+in an attempt to quantify their accuracy as distribution parameter estimators. But these
+sample standard errors, in turn, only estimate the underlying distribution's standard
+errors and will have low accuracy for small or \mbox{non-i.i.d.} samples, outliers in samples,
+or heavy-tailed distributions.
+\item We assume that the quantitative (scale) feature columns do not contain missing
+values, infinite values, \texttt{NaN}s, or coded non-numeric values, unless otherwise
+specified. We assume that each categorical feature column contains positive integers
+from 1 to the number of categories; for ordinal features, the natural order on
+the integers should coincide with the order on the categories.
+\end{Enumerate}
+
+\input{DescriptiveUnivarStats}
+
+\input{DescriptiveBivarStats}
+
+\input{DescriptiveStratStats}
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/DescriptiveStratStats.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/DescriptiveStratStats.tex b/Algorithms Reference/DescriptiveStratStats.tex
new file mode 100644
index 0000000..716cf35
--- /dev/null
+++ b/Algorithms Reference/DescriptiveStratStats.tex
@@ -0,0 +1,285 @@
+\subsection{Stratified Bivariate Statistics}
+
+\noindent{\bf Description}
+\smallskip
+
+The {\tt stratstats.dml} script computes common bivariate statistics, such
+as correlation, slope, and their p-value, in parallel for many pairs of input
+variables in the presence of a confounding categorical variable. The values
+of this confounding variable group the records into strata (subpopulations),
+in which all bivariate pairs are assumed free of confounding. The script
+uses the same data model as in one-way analysis of covariance (ANCOVA), with
+strata representing population samples. It also outputs univariate stratified
+and bivariate unstratified statistics.
+
+\begin{table}[t]\hfil
+\begin{tabular}{|l|ll|ll|ll||ll|}
+\hline
+Month of the year & \multicolumn{2}{l|}{October} & \multicolumn{2}{l|}{November} &
+ \multicolumn{2}{l||}{December} & \multicolumn{2}{c|}{Oct$\,$--$\,$Dec} \\
+Customers, millions & 0.6 & 1.4 & 1.4 & 0.6 & 3.0 & 1.0 & 5.0 & 3.0 \\
+\hline
+Promotion (0 or 1) & 0 & 1 & 0 & 1 & 0 & 1 & 0 & 1 \\
+Avg.\ sales per 1000 & 0.4 & 0.5 & 0.9 & 1.0 & 2.5 & 2.6 & 1.8 & 1.3 \\
+\hline
+\end{tabular}\hfil
+\caption{Stratification example: the effect of the promotion on average sales
+becomes reversed and amplified (from $+0.1$ to $-0.5$) if we ignore the months.}
+\label{table:stratexample}
+\end{table}
+
+To see how data stratification mitigates confounding, consider an (artificial)
+example in Table~\ref{table:stratexample}. A highly seasonal retail item
+was marketed with and without a promotion over the final 3~months of the year.
+In each month the sale was more likely with the promotion than without it.
+But during the peak holiday season, when shoppers came in greater numbers and
+bought the item more often, the promotion was less frequently used. As a result,
+if the 4-th quarter data is pooled together, the promotion's effect becomes
+reversed and magnified. Stratifying by month restores the positive correlation.
+
+The script computes its statistics in parallel over all possible pairs from two
+specified sets of covariates. The 1-st covariate is a column in input matrix~$X$
+and the 2-nd covariate is a column in input matrix~$Y$; matrices $X$ and~$Y$ may
+be the same or different. The columns of interest are given by their index numbers
+in special matrices. The stratum column, specified in its own matrix, is the same
+for all covariate pairs.
+
+Both covariates in each pair must be numerical, with the 2-nd covariate normally
+distributed given the 1-st covariate (see~Details). Missing covariate values or
+strata are represented by~``NaN''. Records with NaN's are selectively omitted
+wherever their NaN's are material to the output statistic.
+
+\smallskip
+\pagebreak[3]
+
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+{\tt{}-f }path/\/{\tt{}stratstats.dml}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} Xcid=}path/file
+{\tt{} Y=}path/file
+{\tt{} Ycid=}path/file
+{\tt{} S=}path/file
+{\tt{} Scid=}int
+{\tt{} O=}path/file
+{\tt{} fmt=}format
+
+}
+
+
+\smallskip
+\noindent{\bf Arguments}
+\begin{Description}
+\item[{\tt X}:]
+Location (on HDFS) to read matrix $X$ whose columns we want to use as
+the 1-st covariate (i.e.~as the feature variable)
+\item[{\tt Xcid}:] (default:\mbox{ }{\tt " "})
+Location to read the single-row matrix that lists all index numbers
+of the $X$-columns used as the 1-st covariate; the default value means
+``use all $X$-columns''
+\item[{\tt Y}:] (default:\mbox{ }{\tt " "})
+Location to read matrix $Y$ whose columns we want to use as the 2-nd
+covariate (i.e.~as the response variable); the default value means
+``use $X$ in place of~$Y$''
+\item[{\tt Ycid}:] (default:\mbox{ }{\tt " "})
+Location to read the single-row matrix that lists all index numbers
+of the $Y$-columns used as the 2-nd covariate; the default value means
+``use all $Y$-columns''
+\item[{\tt S}:] (default:\mbox{ }{\tt " "})
+Location to read matrix $S$ that has the stratum column.
+Note: the stratum column must contain small positive integers; all fractional
+values are rounded; stratum IDs of value ${\leq}\,0$ or NaN are treated as
+missing. The default value for {\tt S} means ``use $X$ in place of~$S$''
+\item[{\tt Scid}:] (default:\mbox{ }{\tt 1})
+The index number of the stratum column in~$S$
+\item[{\tt O}:]
+Location to store the output matrix defined in Table~\ref{table:stratoutput}
+\item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+see read/write functions in SystemML Language Reference for details.
+\end{Description}
+
+
+\begin{table}[t]\small\hfil
+\begin{tabular}{|rcl|rcl|}
+\hline
+& Col.\# & Meaning & & Col.\# & Meaning \\
+\hline
+\multirow{9}{*}{\begin{sideways}1-st covariate\end{sideways}}\hspace{-1em}
+& 01 & $X$-column number &
+\multirow{9}{*}{\begin{sideways}2-nd covariate\end{sideways}}\hspace{-1em}
+& 11 & $Y$-column number \\
+& 02 & presence count for $x$ &
+& 12 & presence count for $y$ \\
+& 03 & global mean $(x)$ &
+& 13 & global mean $(y)$ \\
+& 04 & global std.\ dev. $(x)$ &
+& 14 & global std.\ dev. $(y)$ \\
+& 05 & stratified std.\ dev. $(x)$ &
+& 15 & stratified std.\ dev. $(y)$ \\
+& 06 & $R^2$ for $x \sim {}$strata &
+& 16 & $R^2$ for $y \sim {}$strata \\
+& 07 & adjusted $R^2$ for $x \sim {}$strata &
+& 17 & adjusted $R^2$ for $y \sim {}$strata \\
+& 08 & p-value, $x \sim {}$strata &
+& 18 & p-value, $y \sim {}$strata \\
+& 09--10 & reserved &
+& 19--20 & reserved \\
+\hline
+\multirow{9}{*}{\begin{sideways}$y\sim x$, NO strata\end{sideways}}\hspace{-1.15em}
+& 21 & presence count $(x, y)$ &
+\multirow{10}{*}{\begin{sideways}$y\sim x$ AND strata$\!\!\!\!$\end{sideways}}\hspace{-1.15em}
+& 31 & presence count $(x, y, s)$ \\
+& 22 & regression slope &
+& 32 & regression slope \\
+& 23 & regres.\ slope std.\ dev. &
+& 33 & regres.\ slope std.\ dev. \\
+& 24 & correlation${} = \pm\sqrt{R^2}$ &
+& 34 & correlation${} = \pm\sqrt{R^2}$ \\
+& 25 & residual std.\ dev. &
+& 35 & residual std.\ dev. \\
+& 26 & $R^2$ in $y$ due to $x$ &
+& 36 & $R^2$ in $y$ due to $x$ \\
+& 27 & adjusted $R^2$ in $y$ due to $x$ &
+& 37 & adjusted $R^2$ in $y$ due to $x$ \\
+& 28 & p-value for ``slope = 0'' &
+& 38 & p-value for ``slope = 0'' \\
+& 29 & reserved &
+& 39 & \# strata with ${\geq}\,2$ count \\
+& 30 & reserved &
+& 40 & reserved \\
+\hline
+\end{tabular}\hfil
+\caption{The {\tt stratstats.dml} output matrix has one row per each distinct
+pair of 1-st and 2-nd covariates, and 40 columns with the statistics described
+here.}
+\label{table:stratoutput}
+\end{table}
+
+
+
+
+\noindent{\bf Details}
+\smallskip
+
+Suppose we have $n$ records of format $(i, x, y)$, where $i\in\{1,\ldots, k\}$ is
+a stratum number and $(x, y)$ are two numerical covariates. We want to analyze
+conditional linear relationship between $y$ and $x$ conditioned by~$i$.
+Note that $x$, but not~$y$, may represent a categorical variable if we assign a
+numerical value to each category, for example 0 and 1 for two categories.
+
+We assume a linear regression model for~$y$:
+\begin{equation}
+y_{i,j} \,=\, \alpha_i + \beta x_{i,j} + \eps_{i,j}\,, \quad\textrm{where}\,\,\,\,
+\eps_{i,j} \sim \Normal(0, \sigma^2)
+\label{eqn:stratlinmodel}
+\end{equation}
+Here $i = 1\ldots k$ is a stratum number and $j = 1\ldots n_i$ is a record number
+in stratum~$i$; by $n_i$ we denote the number of records available in stratum~$i$.
+The noise term~$\eps_{i,j}$ is assumed to have the same variance in all strata.
+When $n_i\,{>}\,0$, we can estimate the means of $x_{i, j}$ and $y_{i, j}$ in
+stratum~$i$ as
+\begin{equation*}
+\bar{x}_i \,= \Big(\sum\nolimits_{j=1}^{n_i} \,x_{i, j}\Big) / n_i\,;\quad
+\bar{y}_i \,= \Big(\sum\nolimits_{j=1}^{n_i} \,y_{i, j}\Big) / n_i
+\end{equation*}
+If $\beta$ is known, the best estimate for $\alpha_i$ is $\bar{y}_i - \beta \bar{x}_i$,
+which gives the prediction error sum-of-squares of
+\begin{equation}
+\sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(y_{i,j} - \beta x_{i,j} - (\bar{y}_i - \beta \bar{x}_i)\big)^2
+\,\,=\,\, \beta^{2\,}V_x \,-\, 2\beta \,V_{x,y} \,+\, V_y
+\label{eqn:stratsumsq}
+\end{equation}
+where $V_x$, $V_y$, and $V_{x, y}$ are, correspondingly, the ``stratified'' sample
+estimates of variance $\Var(x)$ and $\Var(y)$ and covariance $\Cov(x,y)$ computed as
+\begin{align*}
+V_x \,&=\, \sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(x_{i,j} - \bar{x}_i\big)^2; \quad
+V_y \,=\, \sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(y_{i,j} - \bar{y}_i\big)^2;\\
+V_{x,y} \,&=\, \sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(x_{i,j} - \bar{x}_i\big)\big(y_{i,j} - \bar{y}_i\big)
+\end{align*}
+They are stratified because we compute the sample (co-)variances in each stratum~$i$
+separately, then combine by summation. The stratified estimates for $\Var(X)$ and $\Var(Y)$
+tend to be smaller than the non-stratified ones (with the global mean instead of $\bar{x}_i$
+and~$\bar{y}_i$) since $\bar{x}_i$ and $\bar{y}_i$ fit closer to $x_{i,j}$ and $y_{i,j}$
+than the global means. The stratified variance estimates the uncertainty in $x_{i,j}$
+and~$y_{i,j}$ given their stratum~$i$.
+
+Minimizing over~$\beta$ the error sum-of-squares~(\ref{eqn:stratsumsq})
+gives us the regression slope estimate \mbox{$\hat{\beta} = V_{x,y} / V_x$},
+with~(\ref{eqn:stratsumsq}) becoming the residual sum-of-squares~(RSS):
+\begin{equation*}
+\mathrm{RSS} \,\,=\, \,
+\sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(y_{i,j} -
+\hat{\beta} x_{i,j} - (\bar{y}_i - \hat{\beta} \bar{x}_i)\big)^2
+\,\,=\,\, V_y \,\big(1 \,-\, V_{x,y}^2 / (V_x V_y)\big)
+\end{equation*}
+The quantity $\hat{R}^2 = V_{x,y}^2 / (V_x V_y)$, called \emph{$R$-squared}, estimates the fraction
+of stratified variance in~$y_{i,j}$ explained by covariate $x_{i, j}$ in the linear
+regression model~(\ref{eqn:stratlinmodel}). We define \emph{stratified correlation} as the
+square root of~$\hat{R}^2$ taken with the sign of~$V_{x,y}$. We also use RSS to estimate
+the residual standard deviation $\sigma$ in~(\ref{eqn:stratlinmodel}) that models the prediction error
+of $y_{i,j}$ given $x_{i,j}$ and the stratum:
+\begin{equation*}
+\hat{\beta}\, =\, \frac{V_{x,y}}{V_x}; \,\,\,\, \hat{R} \,=\, \frac{V_{x,y}}{\sqrt{V_x V_y}};
+\,\,\,\, \hat{R}^2 \,=\, \frac{V_{x,y}^2}{V_x V_y};
+\,\,\,\, \hat{\sigma} \,=\, \sqrt{\frac{\mathrm{RSS}}{n - k - 1}}\,\,\,\,
+\Big(n = \sum_{i=1}^k n_i\Big)
+\end{equation*}
+
+The $t$-test and the $F$-test for the null-hypothesis of ``$\beta = 0$'' are
+obtained by considering the effect of $\hat{\beta}$ on the residual sum-of-squares,
+measured by the decrease from $V_y$ to~RSS.
+The $F$-statistic is the ratio of the ``explained'' sum-of-squares
+to the residual sum-of-squares, divided by their corresponding degrees of freedom.
+There are $n\,{-}\,k$ degrees of freedom for~$V_y$, parameter $\beta$ reduces that
+to $n\,{-}\,k\,{-}\,1$ for~RSS, and their difference $V_y - {}$RSS has just 1 degree
+of freedom:
+\begin{equation*}
+F \,=\, \frac{(V_y - \mathrm{RSS})/1}{\mathrm{RSS}/(n\,{-}\,k\,{-}\,1)}
+\,=\, \frac{\hat{R}^2\,(n\,{-}\,k\,{-}\,1)}{1-\hat{R}^2}; \quad
+t \,=\, \hat{R}\, \sqrt{\frac{n\,{-}\,k\,{-}\,1}{1-\hat{R}^2}}.
+\end{equation*}
+The $t$-statistic is simply the square root of the $F$-statistic with the appropriate
+choice of sign. If the null hypothesis and the linear model are both true, the $t$-statistic
+has Student $t$-distribution with $n\,{-}\,k\,{-}\,1$ degrees of freedom. We can
+also compute it if we divide $\hat{\beta}$ by its estimated standard deviation:
+\begin{equation*}
+\stdev(\hat{\beta})_{\mathrm{est}} \,=\, \hat{\sigma}\,/\sqrt{V_x} \quad\Longrightarrow\quad
+t \,=\, \hat{R}\sqrt{V_y} \,/\, \hat{\sigma} \,=\, \beta \,/\, \stdev(\hat{\beta})_{\mathrm{est}}
+\end{equation*}
+The standard deviation estimate for~$\beta$ is included in {\tt stratstats.dml} output.
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+The output matrix format is defined in Table~\ref{table:stratoutput}.
+
+\smallskip
+\noindent{\bf Examples}
+\smallskip
+
+{\hangindent=\parindent\noindent\tt
+\hml -f stratstats.dml -nvargs X=/user/biadmin/X.mtx Xcid=/user/biadmin/Xcid.mtx
+ Y=/user/biadmin/Y.mtx Ycid=/user/biadmin/Ycid.mtx S=/user/biadmin/S.mtx Scid=2
+ O=/user/biadmin/Out.mtx fmt=csv
+
+}
+{\hangindent=\parindent\noindent\tt
+\hml -f stratstats.dml -nvargs X=/user/biadmin/Data.mtx Xcid=/user/biadmin/Xcid.mtx
+ Ycid=/user/biadmin/Ycid.mtx Scid=7 O=/user/biadmin/Out.mtx
+
+}
+
+%\smallskip
+%\noindent{\bf See Also}
+%\smallskip
+%
+%For non-stratified bivariate statistics with a wider variety of input data types
+%and statistical tests, see \ldots. For general linear regression, see
+%{\tt LinearRegDS.dml} and {\tt LinearRegCG.dml}. For logistic regression, appropriate
+%when the response variable is categorical, see {\tt MultiLogReg.dml}.
+
[33/47] incubator-systemml git commit: Generalization wsloss
compiler/runtime (post_nz patterns), for als_cg
Posted by du...@apache.org.
Generalization wsloss compiler/runtime (post_nz patterns), for als_cg
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/43da6de9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/43da6de9
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/43da6de9
Branch: refs/heads/gh-pages
Commit: 43da6de9314d0a8659877ebb079ec5bf8a0883d0
Parents: 62b9b9c
Author: Matthias Boehm <mb...@us.ibm.com>
Authored: Tue Sep 29 00:17:52 2015 -0700
Committer: Matthias Boehm <mb...@us.ibm.com>
Committed: Tue Sep 29 09:52:39 2015 -0700
----------------------------------------------------------------------
devdocs/MatrixMultiplicationOperators.txt | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/43da6de9/devdocs/MatrixMultiplicationOperators.txt
----------------------------------------------------------------------
diff --git a/devdocs/MatrixMultiplicationOperators.txt b/devdocs/MatrixMultiplicationOperators.txt
index 02668a3..54cb5ec 100644
--- a/devdocs/MatrixMultiplicationOperators.txt
+++ b/devdocs/MatrixMultiplicationOperators.txt
@@ -97,7 +97,8 @@ C) CORE MATRIX MULT PRIMITIVES LibMatrixMult (incl related script patterns)
(c) sum((X-(U%*%t(V)))^2)), (d) sum(W*(U%*%t(V)-X)^2),
(e) sum((W*(U%*%t(V))-X)^2), (f) sum(((U%*%t(V))-X)^2))
- sequential / multi-threaded (same block ops, par over rows in W/X)
- - all dense, sparse-dense factors, sparse/dense-* x 3 patterns
+ - all dense, sparse-dense factors, sparse/dense-* x 3 patterns
+ - special patterns for (a) and (d) if W is X!=0
* 6) wsigmoid ((a) W*sigmoid(Y%*%t(X))), (b) W*sigmoid(-(Y%*%t(X))),
(c) W*log(sigmoid(Y%*%t(X))), (d) W*log(sigmoid(-(Y%*%t(X)))))
[23/47] incubator-systemml git commit: [SYSML-283] Create user guide
for SystemML Debugger
Posted by du...@apache.org.
[SYSML-283] Create user guide for SystemML Debugger
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/c0d0fded
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/c0d0fded
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/c0d0fded
Branch: refs/heads/gh-pages
Commit: c0d0fded1cbba26961a3ba5fae27b2c4d95488dd
Parents: 82895b1
Author: Glenn Weidner <gw...@us.ibm.com>
Authored: Thu Sep 10 14:01:53 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Thu Sep 10 14:01:53 2015 -0700
----------------------------------------------------------------------
debugger-guide.md | 477 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 477 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/c0d0fded/debugger-guide.md
----------------------------------------------------------------------
diff --git a/debugger-guide.md b/debugger-guide.md
new file mode 100644
index 0000000..f343a20
--- /dev/null
+++ b/debugger-guide.md
@@ -0,0 +1,477 @@
+---
+layout: global
+title: SystemML Debugger Guide
+displayTitle: SystemML Debugger Guide
+displayTitle: <a href="debugger-guide.html">SystemML Debugger Guide</a>
+---
+
+
+# SystemML Debugger
+
+SystemML supports DML script-level debugging through a command line interface. The SystemML debugger provides functionality typically found in a debugging environment like setting breakpoints, controlling program execution, and inspecting variables. To run a script in debug mode, specify the '-debug' option as shown in below example.
+
+ hadoop jar SystemML.jar -f test.dml -debug
+
+
+## Debugger Commands
+
+After starting a SystemML debug session, a list of available commands is automatically displayed. Debugger commands can be entered at the SystemML debugger prompt (SystemMLdb).
+The following sections describe each command along with example usage.
+
+ * [Help](#help)
+ * [Quit](#quit)
+ * [List](#list)
+ * [Step](#step)
+ * [Break](#break)
+ * [Delete](#delete)
+ * [Info break](#info-break)
+ * [Continue](#continue)
+ * [Run](#run)
+ * [Whatis](#whatis)
+ * [Print](#print)
+ * [Set](#set)
+ * [Info frame](#info-frame)
+ * [List instruction](#list-instruction)
+ * [Step instruction](#step-instruction)
+
+
+
+### Help
+
+Type h for help to display a summary of available debugger commands.
+
+ (SystemMLdb) h
+
+ SystemMLdb commands:
+ h,help list debugger functions
+ r,run start your DML script
+ q,quit exit debug mode
+ c,continue continue running your DML script
+ l,list <[next numlines] | [prev numlines] | [all]> display DML script source lines. Default: numlines = 10
+ b,break <line-number> set breakpoint at given line number
+ d,delete <line-number> delete breakpoint at given line number
+ s,step next line, stepping into function calls
+ i,info <break | frame> show all breakpoints or frames (info <break | frame>)
+ p,print <varName> display contents of a scalar or matrix variable or
+ rows/columns/cell of matrix. (Eg: 'p alpha' or 'p A' or 'p A[1,]')
+ set <varName value> set value of a scalar or specified cell of a matrix variable. (Eg:
+ 'set alpha 0.1' or 'set A[1,2] 20')
+ whatis <varName> display the type (and metadata) of a variable. (Eg: 'whatis alpha'
+ or 'whatis A')
+ li,listi <[next numlines] | [prev numlines] | [all]> display corresponding instructions for DML script source lines.
+ Default: numlines = 10 (for advanced users)
+ si,stepi next runtime instruction rather than DML source lines (for advanced
+ users)
+
+ (SystemMLdb)
+
+
+
+### Quit
+
+To exit a debug session, simply type q.
+
+ q,quit exit debug mode
+
+This returns control to the terminal or console shell which was used to launch the session.
+
+ (SystemMLdb) q
+ $
+
+
+## Debugger commands for controlling script execution
+
+This section describes commands that can be used to view and control script execution. The following example script test.dml is used to demonstrate simple command usage scenarios.
+
+ A = rand (rows=10, cols=5);
+ B = rand (rows=5, cols=4);
+ D = sum(A);
+ print("Sum(A)=" + D);
+ C = A %*% B;
+ write(C, "output.csv", format="csv");
+
+
+
+
+### List
+
+After initially launching a debug session, the script is loaded and ready to be run. The l command can be used to display the source lines of the loaded script.
+
+ l,list <[next numlines] | [prev numlines] | [all]> display DML script source lines. Default: numlines = 10
+
+Without specifying any options, the list shows up to the next 10 lines of the script. For example:
+
+ (SystemMLdb) l
+ line 1: A = rand (rows=10, cols=5);
+ line 2: B = rand (rows=5, cols=4);
+ line 3: D = sum(A);
+ line 4: print("Sum(A)=" + D);
+ line 5: C = A %*% B;
+ line 6: write(C, "output.csv", format="csv");
+
+
+
+
+### Step
+
+Each line of the script can be stepped through using the s command.
+
+ s,step next line, stepping into function calls
+
+So continuing with the example from previous section, typing s executes the current line 1:
+
+ (SystemMLdb) s
+ Step reached at .defaultNS::main: (line 2).
+ 2 B = rand (rows=5, cols=4);
+ (SystemMLdb)
+
+As can be seen from the output, the debugger executed line 1 and advanced to the next line in script. The current line is automatically displayed.
+
+
+
+
+### Break
+
+To execute a group of instructions up to a specific line, breakpoints can be used. To set a breakpoint, use the b command.
+
+ b,break <line-number> set breakpoint at given line number
+
+
+Continuing the example from step command, the current line was 2. The below command sets a breakpoint at script source line number 4.
+
+ (SystemMLdb) b 4
+ Breakpoint added at .defaultNS::main, line 4.
+ (SystemMLdb)
+
+
+
+
+### Delete
+
+Use the d command to remove a breakpoint.
+
+ d,delete <line-number> delete breakpoint at given line number
+
+Below is sample output when removing a breakpoint.
+
+ (SystemMLdb) d 4
+ Breakpoint updated at .defaultNS::main, line 4.
+ (SystemMLdb)
+
+If no breakpoint was set at the specified line number, then an appropriate message is displayed.
+
+ (SystemMLdb) d 4
+ Sorry, a breakpoint cannot be deleted at line 4. Please try a different line number.
+ (SystemMLdb)
+
+
+
+
+### Info break
+
+To see a list of breakpoints, use the i command with the break option.
+
+ i,info break show all breakpoints
+
+Below is sample output after setting breakpoints at lines 2 and 4 of test.dml script.
+
+ (SystemMLdb) i break
+ Breakpoint 1, at line 2 (enabled)
+ Breakpoint 2, at line 4 (enabled)
+ (SystemMLdb)
+
+The info command also has a frame option which is discussed in the section related to inspecting script variables.
+
+ i,info <break | frame> show all breakpoints or frames (info <break | frame>)
+
+
+
+
+### Continue
+
+The continue command resumes script execution from the current line up to the next breakpoint. If no breakpoints are set, then the rest of the script will be executed and the debugger session terminated.
+
+ c,continue continue running your DML script
+
+Since the previous section set a breakpoint at line number 4, typing c to continue executes from the current line (2) up to but not including line 4 (i.e., the line with the breakpoint).
+
+ (SystemMLdb) c
+ Resuming DML script execution ...
+ Breakpoint reached at .defaultNS::main instID 1: (line 4).
+ 4 print("Sum(A)=" + D);
+ (SystemMLdb)
+
+Note that continue is not a valid command if the SystemML runtime has not been started.
+
+ (SystemMLdb) c
+ Runtime has not been started. Try "r" to start DML runtime execution.
+ (SystemMLdb)
+
+
+
+
+### Run
+
+There are two ways of starting the SystemML runtime for a debug session - the step command or the run command. A common scenario is to set breakpoint(s) in the beginning of a debug session, then use r to start the runtime and run until the breakpoint is reached or script completion.
+
+ r,run start your DML script
+
+Using the same script from the previous example, the r command can be used in the beginning of the session to run the script up to a breakpoint or program completion if no breakpoint were set or reached.
+
+ (SystemMLdb) l
+ line 1: A = rand (rows=10, cols=5);
+ line 2: B = rand (rows=5, cols=4);
+ line 3: D = sum(A);
+ line 4: print("Sum(A)=" + D);
+ line 5: C = A %*% B;
+ line 6: write(C, "output.csv", format="csv");
+ (SystemMLdb) b 4
+ Breakpoint added at .defaultNS::main, line 4.
+ (SystemMLdb) r
+ Breakpoint reached at .defaultNS::main instID 1: (line 4).
+ 4 print("Sum(A)=" + D);
+ (SystemMLdb)
+
+Note the run command is not valid if the runtime has already been started. In that case, use continue or step to execute line(s) of the script.
+
+ (SystemMLdb) r
+ Runtime has already started. Try "s" to go to next line, or "c" to continue running your DML script.
+ (SystemMLdb)
+
+
+## Debugger Commands for inspecting or modifying script variables
+
+Variables that are in scope can be displayed in multiple ways. The same test.dml script is used for showing sample command usage.
+
+ A = rand (rows=10, cols=5);
+ B = rand (rows=5, cols=4);
+ D = sum(A);
+ print("Sum(A)=" + D);
+ C = A %*% B;
+ write(C, "output.csv", format="csv");
+
+
+
+
+### Whatis
+
+To display the type of a variable, use the whatis command.
+
+ whatis <varName> display the type (and metadata) of a variable. (Eg: 'whatis alpha'
+ or 'whatis A')
+
+Given sample test.dml script with current line 4, then the metadata of variables A, B, D can be shown.
+
+ (SystemMLdb) whatis A
+ Metadata of A: matrix[rows = 10, cols = 5, rpb = 1000, cpb = 1000]
+ (SystemMLdb) whatis B
+ Metadata of B: matrix[rows = 5, cols = 4, rpb = 1000, cpb = 1000]
+ (SystemMLdb) whatis D
+ Metadata of D: DataType.SCALAR
+ (SystemMLdb)
+
+
+
+
+### Print
+
+To view the contents of a variable, use the p command.
+
+ p,print <varName> display contents of a scalar or matrix variable or
+ rows/columns/cell of matrix. (Eg: 'p alpha' or 'p A' or 'p A[1,]')
+
+Below is sample print output for the same variables used in previous section.
+
+ (SystemMLdb) p A
+ 0.6911 0.0533 0.7659 0.9130 0.1196
+ 0.8153 0.6145 0.5440 0.2916 0.7330
+ 0.0520 0.9484 0.2044 0.5571 0.6952
+ 0.7422 0.4134 0.5388 0.1192 0.8733
+ 0.6413 0.1825 0.4818 0.9019 0.7446
+ 0.5984 0.8577 0.7151 0.3002 0.2228
+ 0.0090 0.1429 0.2569 0.1421 0.1357
+ 0.6778 0.8078 0.5075 0.0085 0.5159
+ 0.8835 0.5621 0.7637 0.4362 0.4392
+ 0.6108 0.5600 0.6140 0.0163 0.8640
+ (SystemMLdb) p B
+ 0.4141 0.9905 0.1642 0.7545
+ 0.5733 0.1489 0.1204 0.5375
+ 0.5202 0.9833 0.3421 0.7099
+ 0.5846 0.7585 0.9751 0.1174
+ 0.8431 0.5806 0.4122 0.3694
+ (SystemMLdb) p D
+ D = 25.28558886582987
+ (SystemMLdb)
+
+To display a specific element of a matrix, use [row,column] notation.
+
+ (SystemMLdb) p A[1,1]
+ 0.6911
+ (SystemMLdb) p A[10,5]
+ 0.8640
+ (SystemMLdb)
+
+Specific rows or columns of a matrix can also be displayed. The below examples show the first row and the fifth column of matrix A.
+
+ (SystemMLdb) p A[1,]
+ 0.6911 0.0533 0.7659 0.9130 0.1196
+ (SystemMLdb) p A[,5]
+ 0.1196
+ 0.7330
+ 0.6952
+ 0.8733
+ 0.7446
+ 0.2228
+ 0.1357
+ 0.5159
+ 0.4392
+ 0.8640
+ (SystemMLdb)
+
+
+
+
+### Set
+
+The set command is used for modifying variable contents.
+
+ set <varName value> set value of a scalar or specified cell of a matrix variable. (Eg:
+ 'set alpha 0.1' or 'set A[1,2] 20')
+
+The following example modifies the first cell in matrix A.
+
+ (SystemMLdb) set A[1,1] 0.3299
+ A[1,1] = 0.3299
+ (SystemMLdb)
+
+This example updates scalar D. Note an equals sign is not needed when setting a variable.
+
+ (SystemMLdb) set D 25.0
+ D = 25.0
+ (SystemMLdb)
+
+
+
+
+### Info frame
+
+In addition to being used for displaying breakpoints, the i command is used for displaying frames.
+
+ i,info frame show all frames
+
+So if our test.xml script was executed up to line 4, then the following frame information is shown.
+
+ (SystemMLdb) i frame
+ Current frame id: 0
+ Current program counter at .defaultNS::main instID -1: (line 4)
+ Local variables:
+ Variable name Variable value
+ A Matrix: scratch_space//_p48857_9.30.252.162//_t0/temp1_1, [10 x 5, nnz=50, blocks (1000 x 1000)], binaryblock, dirty
+ B Matrix: scratch_space//_p48857_9.30.252.162//_t0/temp2_2, [5 x 4, nnz=20, blocks (1000 x 1000)], binaryblock, dirty
+ D 25.28558886582987
+ (SystemMLdb)
+
+Note only variables that are in scope are included (e.g., the variable C is not part of the frame since not yet in scope).
+
+
+## Advanced Debugger Commands
+
+This section describes commands for advanced users. The same test.dml script is used for showing sample command usage.
+
+ A = rand (rows=10, cols=5);
+ B = rand (rows=5, cols=4);
+ D = sum(A);
+ print("Sum(A)=" + D);
+ C = A %*% B;
+ write(C, "output.csv", format="csv");
+
+
+
+
+### List Instruction
+
+The li command can be used to display lower-level instructions along with the source lines of the loaded script.
+
+ li,listi <[next numlines] | [prev numlines] | [all]> display corresponding instructions for DML script source lines.
+ Default: numlines = 10 (for advanced users)
+
+For example:
+
+ (SystemMLdb) li
+ line 1: A = rand (rows=10, cols=5);
+ id -1: CP createvar _mVar1 scratch_space//_p1939_9.30.252.162//_t0/temp1 true binaryblock 10 5 1000 1000 50
+ id -1: CP rand 10 5 1000 1000 0.0 1.0 1.0 -1 uniform 1.0 4 _mVar1.MATRIX.DOUBLE
+ id -1: CP cpvar _mVar1 A
+ id -1: CP rmvar _mVar1
+ line 2: B = rand (rows=5, cols=4);
+ id -1: CP createvar _mVar2 scratch_space//_p1939_9.30.252.162//_t0/temp2 true binaryblock 5 4 1000 1000 20
+ id -1: CP rand 5 4 1000 1000 0.0 1.0 1.0 -1 uniform 1.0 4 _mVar2.MATRIX.DOUBLE
+ id -1: CP cpvar _mVar2 B
+ id -1: CP rmvar _mVar2
+ line 3: D = sum(A);
+ id -1: CP uak+ A.MATRIX.DOUBLE _Var3.SCALAR.DOUBLE
+ id -1: CP assignvar _Var3.SCALAR.DOUBLE.false D.SCALAR.DOUBLE
+ id -1: CP rmvar _Var3
+ line 4: print("Sum(A)=" + D);
+ id -1: CP + Sum(A)=.SCALAR.STRING.true D.SCALAR.DOUBLE.false _Var4.SCALAR.STRING
+ id -1: CP print _Var4.SCALAR.STRING.false _Var5.SCALAR.STRING
+ id -1: CP rmvar _Var4
+ id -1: CP rmvar _Var5
+ id -1: CP rmvar D
+ line 5: C = A %*% B;
+ id -1: CP createvar _mVar6 scratch_space//_p1939_9.30.252.162//_t0/temp3 true binaryblock 10 4 1000 1000 -1
+ id -1: CP ba+* A.MATRIX.DOUBLE B.MATRIX.DOUBLE _mVar6.MATRIX.DOUBLE 4
+ id -1: CP cpvar _mVar6 C
+ id -1: CP rmvar _mVar6
+ id -1: CP rmvar A
+ id -1: CP rmvar B
+ line 6: write(C, "output.csv", format="csv");
+ id -1: CP write C.MATRIX.DOUBLE output.csv.SCALAR.STRING.true csv.SCALAR.STRING.true false , false
+ id -1: CP rmvar C
+ (SystemMLdb)
+
+
+
+
+### Step Instruction
+
+The si command can be used to step through the lower level instructions of an individual source line in a DML script.
+
+ si,stepi next runtime instruction rather than DML source lines (for advanced
+ users)
+
+The first DML source line in test.dml consists of four instructions.
+
+
+ (SystemMLdb) li next 0
+ line 1: A = rand (rows=10, cols=5);
+ id -1: CP createvar _mVar1 scratch_space//_p34473_9.30.252.162//_t0/temp1 true binaryblock 10 5 1000 1000 50
+ id -1: CP rand 10 5 1000 1000 0.0 1.0 1.0 -1 uniform 1.0 4 _mVar1.MATRIX.DOUBLE
+ id -1: CP cpvar _mVar1 A
+ id -1: CP rmvar _mVar1
+ (SystemMLdb)
+
+Type si to step through each individual instruction.
+
+ (SystemMLdb) si
+ Step instruction reached at .defaultNS::main instID -1: (line 1).
+ 1 A = rand (rows=10, cols=5);
+ (SystemMLdb) si
+ Step instruction reached at .defaultNS::main instID -1: (line 1).
+ 1 A = rand (rows=10, cols=5);
+ (SystemMLdb) si
+ Step instruction reached at .defaultNS::main instID -1: (line 1).
+ 1 A = rand (rows=10, cols=5);
+ (SystemMLdb) si
+ Step instruction reached at .defaultNS::main instID -1: (line 1).
+ 1 A = rand (rows=10, cols=5);
+ (SystemMLdb)
+
+Typing si again starts executing instructions of the next DML source line.
+
+ (SystemMLdb) si
+ Step instruction reached at .defaultNS::main instID -1: (line 2).
+ 2 B = rand (rows=5, cols=4);
+ (SystemMLdb)
+
+* * *
[10/47] incubator-systemml git commit: initial port of documentation
to md, algorithms reference converted to md, template based on spark docs
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/js/vendor/bootstrap.min.js
----------------------------------------------------------------------
diff --git a/js/vendor/bootstrap.min.js b/js/vendor/bootstrap.min.js
new file mode 100644
index 0000000..66e887b
--- /dev/null
+++ b/js/vendor/bootstrap.min.js
@@ -0,0 +1,6 @@
+/*!
+* Bootstrap.js by @fat & @mdo
+* Copyright 2012 Twitter, Inc.
+* http://www.apache.org/licenses/LICENSE-2.0.txt
+*/
+!function(e){e(function(){"use strict";e.support.transition=function(){var e=function(){var e=document.createElement("bootstrap"),t={WebkitTransition:"webkitTransitionEnd",MozTransition:"transitionend",OTransition:"oTransitionEnd otransitionend",transition:"transitionend"},n;for(n in t)if(e.style[n]!==undefined)return t[n]}();return e&&{end:e}}()})}(window.jQuery),!function(e){"use strict";var t='[data-dismiss="alert"]',n=function(n){e(n).on("click",t,this.close)};n.prototype.close=function(t){function s(){i.trigger("closed").remove()}var n=e(this),r=n.attr("data-target"),i;r||(r=n.attr("href"),r=r&&r.replace(/.*(?=#[^\s]*$)/,"")),i=e(r),t&&t.preventDefault(),i.length||(i=n.hasClass("alert")?n:n.parent()),i.trigger(t=e.Event("close"));if(t.isDefaultPrevented())return;i.removeClass("in"),e.support.transition&&i.hasClass("fade")?i.on(e.support.transition.end,s):s()},e.fn.alert=function(t){return this.each(function(){var r=e(this),i=r.data("alert");i||r.data("alert",i=new n(this)),type
of t=="string"&&i[t].call(r)})},e.fn.alert.Constructor=n,e(function(){e("body").on("click.alert.data-api",t,n.prototype.close)})}(window.jQuery),!function(e){"use strict";var t=function(t,n){this.$element=e(t),this.options=e.extend({},e.fn.button.defaults,n)};t.prototype.setState=function(e){var t="disabled",n=this.$element,r=n.data(),i=n.is("input")?"val":"html";e+="Text",r.resetText||n.data("resetText",n[i]()),n[i](r[e]||this.options[e]),setTimeout(function(){e=="loadingText"?n.addClass(t).attr(t,t):n.removeClass(t).removeAttr(t)},0)},t.prototype.toggle=function(){var e=this.$element.parent('[data-toggle="buttons-radio"]');e&&e.find(".active").removeClass("active"),this.$element.toggleClass("active")},e.fn.button=function(n){return this.each(function(){var r=e(this),i=r.data("button"),s=typeof n=="object"&&n;i||r.data("button",i=new t(this,s)),n=="toggle"?i.toggle():n&&i.setState(n)})},e.fn.button.defaults={loadingText:"loading..."},e.fn.button.Constructor=t,e(function(){e("body")
.on("click.button.data-api","[data-toggle^=button]",function(t){var n=e(t.target);n.hasClass("btn")||(n=n.closest(".btn")),n.button("toggle")})})}(window.jQuery),!function(e){"use strict";var t=function(t,n){this.$element=e(t),this.options=n,this.options.slide&&this.slide(this.options.slide),this.options.pause=="hover"&&this.$element.on("mouseenter",e.proxy(this.pause,this)).on("mouseleave",e.proxy(this.cycle,this))};t.prototype={cycle:function(t){return t||(this.paused=!1),this.options.interval&&!this.paused&&(this.interval=setInterval(e.proxy(this.next,this),this.options.interval)),this},to:function(t){var n=this.$element.find(".item.active"),r=n.parent().children(),i=r.index(n),s=this;if(t>r.length-1||t<0)return;return this.sliding?this.$element.one("slid",function(){s.to(t)}):i==t?this.pause().cycle():this.slide(t>i?"next":"prev",e(r[t]))},pause:function(t){return t||(this.paused=!0),this.$element.find(".next, .prev").length&&e.support.transition.end&&(this.$element.trigger(e.su
pport.transition.end),this.cycle()),clearInterval(this.interval),this.interval=null,this},next:function(){if(this.sliding)return;return this.slide("next")},prev:function(){if(this.sliding)return;return this.slide("prev")},slide:function(t,n){var r=this.$element.find(".item.active"),i=n||r[t](),s=this.interval,o=t=="next"?"left":"right",u=t=="next"?"first":"last",a=this,f=e.Event("slide",{relatedTarget:i[0]});this.sliding=!0,s&&this.pause(),i=i.length?i:this.$element.find(".item")[u]();if(i.hasClass("active"))return;if(e.support.transition&&this.$element.hasClass("slide")){this.$element.trigger(f);if(f.isDefaultPrevented())return;i.addClass(t),i[0].offsetWidth,r.addClass(o),i.addClass(o),this.$element.one(e.support.transition.end,function(){i.removeClass([t,o].join(" ")).addClass("active"),r.removeClass(["active",o].join(" ")),a.sliding=!1,setTimeout(function(){a.$element.trigger("slid")},0)})}else{this.$element.trigger(f);if(f.isDefaultPrevented())return;r.removeClass("active"),i.ad
dClass("active"),this.sliding=!1,this.$element.trigger("slid")}return s&&this.cycle(),this}},e.fn.carousel=function(n){return this.each(function(){var r=e(this),i=r.data("carousel"),s=e.extend({},e.fn.carousel.defaults,typeof n=="object"&&n),o=typeof n=="string"?n:s.slide;i||r.data("carousel",i=new t(this,s)),typeof n=="number"?i.to(n):o?i[o]():s.interval&&i.cycle()})},e.fn.carousel.defaults={interval:5e3,pause:"hover"},e.fn.carousel.Constructor=t,e(function(){e("body").on("click.carousel.data-api","[data-slide]",function(t){var n=e(this),r,i=e(n.attr("data-target")||(r=n.attr("href"))&&r.replace(/.*(?=#[^\s]+$)/,"")),s=!i.data("modal")&&e.extend({},i.data(),n.data());i.carousel(s),t.preventDefault()})})}(window.jQuery),!function(e){"use strict";var t=function(t,n){this.$element=e(t),this.options=e.extend({},e.fn.collapse.defaults,n),this.options.parent&&(this.$parent=e(this.options.parent)),this.options.toggle&&this.toggle()};t.prototype={constructor:t,dimension:function(){var e=th
is.$element.hasClass("width");return e?"width":"height"},show:function(){var t,n,r,i;if(this.transitioning)return;t=this.dimension(),n=e.camelCase(["scroll",t].join("-")),r=this.$parent&&this.$parent.find("> .accordion-group > .in");if(r&&r.length){i=r.data("collapse");if(i&&i.transitioning)return;r.collapse("hide"),i||r.data("collapse",null)}this.$element[t](0),this.transition("addClass",e.Event("show"),"shown"),e.support.transition&&this.$element[t](this.$element[0][n])},hide:function(){var t;if(this.transitioning)return;t=this.dimension(),this.reset(this.$element[t]()),this.transition("removeClass",e.Event("hide"),"hidden"),this.$element[t](0)},reset:function(e){var t=this.dimension();return this.$element.removeClass("collapse")[t](e||"auto")[0].offsetWidth,this.$element[e!==null?"addClass":"removeClass"]("collapse"),this},transition:function(t,n,r){var i=this,s=function(){n.type=="show"&&i.reset(),i.transitioning=0,i.$element.trigger(r)};this.$element.trigger(n);if(n.isDefaultPr
evented())return;this.transitioning=1,this.$element[t]("in"),e.support.transition&&this.$element.hasClass("collapse")?this.$element.one(e.support.transition.end,s):s()},toggle:function(){this[this.$element.hasClass("in")?"hide":"show"]()}},e.fn.collapse=function(n){return this.each(function(){var r=e(this),i=r.data("collapse"),s=typeof n=="object"&&n;i||r.data("collapse",i=new t(this,s)),typeof n=="string"&&i[n]()})},e.fn.collapse.defaults={toggle:!0},e.fn.collapse.Constructor=t,e(function(){e("body").on("click.collapse.data-api","[data-toggle=collapse]",function(t){var n=e(this),r,i=n.attr("data-target")||t.preventDefault()||(r=n.attr("href"))&&r.replace(/.*(?=#[^\s]+$)/,""),s=e(i).data("collapse")?"toggle":n.data();n[e(i).hasClass("in")?"addClass":"removeClass"]("collapsed"),e(i).collapse(s)})})}(window.jQuery),!function(e){"use strict";function r(){i(e(t)).removeClass("open")}function i(t){var n=t.attr("data-target"),r;return n||(n=t.attr("href"),n=n&&n.replace(/.*(?=#[^\s]*$)/,"
")),r=e(n),r.length||(r=t.parent()),r}var t="[data-toggle=dropdown]",n=function(t){var n=e(t).on("click.dropdown.data-api",this.toggle);e("html").on("click.dropdown.data-api",function(){n.parent().removeClass("open")})};n.prototype={constructor:n,toggle:function(t){var n=e(this),s,o;if(n.is(".disabled, :disabled"))return;return s=i(n),o=s.hasClass("open"),r(),o||(s.toggleClass("open"),n.focus()),!1},keydown:function(t){var n,r,s,o,u,a;if(!/(38|40|27)/.test(t.keyCode))return;n=e(this),t.preventDefault(),t.stopPropagation();if(n.is(".disabled, :disabled"))return;o=i(n),u=o.hasClass("open");if(!u||u&&t.keyCode==27)return n.click();r=e("[role=menu] li:not(.divider) a",o);if(!r.length)return;a=r.index(r.filter(":focus")),t.keyCode==38&&a>0&&a--,t.keyCode==40&&a<r.length-1&&a++,~a||(a=0),r.eq(a).focus()}},e.fn.dropdown=function(t){return this.each(function(){var r=e(this),i=r.data("dropdown");i||r.data("dropdown",i=new n(this)),typeof t=="string"&&i[t].call(r)})},e.fn.dropdown.Constructor
=n,e(function(){e("html").on("click.dropdown.data-api touchstart.dropdown.data-api",r),e("body").on("click.dropdown touchstart.dropdown.data-api",".dropdown",function(e){e.stopPropagation()}).on("click.dropdown.data-api touchstart.dropdown.data-api",t,n.prototype.toggle).on("keydown.dropdown.data-api touchstart.dropdown.data-api",t+", [role=menu]",n.prototype.keydown)})}(window.jQuery),!function(e){"use strict";var t=function(t,n){this.options=n,this.$element=e(t).delegate('[data-dismiss="modal"]',"click.dismiss.modal",e.proxy(this.hide,this)),this.options.remote&&this.$element.find(".modal-body").load(this.options.remote)};t.prototype={constructor:t,toggle:function(){return this[this.isShown?"hide":"show"]()},show:function(){var t=this,n=e.Event("show");this.$element.trigger(n);if(this.isShown||n.isDefaultPrevented())return;e("body").addClass("modal-open"),this.isShown=!0,this.escape(),this.backdrop(function(){var n=e.support.transition&&t.$element.hasClass("fade");t.$element.paren
t().length||t.$element.appendTo(document.body),t.$element.show(),n&&t.$element[0].offsetWidth,t.$element.addClass("in").attr("aria-hidden",!1).focus(),t.enforceFocus(),n?t.$element.one(e.support.transition.end,function(){t.$element.trigger("shown")}):t.$element.trigger("shown")})},hide:function(t){t&&t.preventDefault();var n=this;t=e.Event("hide"),this.$element.trigger(t);if(!this.isShown||t.isDefaultPrevented())return;this.isShown=!1,e("body").removeClass("modal-open"),this.escape(),e(document).off("focusin.modal"),this.$element.removeClass("in").attr("aria-hidden",!0),e.support.transition&&this.$element.hasClass("fade")?this.hideWithTransition():this.hideModal()},enforceFocus:function(){var t=this;e(document).on("focusin.modal",function(e){t.$element[0]!==e.target&&!t.$element.has(e.target).length&&t.$element.focus()})},escape:function(){var e=this;this.isShown&&this.options.keyboard?this.$element.on("keyup.dismiss.modal",function(t){t.which==27&&e.hide()}):this.isShown||this.$ele
ment.off("keyup.dismiss.modal")},hideWithTransition:function(){var t=this,n=setTimeout(function(){t.$element.off(e.support.transition.end),t.hideModal()},500);this.$element.one(e.support.transition.end,function(){clearTimeout(n),t.hideModal()})},hideModal:function(e){this.$element.hide().trigger("hidden"),this.backdrop()},removeBackdrop:function(){this.$backdrop.remove(),this.$backdrop=null},backdrop:function(t){var n=this,r=this.$element.hasClass("fade")?"fade":"";if(this.isShown&&this.options.backdrop){var i=e.support.transition&&r;this.$backdrop=e('<div class="modal-backdrop '+r+'" />').appendTo(document.body),this.options.backdrop!="static"&&this.$backdrop.click(e.proxy(this.hide,this)),i&&this.$backdrop[0].offsetWidth,this.$backdrop.addClass("in"),i?this.$backdrop.one(e.support.transition.end,t):t()}else!this.isShown&&this.$backdrop?(this.$backdrop.removeClass("in"),e.support.transition&&this.$element.hasClass("fade")?this.$backdrop.one(e.support.transition.end,e.proxy(this.rem
oveBackdrop,this)):this.removeBackdrop()):t&&t()}},e.fn.modal=function(n){return this.each(function(){var r=e(this),i=r.data("modal"),s=e.extend({},e.fn.modal.defaults,r.data(),typeof n=="object"&&n);i||r.data("modal",i=new t(this,s)),typeof n=="string"?i[n]():s.show&&i.show()})},e.fn.modal.defaults={backdrop:!0,keyboard:!0,show:!0},e.fn.modal.Constructor=t,e(function(){e("body").on("click.modal.data-api",'[data-toggle="modal"]',function(t){var n=e(this),r=n.attr("href"),i=e(n.attr("data-target")||r&&r.replace(/.*(?=#[^\s]+$)/,"")),s=i.data("modal")?"toggle":e.extend({remote:!/#/.test(r)&&r},i.data(),n.data());t.preventDefault(),i.modal(s).one("hide",function(){n.focus()})})})}(window.jQuery),!function(e){"use strict";var t=function(e,t){this.init("tooltip",e,t)};t.prototype={constructor:t,init:function(t,n,r){var i,s;this.type=t,this.$element=e(n),this.options=this.getOptions(r),this.enabled=!0,this.options.trigger=="click"?this.$element.on("click."+this.type,this.options.selector,
e.proxy(this.toggle,this)):this.options.trigger!="manual"&&(i=this.options.trigger=="hover"?"mouseenter":"focus",s=this.options.trigger=="hover"?"mouseleave":"blur",this.$element.on(i+"."+this.type,this.options.selector,e.proxy(this.enter,this)),this.$element.on(s+"."+this.type,this.options.selector,e.proxy(this.leave,this))),this.options.selector?this._options=e.extend({},this.options,{trigger:"manual",selector:""}):this.fixTitle()},getOptions:function(t){return t=e.extend({},e.fn[this.type].defaults,t,this.$element.data()),t.delay&&typeof t.delay=="number"&&(t.delay={show:t.delay,hide:t.delay}),t},enter:function(t){var n=e(t.currentTarget)[this.type](this._options).data(this.type);if(!n.options.delay||!n.options.delay.show)return n.show();clearTimeout(this.timeout),n.hoverState="in",this.timeout=setTimeout(function(){n.hoverState=="in"&&n.show()},n.options.delay.show)},leave:function(t){var n=e(t.currentTarget)[this.type](this._options).data(this.type);this.timeout&&clearTimeout(t
his.timeout);if(!n.options.delay||!n.options.delay.hide)return n.hide();n.hoverState="out",this.timeout=setTimeout(function(){n.hoverState=="out"&&n.hide()},n.options.delay.hide)},show:function(){var e,t,n,r,i,s,o;if(this.hasContent()&&this.enabled){e=this.tip(),this.setContent(),this.options.animation&&e.addClass("fade"),s=typeof this.options.placement=="function"?this.options.placement.call(this,e[0],this.$element[0]):this.options.placement,t=/in/.test(s),e.remove().css({top:0,left:0,display:"block"}).appendTo(t?this.$element:document.body),n=this.getPosition(t),r=e[0].offsetWidth,i=e[0].offsetHeight;switch(t?s.split(" ")[1]:s){case"bottom":o={top:n.top+n.height,left:n.left+n.width/2-r/2};break;case"top":o={top:n.top-i,left:n.left+n.width/2-r/2};break;case"left":o={top:n.top+n.height/2-i/2,left:n.left-r};break;case"right":o={top:n.top+n.height/2-i/2,left:n.left+n.width}}e.css(o).addClass(s).addClass("in")}},setContent:function(){var e=this.tip(),t=this.getTitle();e.find(".tooltip-
inner")[this.options.html?"html":"text"](t),e.removeClass("fade in top bottom left right")},hide:function(){function r(){var t=setTimeout(function(){n.off(e.support.transition.end).remove()},500);n.one(e.support.transition.end,function(){clearTimeout(t),n.remove()})}var t=this,n=this.tip();return n.removeClass("in"),e.support.transition&&this.$tip.hasClass("fade")?r():n.remove(),this},fixTitle:function(){var e=this.$element;(e.attr("title")||typeof e.attr("data-original-title")!="string")&&e.attr("data-original-title",e.attr("title")||"").removeAttr("title")},hasContent:function(){return this.getTitle()},getPosition:function(t){return e.extend({},t?{top:0,left:0}:this.$element.offset(),{width:this.$element[0].offsetWidth,height:this.$element[0].offsetHeight})},getTitle:function(){var e,t=this.$element,n=this.options;return e=t.attr("data-original-title")||(typeof n.title=="function"?n.title.call(t[0]):n.title),e},tip:function(){return this.$tip=this.$tip||e(this.options.template)},v
alidate:function(){this.$element[0].parentNode||(this.hide(),this.$element=null,this.options=null)},enable:function(){this.enabled=!0},disable:function(){this.enabled=!1},toggleEnabled:function(){this.enabled=!this.enabled},toggle:function(){this[this.tip().hasClass("in")?"hide":"show"]()},destroy:function(){this.hide().$element.off("."+this.type).removeData(this.type)}},e.fn.tooltip=function(n){return this.each(function(){var r=e(this),i=r.data("tooltip"),s=typeof n=="object"&&n;i||r.data("tooltip",i=new t(this,s)),typeof n=="string"&&i[n]()})},e.fn.tooltip.Constructor=t,e.fn.tooltip.defaults={animation:!0,placement:"top",selector:!1,template:'<div class="tooltip"><div class="tooltip-arrow"></div><div class="tooltip-inner"></div></div>',trigger:"hover",title:"",delay:0,html:!0}}(window.jQuery),!function(e){"use strict";var t=function(e,t){this.init("popover",e,t)};t.prototype=e.extend({},e.fn.tooltip.Constructor.prototype,{constructor:t,setContent:function(){var e=this.tip(),t=this
.getTitle(),n=this.getContent();e.find(".popover-title")[this.options.html?"html":"text"](t),e.find(".popover-content > *")[this.options.html?"html":"text"](n),e.removeClass("fade top bottom left right in")},hasContent:function(){return this.getTitle()||this.getContent()},getContent:function(){var e,t=this.$element,n=this.options;return e=t.attr("data-content")||(typeof n.content=="function"?n.content.call(t[0]):n.content),e},tip:function(){return this.$tip||(this.$tip=e(this.options.template)),this.$tip},destroy:function(){this.hide().$element.off("."+this.type).removeData(this.type)}}),e.fn.popover=function(n){return this.each(function(){var r=e(this),i=r.data("popover"),s=typeof n=="object"&&n;i||r.data("popover",i=new t(this,s)),typeof n=="string"&&i[n]()})},e.fn.popover.Constructor=t,e.fn.popover.defaults=e.extend({},e.fn.tooltip.defaults,{placement:"right",trigger:"click",content:"",template:'<div class="popover"><div class="arrow"></div><div class="popover-inner"><h3 class="p
opover-title"></h3><div class="popover-content"><p></p></div></div></div>'})}(window.jQuery),!function(e){"use strict";function t(t,n){var r=e.proxy(this.process,this),i=e(t).is("body")?e(window):e(t),s;this.options=e.extend({},e.fn.scrollspy.defaults,n),this.$scrollElement=i.on("scroll.scroll-spy.data-api",r),this.selector=(this.options.target||(s=e(t).attr("href"))&&s.replace(/.*(?=#[^\s]+$)/,"")||"")+" .nav li > a",this.$body=e("body"),this.refresh(),this.process()}t.prototype={constructor:t,refresh:function(){var t=this,n;this.offsets=e([]),this.targets=e([]),n=this.$body.find(this.selector).map(function(){var t=e(this),n=t.data("target")||t.attr("href"),r=/^#\w/.test(n)&&e(n);return r&&r.length&&[[r.position().top,n]]||null}).sort(function(e,t){return e[0]-t[0]}).each(function(){t.offsets.push(this[0]),t.targets.push(this[1])})},process:function(){var e=this.$scrollElement.scrollTop()+this.options.offset,t=this.$scrollElement[0].scrollHeight||this.$body[0].scrollHeight,n=t-this
.$scrollElement.height(),r=this.offsets,i=this.targets,s=this.activeTarget,o;if(e>=n)return s!=(o=i.last()[0])&&this.activate(o);for(o=r.length;o--;)s!=i[o]&&e>=r[o]&&(!r[o+1]||e<=r[o+1])&&this.activate(i[o])},activate:function(t){var n,r;this.activeTarget=t,e(this.selector).parent(".active").removeClass("active"),r=this.selector+'[data-target="'+t+'"],'+this.selector+'[href="'+t+'"]',n=e(r).parent("li").addClass("active"),n.parent(".dropdown-menu").length&&(n=n.closest("li.dropdown").addClass("active")),n.trigger("activate")}},e.fn.scrollspy=function(n){return this.each(function(){var r=e(this),i=r.data("scrollspy"),s=typeof n=="object"&&n;i||r.data("scrollspy",i=new t(this,s)),typeof n=="string"&&i[n]()})},e.fn.scrollspy.Constructor=t,e.fn.scrollspy.defaults={offset:10},e(window).on("load",function(){e('[data-spy="scroll"]').each(function(){var t=e(this);t.scrollspy(t.data())})})}(window.jQuery),!function(e){"use strict";var t=function(t){this.element=e(t)};t.prototype={constructo
r:t,show:function(){var t=this.element,n=t.closest("ul:not(.dropdown-menu)"),r=t.attr("data-target"),i,s,o;r||(r=t.attr("href"),r=r&&r.replace(/.*(?=#[^\s]*$)/,""));if(t.parent("li").hasClass("active"))return;i=n.find(".active a").last()[0],o=e.Event("show",{relatedTarget:i}),t.trigger(o);if(o.isDefaultPrevented())return;s=e(r),this.activate(t.parent("li"),n),this.activate(s,s.parent(),function(){t.trigger({type:"shown",relatedTarget:i})})},activate:function(t,n,r){function o(){i.removeClass("active").find("> .dropdown-menu > .active").removeClass("active"),t.addClass("active"),s?(t[0].offsetWidth,t.addClass("in")):t.removeClass("fade"),t.parent(".dropdown-menu")&&t.closest("li.dropdown").addClass("active"),r&&r()}var i=n.find("> .active"),s=r&&e.support.transition&&i.hasClass("fade");s?i.one(e.support.transition.end,o):o(),i.removeClass("in")}},e.fn.tab=function(n){return this.each(function(){var r=e(this),i=r.data("tab");i||r.data("tab",i=new t(this)),typeof n=="string"&&i[n]()})}
,e.fn.tab.Constructor=t,e(function(){e("body").on("click.tab.data-api",'[data-toggle="tab"], [data-toggle="pill"]',function(t){t.preventDefault(),e(this).tab("show")})})}(window.jQuery),!function(e){"use strict";var t=function(t,n){this.$element=e(t),this.options=e.extend({},e.fn.typeahead.defaults,n),this.matcher=this.options.matcher||this.matcher,this.sorter=this.options.sorter||this.sorter,this.highlighter=this.options.highlighter||this.highlighter,this.updater=this.options.updater||this.updater,this.$menu=e(this.options.menu).appendTo("body"),this.source=this.options.source,this.shown=!1,this.listen()};t.prototype={constructor:t,select:function(){var e=this.$menu.find(".active").attr("data-value");return this.$element.val(this.updater(e)).change(),this.hide()},updater:function(e){return e},show:function(){var t=e.extend({},this.$element.offset(),{height:this.$element[0].offsetHeight});return this.$menu.css({top:t.top+t.height,left:t.left}),this.$menu.show(),this.shown=!0,this},h
ide:function(){return this.$menu.hide(),this.shown=!1,this},lookup:function(t){var n;return this.query=this.$element.val(),!this.query||this.query.length<this.options.minLength?this.shown?this.hide():this:(n=e.isFunction(this.source)?this.source(this.query,e.proxy(this.process,this)):this.source,n?this.process(n):this)},process:function(t){var n=this;return t=e.grep(t,function(e){return n.matcher(e)}),t=this.sorter(t),t.length?this.render(t.slice(0,this.options.items)).show():this.shown?this.hide():this},matcher:function(e){return~e.toLowerCase().indexOf(this.query.toLowerCase())},sorter:function(e){var t=[],n=[],r=[],i;while(i=e.shift())i.toLowerCase().indexOf(this.query.toLowerCase())?~i.indexOf(this.query)?n.push(i):r.push(i):t.push(i);return t.concat(n,r)},highlighter:function(e){var t=this.query.replace(/[\-\[\]{}()*+?.,\\\^$|#\s]/g,"\\$&");return e.replace(new RegExp("("+t+")","ig"),function(e,t){return"<strong>"+t+"</strong>"})},render:function(t){var n=this;return t=e(t).map
(function(t,r){return t=e(n.options.item).attr("data-value",r),t.find("a").html(n.highlighter(r)),t[0]}),t.first().addClass("active"),this.$menu.html(t),this},next:function(t){var n=this.$menu.find(".active").removeClass("active"),r=n.next();r.length||(r=e(this.$menu.find("li")[0])),r.addClass("active")},prev:function(e){var t=this.$menu.find(".active").removeClass("active"),n=t.prev();n.length||(n=this.$menu.find("li").last()),n.addClass("active")},listen:function(){this.$element.on("blur",e.proxy(this.blur,this)).on("keypress",e.proxy(this.keypress,this)).on("keyup",e.proxy(this.keyup,this)),(e.browser.webkit||e.browser.msie)&&this.$element.on("keydown",e.proxy(this.keydown,this)),this.$menu.on("click",e.proxy(this.click,this)).on("mouseenter","li",e.proxy(this.mouseenter,this))},move:function(e){if(!this.shown)return;switch(e.keyCode){case 9:case 13:case 27:e.preventDefault();break;case 38:e.preventDefault(),this.prev();break;case 40:e.preventDefault(),this.next()}e.stopPropagati
on()},keydown:function(t){this.suppressKeyPressRepeat=!~e.inArray(t.keyCode,[40,38,9,13,27]),this.move(t)},keypress:function(e){if(this.suppressKeyPressRepeat)return;this.move(e)},keyup:function(e){switch(e.keyCode){case 40:case 38:break;case 9:case 13:if(!this.shown)return;this.select();break;case 27:if(!this.shown)return;this.hide();break;default:this.lookup()}e.stopPropagation(),e.preventDefault()},blur:function(e){var t=this;setTimeout(function(){t.hide()},150)},click:function(e){e.stopPropagation(),e.preventDefault(),this.select()},mouseenter:function(t){this.$menu.find(".active").removeClass("active"),e(t.currentTarget).addClass("active")}},e.fn.typeahead=function(n){return this.each(function(){var r=e(this),i=r.data("typeahead"),s=typeof n=="object"&&n;i||r.data("typeahead",i=new t(this,s)),typeof n=="string"&&i[n]()})},e.fn.typeahead.defaults={source:[],items:8,menu:'<ul class="typeahead dropdown-menu"></ul>',item:'<li><a href="#"></a></li>',minLength:1},e.fn.typeahead.Const
ructor=t,e(function(){e("body").on("focus.typeahead.data-api",'[data-provide="typeahead"]',function(t){var n=e(this);if(n.data("typeahead"))return;t.preventDefault(),n.typeahead(n.data())})})}(window.jQuery),!function(e){"use strict";var t=function(t,n){this.options=e.extend({},e.fn.affix.defaults,n),this.$window=e(window).on("scroll.affix.data-api",e.proxy(this.checkPosition,this)),this.$element=e(t),this.checkPosition()};t.prototype.checkPosition=function(){if(!this.$element.is(":visible"))return;var t=e(document).height(),n=this.$window.scrollTop(),r=this.$element.offset(),i=this.options.offset,s=i.bottom,o=i.top,u="affix affix-top affix-bottom",a;typeof i!="object"&&(s=o=i),typeof o=="function"&&(o=i.top()),typeof s=="function"&&(s=i.bottom()),a=this.unpin!=null&&n+this.unpin<=r.top?!1:s!=null&&r.top+this.$element.height()>=t-s?"bottom":o!=null&&n<=o?"top":!1;if(this.affixed===a)return;this.affixed=a,this.unpin=a=="bottom"?r.top-n:null,this.$element.removeClass(u).addClass("affi
x"+(a?"-"+a:""))},e.fn.affix=function(n){return this.each(function(){var r=e(this),i=r.data("affix"),s=typeof n=="object"&&n;i||r.data("affix",i=new t(this,s)),typeof n=="string"&&i[n]()})},e.fn.affix.Constructor=t,e.fn.affix.defaults={offset:0},e(window).on("load",function(){e('[data-spy="affix"]').each(function(){var t=e(this),n=t.data();n.offset=n.offset||{},n.offsetBottom&&(n.offset.bottom=n.offsetBottom),n.offsetTop&&(n.offset.top=n.offsetTop),t.affix(n)})})}(window.jQuery);
\ No newline at end of file
[17/47] incubator-systemml git commit: initial port of documentation
to md, algorithms reference converted to md, template based on spark docs
Posted by du...@apache.org.
initial port of documentation to md, algorithms reference converted to md, template based on spark docs
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/d47e76df
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/d47e76df
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/d47e76df
Branch: refs/heads/gh-pages
Commit: d47e76df39ac361809b6fb7bf0671a0520d2f849
Parents: 9bb58c6
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Fri Aug 21 15:13:55 2015 -0700
Committer: Deron Eriksson <de...@us.ibm.com>
Committed: Fri Aug 21 15:13:55 2015 -0700
----------------------------------------------------------------------
README.md | 21 +
_config.yml | 16 +
_layouts/global.html | 95 +
algorithms-bibliography.md | 93 +
algorithms-classification.md | 739 +++
algorithms-clustering.md | 479 ++
algorithms-descriptive-statistics.md | 1321 ++++
algorithms-matrix-factorization.md | 143 +
algorithms-reference.md | 30 +
algorithms-regression.md | 1404 +++++
css/bootstrap-responsive.css | 1040 ++++
css/bootstrap-responsive.min.css | 9 +
css/bootstrap.css | 5624 ++++++++++++++++++
css/bootstrap.min.css | 9 +
css/main.css | 172 +
css/pygments-default.css | 76 +
...-quartiles-median-and-interquartile-mean.png | Bin 0 -> 26477 bytes
index.md | 22 +
js/main.js | 97 +
js/vendor/anchor.min.js | 6 +
js/vendor/bootstrap.js | 2027 +++++++
js/vendor/bootstrap.min.js | 6 +
js/vendor/jquery-1.8.0.min.js | 2 +
js/vendor/modernizr-2.6.1-respond-1.1.0.min.js | 11 +
24 files changed, 13442 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..0474f00
--- /dev/null
+++ b/README.md
@@ -0,0 +1,21 @@
+# SystemML Documentation
+
+The primary SystemML documentation is written in markdown (*.md) and can be converted to HTML using
+[Jekyll](http://jekyllrb.com).
+
+Jekyll (and optionally Pygments) can be installed on the Mac OS in the following manner.
+
+ $ brew install ruby
+ $ gem install jekyll
+ $ gem install jekyll-redirect-from
+ $ brew install python
+ $ pip install Pygments
+
+For installation on other platforms, please consult the Jekyll documentation.
+
+To generate SystemML documentation in HTML, navigate to the ```docs``` folder, the root directory of the
+documentation. From there, you can have Jekyll convert the markdown files to HTML. If you run in server mode,
+Jekyll will serve up the generated documentation by default at http://127.0.0.1:4000. Modifications
+to *.md files will be converted to HTML and can be viewed in a web browser.
+
+ $ jekyll serve -w
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/_config.yml
----------------------------------------------------------------------
diff --git a/_config.yml b/_config.yml
new file mode 100644
index 0000000..9ea5762
--- /dev/null
+++ b/_config.yml
@@ -0,0 +1,16 @@
+highlighter: pygments
+markdown: kramdown
+gems:
+ - jekyll-redirect-from
+
+# For some reason kramdown seems to behave differently on different
+# OS/packages wrt encoding. So we hard code this config.
+kramdown:
+ entity_output: numeric
+
+include:
+ - _static
+ - _modules
+
+# These allow the documentation to be updated with newer releases
+SYSTEMML_VERSION: 5.2
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/_layouts/global.html
----------------------------------------------------------------------
diff --git a/_layouts/global.html b/_layouts/global.html
new file mode 100644
index 0000000..73551a4
--- /dev/null
+++ b/_layouts/global.html
@@ -0,0 +1,95 @@
+<!DOCTYPE html>
+<!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]-->
+<!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]-->
+<!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]-->
+<!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]-->
+ <head>
+ <title>{{ page.title }} - SystemML {{site.SYSTEMML_VERSION}}</title>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
+ {% if page.description %}
+ <meta name="description" content="{{page.description | replace: 'SYSTEMML_VERSION', site.SYSTEMML_VERSION}}">
+ {% endif %}
+ <meta name="viewport" content="width=device-width">
+ <link rel="stylesheet" href="css/bootstrap.min.css">
+ <link rel="stylesheet" href="css/bootstrap-responsive.min.css">
+ <link rel="stylesheet" href="css/main.css">
+ <link rel="stylesheet" href="css/pygments-default.css">
+ <script src="js/vendor/modernizr-2.6.1-respond-1.1.0.min.js"></script>
+ </head>
+ <body>
+ <!--[if lt IE 7]>
+ <p class="chromeframe">You are using an outdated browser. <a href="http://browsehappy.com/">Upgrade your browser today</a> or <a href="http://www.google.com/chromeframe/?redirect=true">install Google Chrome Frame</a> to better experience this site.</p>
+ <![endif]-->
+
+ <div class="navbar navbar-fixed-top" id="topbar">
+ <div class="navbar-inner">
+ <div class="container">
+ <div class="brand" style="padding: 15px 0px; font-size: 20px; font-style: italic; font-weight: bold;"><a href="index.html">SystemML - {{site.SYSTEMML_VERSION}}</a>
+ </div>
+ <ul class="nav">
+ <li><a href="index.html">Home</a></li>
+
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentation<b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li><a href="http://www.github.com/SparkTC/systemml">SystemML GitHub README</a></li>
+ <li><a href="algorithms-reference.html">Algorithms Reference</a></li>
+ <!-- <li><a href="dml-language-reference.html">DML Language Reference</a></li> -->
+ <li class="divider"></li>
+ <li> More Coming Soon...</li>
+ </ul>
+ </li>
+
+ </ul>
+ </div>
+ </div>
+ </div>
+
+ <div class="container" id="content">
+ {% if page.displayTitle %}
+ <h1 class="title">{{ page.displayTitle }}</h1>
+ {% else %}
+ <h1 class="title">{{ page.title }}</h1>
+ {% endif %}
+
+ {{ content }}
+
+ </div> <!-- /container -->
+
+ <script src="js/vendor/jquery-1.8.0.min.js"></script>
+ <script src="js/vendor/bootstrap.min.js"></script>
+ <script src="js/vendor/anchor.min.js"></script>
+ <script src="js/main.js"></script>
+
+ <!-- MathJax Section -->
+ <script type="text/x-mathjax-config">
+ MathJax.Hub.Config({
+ TeX: { equationNumbers: { autoNumber: "AMS" } }
+ });
+ </script>
+ <script>
+ // Note that we load MathJax this way to work with local file (file://), HTTP and HTTPS.
+ // We could use "//cdn.mathjax...", but that won't support "file://".
+ (function(d, script) {
+ script = d.createElement('script');
+ script.type = 'text/javascript';
+ script.async = true;
+ script.onload = function(){
+ MathJax.Hub.Config({
+ tex2jax: {
+ inlineMath: [ ["$", "$"], ["\\\\(","\\\\)"] ],
+ displayMath: [ ["$$","$$"], ["\\[", "\\]"] ],
+ processEscapes: true,
+ skipTags: ['script', 'noscript', 'style', 'textarea', 'pre']
+ }
+ });
+ };
+ script.src = ('https:' == document.location.protocol ? 'https://' : 'http://') +
+ 'cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML';
+ d.getElementsByTagName('head')[0].appendChild(script);
+ }(document));
+ </script>
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/algorithms-bibliography.md
----------------------------------------------------------------------
diff --git a/algorithms-bibliography.md b/algorithms-bibliography.md
new file mode 100644
index 0000000..c583dc6
--- /dev/null
+++ b/algorithms-bibliography.md
@@ -0,0 +1,93 @@
+---
+layout: global
+title: SystemML Algorithms Reference - Bibliography
+displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
+---
+
+AcockStavig1979, Alan C. Acock and Gordon
+R. Stavig, A Measure of Association for Nonparametric
+Statistics, Social Forces, Oxford University
+Press, Volume 57, Number 4, June, 1979,
+1381--1386.
+
+AgrawalKSX2002, Rakesh Agrawal and
+Jerry Kiernan and Ramakrishnan Srikant and Yirong Xu,
+Hippocratic Databases, Proceedings of the 28-th
+International Conference on Very Large Data Bases (VLDB 2002),
+Hong Kong, China, August 20--23, 2002,
+143--154.
+
+Agresti2002, Alan Agresti, Categorical
+Data Analysis, Second Edition, Wiley Series in
+Probability and Statistics, Wiley-Interscience
+2002, 710.
+
+AloiseDHP2009, Daniel Aloise and Amit
+Deshpande and Pierre Hansen and Preyas Popat, NP-hardness of
+Euclidean Sum-of-squares Clustering, Machine Learning,
+Kluwer Academic Publishers, Volume 75, Number 2,
+May, 2009, 245--248.
+
+ArthurVassilvitskii2007,
+k-means++: The Advantages of Careful Seeding, David
+Arthur and Sergei Vassilvitskii, Proceedings of the 18th
+Annual ACM-SIAM Symposium on Discrete Algorithms (SODA 2007),
+January 7--9, 2007, New Orleans, LA,
+USA, 1027--1035.
+
+Chapelle2007, Olivier Chapelle, Training a Support Vector Machine in the Primal, Neural Computation, 2007.
+
+Cochran1954, William G. Cochran,
+Some Methods for Strengthening the Common $\chi^2$ Tests,
+Biometrics, Volume 10, Number 4, December
+1954, 417--451.
+
+Gill2000, Jeff Gill, Generalized Linear
+Models: A Unified Approach, Sage University Papers Series on
+Quantitative Applications in the Social Sciences, Number 07-134,
+2000, Sage Publications, 101.
+
+Hartigan1975, John A. Hartigan,
+Clustering Algorithms, John Wiley~&~Sons Inc.,
+Probability and Mathematical Statistics, April
+1975, 365.
+
+Hsieh2008, C-J Hsieh, K-W Chang, C-J Lin, S. S. Keerthi and S. Sundararajan, A Dual Coordinate Descent Method for Large-scale Linear SVM, International Conference of Machine Learning (ICML), 2008.
+
+Lin2008, Chih-Jen Lin and Ruby C. Weng and
+S. Sathiya Keerthi, Trust Region Newton Method for
+Large-Scale Logistic Regression, Journal of Machine Learning
+Research, April, 2008, Volume 9, 627--650.
+
+McCallum1998, A. McCallum and K. Nigam, A comparison of event models for naive bayes text classification, AAAI-98 workshop on learning for text categorization, 1998.
+
+McCullagh1989, Peter McCullagh and John Ashworth
+Nelder, Generalized Linear Models, Second Edition,
+Monographs on Statistics and Applied Probability, Number 37,
+1989, Chapman & Hall/CRC, 532.
+
+Nelder1972, John Ashworth Nelder and Robert
+William Maclagan Wedderburn, Generalized Linear Models,
+Journal of the Royal Statistical Society, Series A
+(General), 1972, Volume 135, Number 3,
+370--384.
+
+Nocedal1999, J. Nocedal and S. J. Wright, Numerical Optimization, Springer-Verlag, 1999.
+
+Nocedal2006, Optimization Numerical Optimization,
+Jorge Nocedal and Stephen Wright, Springer Series
+in Operations Research and Financial Engineering, 664,
+Second Edition, Springer, 2006.
+
+Russell2009, S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, Prentice Hall, 2009.
+
+Scholkopf1995, B. Scholkopf, C. Burges and V. Vapnik, Extracting Support Data for a Given Task, International Conference on Knowledge Discovery and Data Mining (ICDM), 1995.
+
+Stevens1946, Stanley Smith Stevens,
+On the Theory of Scales of Measurement, Science
+June 7, 1946, Volume 103, Number 2684,
+677--680.
+
+Vetterling1992,
+W. T. Vetterling and B. P. Flannery,
+Multidimensions in Numerical Recipes in C - The Art in Scientific Computing, W. H. Press and S. A. Teukolsky (eds.), Cambridge University Press, 1992.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/algorithms-classification.md
----------------------------------------------------------------------
diff --git a/algorithms-classification.md b/algorithms-classification.md
new file mode 100644
index 0000000..da46ded
--- /dev/null
+++ b/algorithms-classification.md
@@ -0,0 +1,739 @@
+---
+layout: global
+title: SystemML Algorithms Reference - Classification
+displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
+---
+
+
+# 2. Classification
+
+
+## 2.1. Multinomial Logistic Regression
+
+### Description
+
+The `MultiLogReg.dml` script performs both binomial and multinomial
+logistic regression. The script is given a dataset $(X, Y)$ where matrix
+$X$ has $m$ columns and matrix $Y$ has one column; both $X$ and $Y$ have
+$n$ rows. The rows of $X$ and $Y$ are viewed as a collection of records:
+$(X, Y) = (x_i, y_i)_{i=1}^n$ where $x_i$ is a numerical vector of
+explanatory (feature) variables and $y_i$ is a categorical response
+variable. Each row $x_i$ in $X$ has size $\dim x_i = m$, while its corresponding $y_i$
+is an integer that represents the observed response value for
+record $i$.
+
+The goal of logistic regression is to learn a linear model over the
+feature vector $x_i$ that can be used to predict how likely each
+categorical label is expected to be observed as the actual $y_i$. Note
+that logistic regression predicts more than a label: it predicts the
+probability for every possible label. The binomial case allows only two
+possible labels, the multinomial case has no such restriction.
+
+Just as linear regression estimates the mean value $\mu_i$ of a
+numerical response variable, logistic regression does the same for
+category label probabilities. In linear regression, the mean of $y_i$ is
+estimated as a linear combination of the features:
+$\mu_i = \beta_0 + \beta_1 x_{i,1} + \ldots + \beta_m x_{i,m} = \beta_0 + x_i\beta_{1:m}$.
+In logistic regression, the label probability has to lie between 0
+and 1, so a link function is applied to connect it to
+$\beta_0 + x_i\beta_{1:m}$. If there are just two possible category
+labels, for example 0 and 1, the logistic link looks as follows:
+
+$$Prob[y_i\,{=}\,1\mid x_i; \beta] \,=\,
+\frac{e^{\,\beta_0 + x_i\beta_{1:m}}}{1 + e^{\,\beta_0 + x_i\beta_{1:m}}};
+\quad
+Prob[y_i\,{=}\,0\mid x_i; \beta] \,=\,
+\frac{1}{1 + e^{\,\beta_0 + x_i\beta_{1:m}}}$$
+
+Here category label 0
+serves as the *baseline*, and function $\exp(\beta_0 + x_i\beta_{1:m})$
+shows how likely we expect to see “$y_i = 1$” in comparison to the
+baseline. Like in a loaded coin, the predicted odds of seeing 1 versus 0
+are $\exp(\beta_0 + x_i\beta_{1:m})$ to 1, with each feature $x_{i,j}$
+multiplying its own factor $\exp(\beta_j x_{i,j})$ to the odds. Given a
+large collection of pairs $(x_i, y_i)$, $i=1\ldots n$, logistic
+regression seeks to find the $\beta_j$’s that maximize the product of
+probabilities $Prob[y_i\mid x_i; \beta]$
+for actually observed $y_i$-labels (assuming no
+regularization).
+
+Multinomial logistic regression [[Agresti2002]](algorithms-bibliography.html)
+extends this link to
+$k \geq 3$ possible categories. Again we identify one category as the
+baseline, for example the $k$-th category. Instead of a coin, here we
+have a loaded multisided die, one side per category. Each non-baseline
+category $l = 1\ldots k\,{-}\,1$ has its own vector
+$(\beta_{0,l}, \beta_{1,l}, \ldots, \beta_{m,l})$ of regression
+parameters with the intercept, making up a matrix $B$ of size
+$(m\,{+}\,1)\times(k\,{-}\,1)$. The predicted odds of seeing
+non-baseline category $l$ versus the baseline $k$ are
+$\exp\big(\beta_{0,l} + \sum\nolimits_{j=1}^m x_{i,j}\beta_{j,l}\big)$
+to 1, and the predicted probabilities are:
+
+$$
+\begin{equation}
+l < k: Prob [y_i {=} l \mid x_i; B] \,\,\,{=}\,\,\,
+\frac{\exp\big(\beta_{0,l} + \sum\nolimits_{j=1}^m x_{i,j}\beta_{j,l}\big)}{1 \,+\, \sum_{l'=1}^{k-1}\exp\big(\beta_{0,l'} + \sum\nolimits_{j=1}^m x_{i,j}\beta_{j,l'}\big)};
+\end{equation}
+$$
+
+$$
+\begin{equation}
+Prob [y_i {=} k \mid x_i; B] \,\,\,{=}\,\,\,
+\frac{1}{1 \,+\, \sum_{l'=1}^{k-1}\exp\big(\beta_{0,l'} + \sum\nolimits_{j=1}^m x_{i,j}\beta_{j,l'}\big)}.
+\end{equation}
+$$
+
+The goal of the regression
+is to estimate the parameter matrix $B$ from the provided dataset
+$(X, Y) = (x_i, y_i)_{i=1}^n$ by maximizing the product of over the
+observed labels $y_i$. Taking its logarithm, negating, and adding a
+regularization term gives us a minimization objective:
+
+$$
+\begin{equation}
+f(B; X, Y) \,\,=\,\,
+-\sum_{i=1}^n \,\log Prob[y_i\mid x_i; B] \,+\,
+\frac{\lambda}{2} \sum_{j=1}^m \sum_{l=1}^{k-1} |\beta_{j,l}|^2
+\,\,\to\,\,\min
+\end{equation}
+$$
+
+The optional regularization term is added to
+mitigate overfitting and degeneracy in the data; to reduce bias, the
+intercepts $\beta_{0,l}$ are not regularized. Once the $\beta_{j,l}$’s
+are accurately estimated, we can make predictions about the category
+label $y$ for a new feature vector $x$ using
+Eqs. (1) and (2).
+
+
+### Usage
+
+ hadoop jar SystemML.jar -f MultiLogReg.dml
+ -nvargs X=file
+ Y=file
+ B=file
+ Log=file
+ icpt=int
+ reg=double
+ tol=double
+ moi=int
+ mii=int
+ fmt=format
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read the input matrix of feature vectors; each row
+constitutes one feature vector.
+
+**Y**: Location to read the input one-column matrix of category labels that
+correspond to feature vectors in X. Note the following:\
+– Each non-baseline category label must be a positive integer.\
+– If all labels are positive, the largest represents the baseline
+category.\
+– If non-positive labels such as $-1$ or $0$ are present, then they
+represent the (same) baseline category and are converted to label
+$\max(\texttt{Y})\,{+}\,1$.
+
+**B**: Location to store the matrix of estimated regression parameters (the
+$\beta_{j, l}$’s), with the intercept parameters $\beta_{0, l}$ at
+position B\[$m\,{+}\,1$, $l$\] if available.
+The size of B is $(m\,{+}\,1)\times (k\,{-}\,1)$ with the
+intercepts or $m \times (k\,{-}\,1)$ without the intercepts, one column
+per non-baseline category and one row per feature.
+
+**Log**: (default: " ") Location to store iteration-specific variables for monitoring
+and debugging purposes, see
+[**Table 5**](algorithms-classification.html#table5)
+for details.
+
+**icpt**: (default: 0) Intercept and shifting/rescaling of the features in $X$:\
+0 = no intercept (hence no $\beta_0$), no
+shifting/rescaling of the features;\
+1 = add intercept, but do not shift/rescale the features
+in $X$;\
+2 = add intercept, shift/rescale the features in $X$ to
+mean 0, variance 1
+
+**reg**: (default: 0.0) L2-regularization parameter (lambda)
+
+**tol**: (default: 0.000001) Tolerance (epsilon) used in the convergence criterion
+
+**moi**: (default: 100) Maximum number of outer (Fisher scoring) iterations
+
+**mii**: (default: 0) Maximum number of inner (conjugate gradient) iterations, or 0
+if no maximum limit provided
+
+**fmt**: (default: `"text"`) Matrix file output format, such as text,
+mm, or csv; see read/write functions in
+SystemML Language Reference for details.
+
+
+### Examples
+
+ hadoop jar SystemML.jar -f MultiLogReg.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ fmt=csv
+ icpt=2
+ reg=1.0
+ tol=0.0001
+ moi=100
+ mii=10
+ Log=/user/ml/log.csv
+
+
+* * *
+
+<a name="table5" />
+**Table 5**: The `Log` file for multinomial logistic regression
+contains the following iteration variables in CSV format, each line
+containing triple (Name, Iteration\#, Value) with Iteration\# being 0
+for initial values.
+
+
+| Name | Meaning |
+| ------------------- | -------------------------- |
+| LINEAR\_TERM\_MIN | The minimum value of $X$ %*% $B$, used to check for overflows |
+| LINEAR\_TERM\_MAX | The maximum value of $X$ %*% $B$, used to check for overflows |
+| NUM\_CG\_ITERS | Number of inner (Conj. Gradient) iterations in this outer iteration |
+| IS\_TRUST\_REACHED | $1 = {}$trust region boundary was reached, $0 = {}$otherwise |
+| POINT\_STEP\_NORM | L2-norm of iteration step from old point (matrix $B$) to new point |
+| OBJECTIVE | The loss function we minimize (negative regularized log-likelihood) |
+| OBJ\_DROP\_REAL | Reduction in the objective during this iteration, actual value |
+| OBJ\_DROP\_PRED | Reduction in the objective predicted by a quadratic approximation |
+| OBJ\_DROP\_RATIO | Actual-to-predicted reduction ratio, used to update the trust region |
+| IS\_POINT\_UPDATED | $1 = {}$new point accepted; $0 = {}$new point rejected, old point restored |
+| GRADIENT\_NORM | L2-norm of the loss function gradient (omitted if point is rejected) |
+| RUST\_DELTA | Updated trust region size, the “delta” |
+
+
+* * *
+
+### Details
+
+We estimate the logistic regression parameters via L2-regularized
+negative log-likelihood minimization (3). The
+optimization method used in the script closely follows the trust region
+Newton method for logistic regression described in [[Lin2008]](algorithms-bibliography.html).
+For convenience, let us make some changes in notation:
+
+ * Convert the input vector of observed category labels into an indicator
+matrix $Y$ of size $n \times k$ such that $Y_{i, l} = 1$ if the $i$-th
+category label is $l$ and $Y_{i, l} = 0$ otherwise.
+ * Append an extra column of all ones, i.e. $(1, 1, \ldots, 1)^T$, as the
+$m\,{+}\,1$-st column to the feature matrix $X$ to represent the
+intercept.
+ * Append an all-zero column as the $k$-th column to $B$, the matrix of
+regression parameters, to represent the baseline category.
+ * Convert the regularization constant $\lambda$ into matrix $\Lambda$ of
+the same size as $B$, placing 0’s into the $m\,{+}\,1$-st row to disable
+intercept regularization, and placing $\lambda$’s everywhere else.
+
+Now the ($n\,{\times}\,k$)-matrix of predicted probabilities given by
+(1) and (2) and the
+objective function $f$ in (3) have the matrix form
+
+$$\begin{aligned}
+P \,\,&=\,\, \exp(XB) \,\,/\,\, \big(\exp(XB)\,1_{k\times k}\big)\\
+f \,\,&=\,\, - \,\,{\textstyle\sum} \,\,Y \cdot (X B)\, + \,
+{\textstyle\sum}\,\log\big(\exp(XB)\,1_{k\times 1}\big) \,+ \,
+(1/2)\,\, {\textstyle\sum} \,\,\Lambda \cdot B \cdot B\end{aligned}$$
+
+where operations $\cdot\,$, $/$, $\exp$, and $\log$ are applied
+cellwise, and $\textstyle\sum$ denotes the sum of all cells in a matrix.
+The gradient of $f$ with respect to $B$ can be represented as a matrix
+too:
+
+$$\nabla f \,\,=\,\, X^T (P - Y) \,+\, \Lambda \cdot B$$
+
+The Hessian $\mathcal{H}$ of $f$ is a tensor, but, fortunately, the
+conjugate gradient inner loop of the trust region algorithm
+in [[Lin2008]](algorithms-bibliography.html)
+does not need to instantiate it. We only need to
+multiply $\mathcal{H}$ by ordinary matrices of the same size as $B$ and
+$\nabla f$, and this can be done in matrix form:
+
+$$\mathcal{H}V \,\,=\,\, X^T \big( Q \,-\, P \cdot (Q\,1_{k\times k}) \big) \,+\,
+\Lambda \cdot V, \,\,\,\,\textrm{where}\,\,\,\,Q \,=\, P \cdot (XV)$$
+
+At each Newton iteration (the *outer* iteration) the minimization algorithm
+approximates the difference
+$\varDelta f(S; B) = f(B + S; X, Y) \,-\, f(B; X, Y)$ attained in the
+objective function after a step $B \mapsto B\,{+}\,S$ by a second-degree
+formula
+
+$$\varDelta f(S; B) \,\,\,\approx\,\,\, (1/2)\,\,{\textstyle\sum}\,\,S \cdot \mathcal{H}S
+ \,+\, {\textstyle\sum}\,\,S\cdot \nabla f$$
+
+This approximation is then
+minimized by trust-region conjugate gradient iterations (the *inner*
+iterations) subject to the constraint
+$\|S\|_2 \leq \delta$
+. The trust
+region size $\delta$ is initialized as
+$0.5\sqrt{m}\,/ \max_i \|x_i\|_2$
+and updated as described
+in [[Lin2008]](algorithms-bibliography.html).
+Users can specify the maximum number of the outer
+and the inner iterations with input parameters moi and
+mii, respectively. The iterative minimizer terminates
+successfully if
+$$\|\nabla f\|_2 < \varepsilon \|\nabla f_{B=0} \|_2$$
+, where ${\varepsilon}> 0$ is a tolerance supplied by the user via input
+parameter tol.
+
+
+### Returns
+
+The estimated regression parameters (the
+$$\hat{\beta}_{j, l}$$)
+are
+populated into a matrix and written to an HDFS file whose path/name was
+provided as the “B” input argument. Only the non-baseline
+categories ($1\leq l \leq k\,{-}\,1$) have their
+$$\hat{\beta}_{j, l}$$
+in the output; to add the baseline category, just append a column of zeros.
+If icpt=0 in the input command line, no intercepts are used
+and B has size
+$m\times (k\,{-}\,1)$; otherwise
+B has size
+$(m\,{+}\,1)\times (k\,{-}\,1)$
+and the
+intercepts are in the
+$m\,{+}\,1$-st row. If icpt=2, then
+initially the feature columns in $X$ are shifted to mean${} = 0$ and
+rescaled to variance${} = 1$. After the iterations converge, the
+$\hat{\beta}_{j, l}$’s are rescaled and shifted to work with the
+original features.
+
+
+* * *
+
+## 2.2 Support Vector Machines
+
+### 2.2.1 Binary-Class Support Vector Machines
+
+#### Description
+
+Support Vector Machines are used to model the relationship between a
+categorical dependent variable y and one or more explanatory variables
+denoted X. This implementation learns (and predicts with) a binary class
+support vector machine (y with domain size 2).
+
+
+#### Usage
+
+ hadoop jar SystemML.jar -f l2-svm.dml
+ -nvargs X=file
+ Y=file
+ icpt=int
+ tol=double
+ reg=double
+ maxiter=int
+ model=file
+ Log=file
+ fmt=format
+
+ hadoop jar SystemML.jar -f l2-svm-predict.dml
+ -nvargs X=file
+ Y=file
+ icpt=int
+ model=file
+ scores=file
+ accuracy=file
+ confusion=file
+ fmt=format
+
+
+#### Arguments
+
+**X**: Location (on HDFS) to read the matrix of feature vectors; each
+row constitutes one feature vector.
+
+**Y**: Location to read the one-column matrix of (categorical) labels
+that correspond to feature vectors in X. Binary class labels can be
+expressed in one of two choices: $\pm 1$ or $1/2$. Note that, this
+argument is optional for prediction.
+
+**icpt**: (default: 0) If set to 1 then a constant bias
+column is added to X.
+
+**tol**: (default: 0.001) Procedure terminates early if the
+reduction in objective function value is less than tolerance times
+the initial objective function value.
+
+**reg**: (default: 1) Regularization constant. See details
+to find out where lambda appears in the objective function. If one
+were interested in drawing an analogy with the C parameter in C-SVM,
+then C = 2/lambda. Usually, cross validation is employed to
+determine the optimum value of lambda.
+
+**maxiter**: (default: 100) The maximum number
+of iterations.
+
+**model**: Location (on HDFS) that contains the learnt weights.
+
+**Log**: Location (on HDFS) to collect various metrics (e.g., objective
+function value etc.) that depict progress across iterations
+while training.
+
+**fmt**: (default: `"text"`) Specifies the output format.
+Choice of comma-separated values (csv) or as a sparse-matrix (text).
+
+**scores**: Location (on HDFS) to store scores for a held-out test set.
+Note that, this is an optional argument.
+
+**accuracy**: Location (on HDFS) to store the accuracy computed on a
+held-out test set. Note that, this is an optional argument.
+
+**confusion**: Location (on HDFS) to store the confusion matrix computed
+using a held-out test set. Note that, this is an optional argument.
+
+
+#### Examples
+
+ hadoop jar SystemML.jar -f l2-svm.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ icpt=0
+ tol=0.001
+ fmt=csv
+ reg=1.0
+ maxiter=100
+ model=/user/ml/weights.csv
+ Log=/user/ml/Log.csv
+
+ hadoop jar SystemML.jar -f l2-svm-predict.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ icpt=0
+ fmt=csv
+ model=/user/ml/weights.csv
+ scores=/user/ml/scores.csv
+ accuracy=/user/ml/accuracy.csv
+ confusion=/user/ml/confusion.csv
+
+
+#### Details
+
+Support vector machines learn a classification function by solving the
+following optimization problem ($L_2$-SVM):
+
+$$\begin{aligned}
+&\textrm{argmin}_w& \frac{\lambda}{2} ||w||_2^2 + \sum_i \xi_i^2\\
+&\textrm{subject to:}& y_i w^{\top} x_i \geq 1 - \xi_i ~ \forall i\end{aligned}$$
+
+where $x_i$ is an example from the training set with its label given by
+$y_i$, $w$ is the vector of parameters and $\lambda$ is the
+regularization constant specified by the user.
+
+To account for the missing bias term, one may augment the data with a
+column of constants which is achieved by setting the intercept argument to 1
+[[Hsieh2008]](algorithms-bibliography.html).
+
+This implementation optimizes the primal directly
+[[Chapelle2007]](algorithms-bibliography.html). It
+uses nonlinear conjugate gradient descent to minimize the objective
+function coupled with choosing step-sizes by performing one-dimensional
+Newton minimization in the direction of the gradient.
+
+
+#### Returns
+
+The learnt weights produced by `l2-svm.dml` are populated into a single
+column matrix and written to file on HDFS (see model in section
+Arguments). The number of rows in this matrix is ncol(X) if intercept
+was set to 0 during invocation and ncol(X) + 1 otherwise. The bias term,
+if used, is placed in the last row. Depending on what arguments are
+provided during invocation, `l2-svm-predict.dml` may compute one or more
+of scores, accuracy and confusion matrix in the output format
+specified.
+
+
+* * *
+
+
+### 2.2.2 Multi-Class Support Vector Machines
+
+#### Description
+
+Support Vector Machines are used to model the relationship between a
+categorical dependent variable y and one or more explanatory variables
+denoted X. This implementation supports dependent variables that have
+domain size greater or equal to 2 and hence is not restricted to binary
+class labels.
+
+
+#### Usage
+
+ hadoop jar SystemML.jar -f m-svm.dml
+ -nvargs X=file
+ Y=file
+ icpt=int
+ tol=double
+ reg=double
+ maxiter=int
+ model=file
+ Log=file
+ fmt=format
+
+ hadoop jar SystemML.jar -f m-svm-predict.dml
+ -nvargs X=file
+ Y=file
+ icpt=int
+ model=file
+ scores=file
+ accuracy=file
+ confusion=file
+ fmt=format
+
+
+#### Arguments
+
+**X**: Location (on HDFS) containing the explanatory variables in
+ a matrix. Each row constitutes an example.
+
+**Y**: Location (on HDFS) containing a 1-column matrix specifying the
+ categorical dependent variable (label). Labels are assumed to be
+ contiguously numbered from 1 $\ldots$ \#classes. Note that, this
+ argument is optional for prediction.
+
+**icpt**: (default: 0) If set to 1 then a constant bias
+ column is added to X.
+
+**tol**: (default: 0.001) Procedure terminates early if the
+ reduction in objective function value is less than tolerance times
+ the initial objective function value.
+
+**reg**: (default: 1) Regularization constant. See details
+ to find out where lambda appears in the objective function. If one
+ were interested in drawing an analogy with C-SVM, then C = 2/lambda.
+ Usually, cross validation is employed to determine the optimum value
+ of lambda.
+
+**maxiter**: (default: 100) The maximum number
+ of iterations.
+
+**model**: Location (on HDFS) that contains the learnt weights.
+
+**Log**: Location (on HDFS) to collect various metrics (e.g., objective
+ function value etc.) that depict progress across iterations
+ while training.
+
+**fmt**: (default: `"text"`) Specifies the output format.
+ Choice of comma-separated values (csv) or as a sparse-matrix (text).
+
+**scores**: Location (on HDFS) to store scores for a held-out test set.
+ Note that, this is an optional argument.
+
+**accuracy**: Location (on HDFS) to store the accuracy computed on a
+ held-out test set. Note that, this is an optional argument.
+
+**confusion**: Location (on HDFS) to store the confusion matrix computed
+ using a held-out test set. Note that, this is an optional argument.
+
+
+#### Examples
+
+ hadoop jar SystemML.jar -f m-svm.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ icpt=0
+ tol=0.001
+ reg=1.0
+ maxiter=100
+ fmt=csv
+ model=/user/ml/weights.csv
+ Log=/user/ml/Log.csv
+
+ hadoop jar SystemML.jar -f m-svm-predict.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ icpt=0
+ fmt=csv
+ model=/user/ml/weights.csv
+ scores=/user/ml/scores.csv
+ accuracy=/user/ml/accuracy.csv
+ confusion=/user/ml/confusion.csv
+
+
+#### Details
+
+Support vector machines learn a classification function by solving the
+following optimization problem ($L_2$-SVM):
+
+$$\begin{aligned}
+&\textrm{argmin}_w& \frac{\lambda}{2} ||w||_2^2 + \sum_i \xi_i^2\\
+&\textrm{subject to:}& y_i w^{\top} x_i \geq 1 - \xi_i ~ \forall i\end{aligned}$$
+
+where $x_i$ is an example from the training set with its label given by
+$y_i$, $w$ is the vector of parameters and $\lambda$ is the
+regularization constant specified by the user.
+
+To extend the above formulation (binary class SVM) to the multiclass
+setting, one standard approach is to learn one binary class SVM per
+class that separates data belonging to that class from the rest of the
+training data (one-against-the-rest SVM, see
+[[Scholkopf1995]](algorithms-bibliography.html)).
+
+To account for the missing bias term, one may augment the data with a
+column of constants which is achieved by setting intercept argument to 1
+[[Hsieh2008]](algorithms-bibliography.html).
+
+This implementation optimizes the primal directly
+[[Chapelle2007]](algorithms-bibliography.html). It
+uses nonlinear conjugate gradient descent to minimize the objective
+function coupled with choosing step-sizes by performing one-dimensional
+Newton minimization in the direction of the gradient.
+
+
+#### Returns
+
+The learnt weights produced by `m-svm.dml` are populated into a matrix
+that has as many columns as there are classes in the training data, and
+written to file provided on HDFS (see model in section Arguments). The
+number of rows in this matrix is ncol(X) if intercept was set to 0
+during invocation and ncol(X) + 1 otherwise. The bias terms, if used,
+are placed in the last row. Depending on what arguments are provided
+during invocation, `m-svm-predict.dml` may compute one or more of scores,
+accuracy and confusion matrix in the output format specified.
+
+
+* * *
+
+## 2.3 Naive Bayes
+
+### Description
+
+Naive Bayes is very simple generative model used for classifying data.
+This implementation learns a multinomial naive Bayes classifier which is
+applicable when all features are counts of categorical values.
+
+
+#### Usage
+
+ hadoop jar SystemML.jar -f naive-bayes.dml
+ -nvargs X=file
+ Y=file
+ laplace=double
+ prior=file
+ conditionals=file
+ accuracy=file
+ fmt=format
+
+ hadoop jar SystemML.jar -f naive-bayes-predict.dml
+ -nvargs X=file
+ Y=file
+ prior=file
+ conditionals=file
+ fmt=format
+ accuracy=file
+ confusion=file
+ probabilities=file
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read the matrix of feature vectors; each
+ row constitutes one feature vector.
+
+**Y**: Location (on HDFS) to read the one-column matrix of (categorical)
+ labels that correspond to feature vectors in X. Classes are assumed
+ to be contiguously labeled beginning from 1. Note that, this
+ argument is optional for prediction.
+
+**laplace**: (default: 1) Laplace smoothing specified by
+ the user to avoid creation of 0 probabilities.
+
+**prior**: Location (on HDFS) that contains the class
+ prior probabilites.
+
+**conditionals**: Location (on HDFS) that contains the class conditional
+ feature distributions.
+
+**fmt** (default: `"text"`): Specifies the output format.
+ Choice of comma-separated values (csv) or as a sparse-matrix (text).
+
+**probabilities**: Location (on HDFS) to store class membership
+ probabilities for a held-out test set. Note that, this is an
+ optional argument.
+
+**accuracy**: Location (on HDFS) to store the training accuracy during
+ learning and testing accuracy from a held-out test set
+ during prediction. Note that, this is an optional argument
+ for prediction.
+
+**confusion**: Location (on HDFS) to store the confusion matrix computed
+ using a held-out test set. Note that, this is an optional argument.
+
+
+### Examples
+
+ hadoop jar SystemML.jar -f naive-bayes.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ laplace=1 fmt=csv
+ prior=/user/ml/prior.csv
+ conditionals=/user/ml/conditionals.csv
+ accuracy=/user/ml/accuracy.csv
+
+ hadoop jar SystemML.jar -f naive-bayes-predict.dml
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/y.mtx
+ prior=/user/ml/prior.csv
+ conditionals=/user/ml/conditionals.csv
+ fmt=csv
+ accuracy=/user/ml/accuracy.csv
+ probabilities=/user/ml/probabilities.csv
+ confusion=/user/ml/confusion.csv
+
+
+### Details
+
+Naive Bayes is a very simple generative classification model. It posits
+that given the class label, features can be generated independently of
+each other. More precisely, the (multinomial) naive Bayes model uses the
+following equation to estimate the joint probability of a feature vector
+$x$ belonging to class $y$:
+
+$$\text{Prob}(y, x) = \pi_y \prod_{i \in x} \theta_{iy}^{n(i,x)}$$
+
+where $\pi_y$ denotes the prior probability of class $y$, $i$ denotes a
+feature present in $x$ with $n(i,x)$ denoting its count and
+$\theta_{iy}$ denotes the class conditional probability of feature $i$
+in class $y$. The usual constraints hold on $\pi$ and $\theta$:
+
+$$\begin{aligned}
+&& \pi_y \geq 0, ~ \sum_{y \in \mathcal{C}} \pi_y = 1\\
+\forall y \in \mathcal{C}: && \theta_{iy} \geq 0, ~ \sum_i \theta_{iy} = 1\end{aligned}$$
+
+where $\mathcal{C}$ is the set of classes.
+
+Given a fully labeled training dataset, it is possible to learn a naive
+Bayes model using simple counting (group-by aggregates). To compute the
+class conditional probabilities, it is usually advisable to avoid
+setting $\theta_{iy}$ to 0. One way to achieve this is using additive
+smoothing or Laplace smoothing. Some authors have argued that this
+should in fact be add-one smoothing. This implementation uses add-one
+smoothing by default but lets the user specify her/his own constant, if
+required.
+
+This implementation is sometimes referred to as *multinomial* naive
+Bayes. Other flavours of naive Bayes are also popular.
+
+
+### Returns
+
+The learnt model produced by `naive-bayes.dml` is stored in two separate
+files. The first file stores the class prior (a single-column matrix).
+The second file stores the class conditional probabilities organized
+into a matrix with as many rows as there are class labels and as many
+columns as there are features. Depending on what arguments are provided
+during invocation, `naive-bayes-predict.dml` may compute one or more of
+probabilities, accuracy and confusion matrix in the output format
+specified.
+
+
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/algorithms-clustering.md
----------------------------------------------------------------------
diff --git a/algorithms-clustering.md b/algorithms-clustering.md
new file mode 100644
index 0000000..c84a2ff
--- /dev/null
+++ b/algorithms-clustering.md
@@ -0,0 +1,479 @@
+---
+layout: global
+title: SystemML Algorithms Reference - Clustering
+displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
+---
+
+
+# 3. Clustering
+
+
+## 3.1. K-Means Clustering
+
+### Description
+
+Given a collection of $n$ records with a pairwise similarity measure,
+the goal of clustering is to assign a category label to each record so
+that similar records tend to get the same label. In contrast to
+multinomial logistic regression, clustering is an *unsupervised*
+learning problem with neither category assignments nor label
+interpretations given in advance. In $k$-means clustering, the records
+$x_1, x_2, \ldots, x_n$ are numerical feature vectors of $\dim x_i = m$
+with the squared Euclidean distance $\|x_i - x_{i'}\|_2^2$ as the
+similarity measure. We want to partition $\\{x_1, \ldots, x_n\\}$ into $k$
+clusters $\\{S_1, \ldots, S_k\\}$ so that the aggregated squared distance
+from records to their cluster means is minimized:
+
+$$
+\begin{equation}
+\textrm{WCSS}\,\,=\,\, \sum_{i=1}^n \,\big\|x_i - mean(S_j: x_i\in S_j)\big\|_2^2 \,\,\to\,\,\min
+\end{equation}
+$$
+
+The aggregated distance measure in (1) is
+called the *within-cluster sum of squares* (WCSS). It can be viewed as a
+measure of residual variance that remains in the data after the
+clustering assignment, conceptually similar to the residual sum of
+squares (RSS) in linear regression. However, unlike for the RSS, the
+minimization of (1) is an NP-hard
+problem [[AloiseDHP2009]](algorithms-bibliography.html).
+
+Rather than searching for the global optimum in (1), a
+heuristic algorithm called Lloyd’s algorithm is typically used. This
+iterative algorithm maintains and updates a set of $k$ *centroids*
+$\\{c_1, \ldots, c_k\\}$, one centroid per cluster. It defines each
+cluster $S_j$ as the set of all records closer to $c_j$ than to any
+other centroid. Each iteration of the algorithm reduces the WCSS in two
+steps:
+
+ 1. Assign each record to the closest centroid, making
+$mean(S_j)\neq c_j$
+ 2. Reset each centroid to its cluster’s mean:
+$c_j := mean(S_j)$
+
+After Step 1, the centroids are generally
+different from the cluster means, so we can compute another
+“within-cluster sum of squares” based on the centroids:
+
+$$\textrm{WCSS_C}\,\,=\,\, \sum_{i=1}^n \,\big\|x_i - \mathop{\textrm{centroid}}(S_j: x_i\in S_j)\big\|_2^2
+\label{eqn:WCSS:C}$$
+
+This WCSS\_C after Step 1
+is less than the means-based WCSS before Step 1
+(or equal if convergence achieved), and in Step 2
+the WCSS cannot exceed the WCSS\_C for *the same* clustering; hence the
+WCSS reduction.
+
+Exact convergence is reached when each record becomes closer to its
+cluster’s mean than to any other cluster’s mean, so there are no more
+re-assignments and the centroids coincide with the means. In practice,
+iterations may be stopped when the reduction in WCSS (or in WCSS\_C)
+falls below a minimum threshold, or upon reaching the maximum number of
+iterations. The initialization of the centroids is also an important
+part of the algorithm. The smallest WCSS obtained by the algorithm is
+not the global minimum and varies depending on the initial centroids. We
+implement multiple parallel runs with different initial centroids and
+report the best result.
+
+### Scoring
+
+Our scoring script evaluates the clustering output by comparing it with
+a known category assignment. Since cluster labels have no prior
+correspondence to the categories, we cannot count “correct” and “wrong”
+cluster assignments. Instead, we quantify them in two ways:
+
+ 1. Count how many same-category and different-category pairs of records end
+up in the same cluster or in different clusters;
+ 2. For each category, count the prevalence of its most common cluster; for
+each cluster, count the prevalence of its most common category.
+
+The number of categories and the number of clusters ($k$) do not have to
+be equal. A same-category pair of records clustered into the same
+cluster is viewed as a “true positive,” a different-category pair
+clustered together is a “false positive,” a same-category pair clustered
+apart is a “false negative” etc.
+
+
+### Usage
+
+ hadoop jar SystemML.jar -f Kmeans.dml
+ -nvargs X=file
+ C=file
+ k=int
+ runs=int
+ maxi=int
+ tol=double
+ samp=int
+ isY=int
+ Y=file
+ fmt=format
+ verb=int
+
+ hadoop jar SystemML.jar -f Kmeans-predict.dml
+ -nvargs X=file
+ C=file
+ spY=file
+ prY=file
+ fmt=format
+ O=file
+
+
+
+### Arguments - K-means
+
+**X**: Location to read matrix $X$ with the input data records as rows
+
+**C**: (default: `"C.mtx"`) Location to store the output matrix with the best available
+cluster centroids as rows
+
+**k**: Number of clusters (and centroids)
+
+**runs**: (default: 10) Number of parallel runs, each run with different initial
+centroids
+
+**maxi**: (default: 1000) Maximum number of iterations per run
+
+**tol**: (default: 0.000001) Tolerance (epsilon) for single-iteration WCSS\_C change ratio
+
+**samp**: (default: 50) Average number of records per centroid in data samples used
+in the centroid initialization procedure
+
+**Y**: (default: `"Y.mtx"`) Location to store the one-column matrix $Y$ with the best
+available mapping of records to clusters (defined by the output
+centroids)
+
+**isY**: (default: 0) 0 = do not write matrix $Y$, 1 = write $Y$
+
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
+
+**verb**: (default: 0) 0 = do not print per-iteration statistics for
+each run, 1 = print them (the “verbose” option)
+
+
+### Arguments - K-means Scoring/Prediction
+
+**X**: (default: `" "`) Location to read matrix $X$ with the input data records as
+rows, optional when `prY` input is provided
+
+**C**: (default: `" "`) Location to read matrix $C$ with cluster centroids as rows,
+optional when `prY` input is provided; NOTE: if both
+X and C are provided, `prY` is an
+output, not input
+
+**spY**: (default: `" "`) Location to read a one-column matrix with the externally
+specified “true” assignment of records (rows) to categories, optional
+for prediction without scoring
+
+**prY**: (default: `" "`) Location to read (or write, if X and
+C are present) a column-vector with the predicted
+assignment of rows to clusters; NOTE: No prior correspondence is assumed
+between the predicted cluster labels and the externally specified
+categories
+
+**fmt**: (default: `"text"`) Matrix file output format for `prY`, such as
+`text`, `mm`, or `csv`; see read/write
+functions in SystemML Language Reference for details
+
+**0**: (default: `" "`) Location to write the output statistics defined in
+[**Table 6**](algorithms-clustering.html#table6), by default print them to the
+standard output
+
+
+### Examples
+
+ hadoop jar SystemML.jar -f Kmeans.dml
+ -nvargs X=/user/ml/X.mtx
+ k=5
+ C=/user/ml/centroids.mtx
+ fmt=csv
+
+ hadoop jar SystemML.jar -f Kmeans.dml
+ -nvargs X=/user/ml/X.mtx
+ k=5
+ runs=100
+ maxi=5000
+ tol=0.00000001
+ samp=20
+ C=/user/ml/centroids.mtx
+ isY=1
+ Y=/user/ml/Yout.mtx
+ verb=1
+
+To predict Y given X and C:
+
+ hadoop jar SystemML.jar -f Kmeans-predict.dml
+ -nvargs X=/user/ml/X.mtx
+ C=/user/ml/C.mtx
+ prY=/user/ml/PredY.mtx
+ O=/user/ml/stats.csv
+
+To compare “actual” labels `spY` with “predicted” labels
+given X and C:
+
+ hadoop jar SystemML.jar -f Kmeans-predict.dml
+ -nvargs X=/user/ml/X.mtx
+ C=/user/ml/C.mtx
+ spY=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+
+To compare “actual” labels `spY` with given “predicted”
+labels prY:
+
+ hadoop jar SystemML.jar -f Kmeans-predict.dml
+ -nvargs spY=/user/ml/Y.mtx
+ prY=/user/ml/PredY.mtx
+ O=/user/ml/stats.csv
+
+
+* * *
+
+<a name="table6" />
+**Table 6**: The O-file for Kmeans-predict provides the
+output statistics in CSV format, one per line, in the following
+format: (NAME, \[CID\], VALUE). Note: the 1st group statistics are
+given if X input is available; the 2nd group statistics
+are given if X and C inputs are available;
+the 3rd and 4th group statistics are given if spY input
+is available; only the 4th group statistics contain a nonempty CID
+value; when present, CID contains either the specified category label
+or the predicted cluster label.
+
+<table>
+ <thead>
+ <tr>
+ <th>Inputs Available</th>
+ <th>Name</th>
+ <th>CID</th>
+ <th>Meaning</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td style="text-align: center" rowspan="5">X</td>
+ <td>TSS</td>
+ <td> </td>
+ <td>Total Sum of Squares (from the total mean)</td>
+ </tr>
+ <tr>
+ <td>WCSS_M</td>
+ <td> </td>
+ <td>Within-Cluster Sum of Squares (means as centers)</td>
+ </tr>
+ <tr>
+ <td>WCSS_M_PC</td>
+ <td> </td>
+ <td>Within-Cluster Sum of Squares (means), in % of TSS</td>
+ </tr>
+ <tr>
+ <td>BCSS_M</td>
+ <td> </td>
+ <td>Between-Cluster Sum of Squares (means as centers)</td>
+ </tr>
+ <tr>
+ <td>BCSS_M_PC</td>
+ <td> </td>
+ <td>Between-Cluster Sum of Squares (means), in % of TSS</td>
+ </tr>
+ <tr>
+ <td style="text-align: center" rowspan="4">X and C</td>
+ <td>WCSS_C</td>
+ <td> </td>
+ <td>Within-Cluster Sum of Squares (centroids as centers)</td>
+ </tr>
+ <tr>
+ <td>WCSS_C_PC</td>
+ <td> </td>
+ <td>Within-Cluster Sum of Squares (centroids), % of TSS</td>
+ </tr>
+ <tr>
+ <td>BCSS_C</td>
+ <td> </td>
+ <td>Between-Cluster Sum of Squares (centroids as centers)</td>
+ </tr>
+ <tr>
+ <td>BCSS_C_PC</td>
+ <td> </td>
+ <td>Between-Cluster Sum of Squares (centroids), % of TSS</td>
+ </tr>
+ <tr>
+ <td style="text-align: center" rowspan="8">spY</td>
+ <td>TRUE_SAME_CT</td>
+ <td> </td>
+ <td>Same-category pairs predicted as Same-cluster, count</td>
+ </tr>
+ <tr>
+ <td>TRUE_SAME_PC</td>
+ <td> </td>
+ <td>Same-category pairs predicted as Same-cluster, %</td>
+ </tr>
+ <tr>
+ <td>TRUE_DIFF_CT</td>
+ <td> </td>
+ <td>Diff-category pairs predicted as Diff-cluster, count</td>
+ </tr>
+ <tr>
+ <td>TRUE_DIFF_PC</td>
+ <td> </td>
+ <td>Diff-category pairs predicted as Diff-cluster, %</td>
+ </tr>
+ <tr>
+ <td>FALSE_SAME_CT</td>
+ <td> </td>
+ <td>Diff-category pairs predicted as Same-cluster, count</td>
+ </tr>
+ <tr>
+ <td>FALSE_SAME_PC</td>
+ <td> </td>
+ <td>Diff-category pairs predicted as Same-cluster, %</td>
+ </tr>
+ <tr>
+ <td>FALSE_DIFF_CT</td>
+ <td> </td>
+ <td>Same-category pairs predicted as Diff-cluster, count</td>
+ </tr>
+ <tr>
+ <td>FALSE_DIFF_PC</td>
+ <td> </td>
+ <td>Same-category pairs predicted as Diff-cluster, %</td>
+ </tr>
+ <tr>
+ <td style="text-align: center" rowspan="8">spY</td>
+ <td>SPEC_TO_PRED</td>
+ <td style="text-align: center">+</td>
+ <td>For specified category, the best predicted cluster id</td>
+ </tr>
+ <tr>
+ <td>SPEC_FULL_CT</td>
+ <td style="text-align: center">+</td>
+ <td>For specified category, its full count</td>
+ </tr>
+ <tr>
+ <td>SPEC_MATCH_CT</td>
+ <td style="text-align: center">+</td>
+ <td>For specified category, best-cluster matching count</td>
+ </tr>
+ <tr>
+ <td>SPEC_MATCH_PC</td>
+ <td style="text-align: center">+</td>
+ <td>For specified category, % of matching to full count</td>
+ </tr>
+ <tr>
+ <td>PRED_TO_SPEC</td>
+ <td style="text-align: center">+</td>
+ <td>For predicted cluster, the best specified category id</td>
+ </tr>
+ <tr>
+ <td>PRED_FULL_CT</td>
+ <td style="text-align: center">+</td>
+ <td>For predicted cluster, its full count</td>
+ </tr>
+ <tr>
+ <td>PRED_MATCH_CT</td>
+ <td style="text-align: center">+</td>
+ <td>For predicted cluster, best-category matching count</td>
+ </tr>
+ <tr>
+ <td>PRED_MATCH_PC</td>
+ <td style="text-align: center">+</td>
+ <td>For predicted cluster, % of matching to full count</td>
+ </tr>
+ </tbody>
+</table>
+
+* * *
+
+
+### Details
+
+Our clustering script proceeds in 3 stages: centroid initialization,
+parallel $k$-means iterations, and the best-available output generation.
+Centroids are initialized at random from the input records (the rows
+of $X$), biased towards being chosen far apart from each other. The
+initialization method is based on the `k-means++` heuristic
+from [[ArthurVassilvitskii2007]](algorithms-bibliography.html), with one important difference: to
+reduce the number of passes through $X$, we take a small sample of $X$
+and run the `k-means++` heuristic over this sample. Here is,
+conceptually, our centroid initialization algorithm for one clustering
+run:
+
+ <ol>
+ <li>Sample the rows of $X$ uniformly at random, picking each row with
+probability $p = ks / n$ where
+ <ul>
+ <li>$k$ is the number of centroids</li>
+ <li>$n$ is the number of records</li>
+ <li>$s$ is the samp input parameter</li>
+ </ul>
+ If $ks \geq n$, the entire $X$ is used in place of its sample.
+ </li>
+ <li>Choose the first centroid uniformly at random from the sampled rows.</li>
+ <li>Choose each subsequent centroid from the sampled rows, at random, with
+probability proportional to the squared Euclidean distance between the
+row and the nearest already-chosen centroid.</li>
+ </ol>
+
+The sampling of $X$ and the selection of centroids are performed
+independently and in parallel for each run of the $k$-means algorithm.
+When we sample the rows of $X$, rather than tossing a random coin for
+each row, we compute the number of rows to skip until the next sampled
+row as $\lceil \log(u) / \log(1 - p) \rceil$ where $u\in (0, 1)$ is
+uniformly random. This time-saving trick works because
+
+$$Prob[k-1 < \log_{1-p}(u) < k] \,\,=\,\, p(1-p)^{k-1} \,\,=\,\,
+Prob[\textrm{skip $k-1$ rows}]$$
+
+However, it requires us to estimate the maximum sample size, which we
+set near $ks + 10\sqrt{ks}$ to make it generous enough.
+
+Once we selected the initial centroid sets, we start the $k$-means
+iterations independently in parallel for all clustering runs. The number
+of clustering runs is given as the runs input parameter.
+Each iteration of each clustering run performs the following steps:
+
+ * Compute the centroid-dependent part of squared Euclidean distances from
+all records (rows of $X$) to each of the $k$ centroids using matrix
+product.
+ * Take the minimum of the above for each record.
+ * Update the current within-cluster sum of squares (WCSS) value, with
+centroids substituted instead of the means for efficiency.
+ * Check the convergence
+criterion:
+$$\textrm{WCSS}_{\mathrm{old}} - \textrm{WCSS}_{\mathrm{new}} < {\varepsilon}\cdot\textrm{WCSS}_{\mathrm{new}}$$
+as
+well as the number of iterations limit.
+ * Find the closest centroid for each record, sharing equally any records
+with multiple closest centroids.
+ * Compute the number of records closest to each centroid, checking for
+“runaway” centroids with no records left (in which case the run fails).
+ * Compute the new centroids by averaging the records in their clusters.
+
+When a termination condition is satisfied, we store the centroids and
+the WCSS value and exit this run. A run has to satisfy the WCSS
+convergence criterion to be considered successful. Upon the termination
+of all runs, we select the smallest WCSS value among the successful
+runs, and write out this run’s centroids. If requested, we also compute
+the cluster assignment of all records in $X$, using integers from 1
+to $k$ as the cluster labels. The scoring script can then be used to
+compare the cluster assignment with an externally specified category
+assignment.
+
+
+### Returns
+
+We output the $k$ centroids for the best available clustering,
+i. e. whose WCSS is the smallest of all successful runs. The centroids
+are written as the rows of the $k\,{\times}\,m$-matrix into the output
+file whose path/name was provided as the “C” input
+argument. If the input parameter “isY” was set
+to 1, we also output the one-column matrix with the cluster
+assignment for all the records. This assignment is written into the file
+whose path/name was provided as the “Y” input argument. The
+best WCSS value, as well as some information about the performance of
+the other runs, is printed during the script execution. The scoring
+script `Kmeans-predict.dml` prints all its results in a
+self-explanatory manner, as defined in
+[**Table 6**](algorithms-clustering.html#table6).
+
+
[46/47] incubator-systemml git commit: [SYSML-341] Parfor check
option documented
Posted by du...@apache.org.
[SYSML-341] Parfor check option documented
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/c8cc6eb7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/c8cc6eb7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/c8cc6eb7
Branch: refs/heads/gh-pages
Commit: c8cc6eb7a0cfa31453b6498b8cb145ebf7fd523d
Parents: bf8d44e
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Tue Nov 3 10:35:13 2015 -0300
Committer: Luciano Resende <lr...@apache.org>
Committed: Tue Nov 3 10:35:13 2015 -0300
----------------------------------------------------------------------
dml-language-reference.md | 33 +++++++++++++++++++++++++++++++--
1 file changed, 31 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/c8cc6eb7/dml-language-reference.md
----------------------------------------------------------------------
diff --git a/dml-language-reference.md b/dml-language-reference.md
index f6915a3..dff0886 100644
--- a/dml-language-reference.md
+++ b/dml-language-reference.md
@@ -304,7 +304,7 @@ The for loop body may contain any sequence of statements. The statements in the
#### ParFor Statement
-The syntax and semantics of a parfor statement are equivalent to a for statement except for the different keyword and a list of optional parameters.
+The syntax and semantics of a `parfor` (parallel `for`) statement are equivalent to a `for` statement except for the different keyword and a list of optional parameters.
parfor (var in <for_predicate> <parfor_paramslist> ) {
<statement>*
@@ -329,7 +329,36 @@ The syntax and semantics of a parfor statement are equivalent to a for statement
<result_merge_mode> is one of the following tokens: LOCAL_MEM LOCAL_FILE LOCAL_AUTOMATIC REMOTE_MR
<optimization_mode> is one of the following tokens: NONE RULEBASED HEURISTIC GREEDY FULL_DP
-If any of these parameters is not specified, the following respective defaults are used: check = 1, par = [number of virtual processors on master node], mode = LOCAL, taskpartitioner = FIXED, tasksize =1, datapartitioner = NONE, resultmerge = LOCAL_AUTOMATIC, opt = RULEBASED.
+If any of these parameters is not specified, the following respective defaults are used: `check = 1`, `par = [number of virtual processors on master node]`, `mode = LOCAL`, `taskpartitioner = FIXED`, `tasksize = 1`, `datapartitioner = NONE`, `resultmerge = LOCAL_AUTOMATIC`, `opt = RULEBASED`.
+
+Of particular note is the `check` parameter. SystemML's `parfor` statement by default (`check = 1`) performs dependency analysis in an
+attempt to guarantee result correctness for parallel execution. For example, the following `parfor` statement is **incorrect** because
+the iterations do not act independently, so they are not parallizable. The iterations incorrectly try to increment the same `sum` variable.
+
+ sum = 0
+ parfor(i in 1:3) {
+ sum = sum + i; # not parallizable - generates error
+ }
+ print(sum)
+
+SystemML's `parfor` dependency analysis can occasionally result in false positives, as in the following example. This example creates a 2x30
+matrix. It then utilizes a `parfor` loop to write 10 2x3 matrices into the 2x30 matrix. This `parfor` statement is parallizable and correct,
+but the dependency analysis generates a false positive dependency error for the variable `ms`.
+
+ ms = matrix(0, rows=2, cols=3*10)
+ parfor (v in 1:10) { # parallizable - false positive
+ mv = matrix(v, rows=2, cols=3)
+ ms[,(v-1)*3+1:v*3] = mv
+ }
+
+If a false positive arises but you are certain that the `parfor` is parallizable, the `parfor` dependency check can be disabled via
+the `check = 0` option.
+
+ ms = matrix(0, rows=2, cols=3*10)
+ parfor (v in 1:10, check=0) { # parallizable
+ mv = matrix(v, rows=2, cols=3)
+ ms[,(v-1)*3+1:v*3] = mv
+ }
### User-Defined Function (UDF)
[27/47] incubator-systemml git commit: [SYSML-282] Use "select"
vector to remove empty rows/cols
Posted by du...@apache.org.
[SYSML-282] Use "select" vector to remove empty rows/cols
This change was based on observation that time consumed by eviction was
high in experimental data. In few algorithms such as -- DecisionTree,
RandomForest -- in which Matrix was operated with RemoveEmpty operator.
Before doing RemoveEmpty operation, select matrix was operated on target
matrix with append and elementwise multiplication. This is repeated in
loop operation. Instead of operating select vector on target matrix,
select vector can be passed to RemoveEmpty to avoid expensive operations
causing delay in eviction.
This fix will provide an option for caller to send select vector in
RemoveEmpty operator.
Enhancement to RemoveEmpty include following:
- It will use "Select vector" if passed in to figure out rows/columns
to be removed.
- Memory requirement estimate update.
- Test cases to evaluate this change updated.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/442e83a4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/442e83a4
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/442e83a4
Branch: refs/heads/gh-pages
Commit: 442e83a469b99bcf1f33d2489a25b9ba3857de3a
Parents: b966a81
Author: Arvind Surve <ac...@yahoo.com>
Authored: Tue Sep 15 14:08:04 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Tue Sep 15 14:08:04 2015 -0700
----------------------------------------------------------------------
Language Reference/SystemML_Language_Reference.html | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/442e83a4/Language Reference/SystemML_Language_Reference.html
----------------------------------------------------------------------
diff --git a/Language Reference/SystemML_Language_Reference.html b/Language Reference/SystemML_Language_Reference.html
index 76e2c04..ce3635d 100644
--- a/Language Reference/SystemML_Language_Reference.html
+++ b/Language Reference/SystemML_Language_Reference.html
@@ -4421,8 +4421,9 @@ Manipulation, and Aggregation Built-In Functions<o:p></o:p></span></p>
mso-border-top-alt:solid windowtext .5pt;mso-border-left-alt:solid windowtext .5pt;
mso-border-alt:solid windowtext .5pt;padding:0in 5.4pt 0in 5.4pt'>
<p class=MsoNormal>Removes all empty rows or columns from the input matrix <i>target</i>
- X according to the specified <i>margin.</i><span style='mso-fareast-font-family:
- "Times New Roman";mso-fareast-theme-font:minor-fareast;color:black'><o:p></o:p></span></p>
+ X according to the specified <i>margin.</i> <br>Optionally vector select can be passed to remove empty rows/columns.
+ <br>select vector should contain values either 1.0 or 0.0 for rows/columns to be retained or removed respectively.
+ <span style='mso-fareast-font-family: "Times New Roman";mso-fareast-theme-font:minor-fareast;color:black'><o:p></o:p></span></p>
</td>
<td width=162 valign=top style='width:121.75pt;border-top:none;border-left:
none;border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt;
@@ -4430,7 +4431,7 @@ Manipulation, and Aggregation Built-In Functions<o:p></o:p></span></p>
mso-border-alt:solid windowtext .5pt;padding:0in 5.4pt 0in 5.4pt'>
<p class=MsoNormal><span class=GramE><b style='mso-bidi-font-weight:normal'>Input</b>
:</span><b style='mso-bidi-font-weight:normal'> </b>(target= X
- <matrix>, margin="...")</p>
+ <matrix>, margin="..."[,select=I])</p>
<p class=MsoNormal><b style='mso-bidi-font-weight:normal'>Output</b> :
<matrix></p>
<p class=MsoNormal>Valid values for margin are "rows" or
@@ -4442,7 +4443,9 @@ Manipulation, and Aggregation Built-In Functions<o:p></o:p></span></p>
mso-border-top-alt:solid windowtext .5pt;mso-border-left-alt:solid windowtext .5pt;
mso-border-alt:solid windowtext .5pt;padding:0in 5.4pt 0in 5.4pt'>
<p class=MsoNormal>A = <span class=SpellE>removeEmpty</span>(target=X,
- margin="rows")<span style='mso-fareast-font-family:"Times New Roman";
+ margin="rows") <br> A = <span class=SpellE>removeEmpty</span>(target=X,
+ margin="rows",select=I)
+ <span style='mso-fareast-font-family:"Times New Roman";
mso-fareast-theme-font:minor-fareast;color:black'><o:p></o:p></span></p>
</td>
</tr>
[14/47] incubator-systemml git commit: initial port of documentation
to md, algorithms reference converted to md, template based on spark docs
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/css/bootstrap-responsive.min.css
----------------------------------------------------------------------
diff --git a/css/bootstrap-responsive.min.css b/css/bootstrap-responsive.min.css
new file mode 100644
index 0000000..ab59da3
--- /dev/null
+++ b/css/bootstrap-responsive.min.css
@@ -0,0 +1,9 @@
+/*!
+ * Bootstrap Responsive v2.1.0
+ *
+ * Copyright 2012 Twitter, Inc
+ * Licensed under the Apache License v2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Designed and built with all the love in the world @twitter by @mdo and @fat.
+ */.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;line-height:0;content:""}.clearfix:after{clear:both}.hide-text{font:0/0 a;color:transparent;text-shadow:none;background-color:transparent;border:0}.input-block-level{display:block;width:100%;min-height:30px;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.hidden{display:none;visibility:hidden}.visible-phone{display:none!important}.visible-tablet{display:none!important}.hidden-desktop{display:none!important}.visible-desktop{display:inherit!important}@media(min-width:768px) and (max-width:979px){.hidden-desktop{display:inherit!important}.visible-desktop{display:none!important}.visible-tablet{display:inherit!important}.hidden-tablet{display:none!important}}@media(max-width:767px){.hidden-desktop{display:inherit!important}.visible-desktop{display:none!important}.visible-phone{display:inherit!important}.hidden-phone{display:none!important}}@media(min-width:1200px){.row{margin-left:-30px;*zo
om:1}.row:before,.row:after{display:table;line-height:0;content:""}.row:after{clear:both}[class*="span"]{float:left;margin-left:30px}.container,.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:1170px}.span12{width:1170px}.span11{width:1070px}.span10{width:970px}.span9{width:870px}.span8{width:770px}.span7{width:670px}.span6{width:570px}.span5{width:470px}.span4{width:370px}.span3{width:270px}.span2{width:170px}.span1{width:70px}.offset12{margin-left:1230px}.offset11{margin-left:1130px}.offset10{margin-left:1030px}.offset9{margin-left:930px}.offset8{margin-left:830px}.offset7{margin-left:730px}.offset6{margin-left:630px}.offset5{margin-left:530px}.offset4{margin-left:430px}.offset3{margin-left:330px}.offset2{margin-left:230px}.offset1{margin-left:130px}.row-fluid{width:100%;*zoom:1}.row-fluid:before,.row-fluid:after{display:table;line-height:0;content:""}.row-fluid:after{clear:both}.row-fluid [class*="span"]{display:block;float:left;wid
th:100%;min-height:30px;margin-left:2.564102564102564%;*margin-left:2.5109110747408616%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid [class*="span"]:first-child{margin-left:0}.row-fluid .span12{width:100%;*width:99.94680851063829%}.row-fluid .span11{width:91.45299145299145%;*width:91.39979996362975%}.row-fluid .span10{width:82.90598290598291%;*width:82.8527914166212%}.row-fluid .span9{width:74.35897435897436%;*width:74.30578286961266%}.row-fluid .span8{width:65.81196581196582%;*width:65.75877432260411%}.row-fluid .span7{width:57.26495726495726%;*width:57.21176577559556%}.row-fluid .span6{width:48.717948717948715%;*width:48.664757228587014%}.row-fluid .span5{width:40.17094017094017%;*width:40.11774868157847%}.row-fluid .span4{width:31.623931623931625%;*width:31.570740134569924%}.row-fluid .span3{width:23.076923076923077%;*width:23.023731587561375%}.row-fluid .span2{width:14.52991452991453%;*width:14.476723040552828%}.row-fluid .span1{width
:5.982905982905983%;*width:5.929714493544281%}.row-fluid .offset12{margin-left:105.12820512820512%;*margin-left:105.02182214948171%}.row-fluid .offset12:first-child{margin-left:102.56410256410257%;*margin-left:102.45771958537915%}.row-fluid .offset11{margin-left:96.58119658119658%;*margin-left:96.47481360247316%}.row-fluid .offset11:first-child{margin-left:94.01709401709402%;*margin-left:93.91071103837061%}.row-fluid .offset10{margin-left:88.03418803418803%;*margin-left:87.92780505546462%}.row-fluid .offset10:first-child{margin-left:85.47008547008548%;*margin-left:85.36370249136206%}.row-fluid .offset9{margin-left:79.48717948717949%;*margin-left:79.38079650845607%}.row-fluid .offset9:first-child{margin-left:76.92307692307693%;*margin-left:76.81669394435352%}.row-fluid .offset8{margin-left:70.94017094017094%;*margin-left:70.83378796144753%}.row-fluid .offset8:first-child{margin-left:68.37606837606839%;*margin-left:68.26968539734497%}.row-fluid .offset7{margin-left:62.393162393162385%
;*margin-left:62.28677941443899%}.row-fluid .offset7:first-child{margin-left:59.82905982905982%;*margin-left:59.72267685033642%}.row-fluid .offset6{margin-left:53.84615384615384%;*margin-left:53.739770867430444%}.row-fluid .offset6:first-child{margin-left:51.28205128205128%;*margin-left:51.175668303327875%}.row-fluid .offset5{margin-left:45.299145299145295%;*margin-left:45.1927623204219%}.row-fluid .offset5:first-child{margin-left:42.73504273504273%;*margin-left:42.62865975631933%}.row-fluid .offset4{margin-left:36.75213675213675%;*margin-left:36.645753773413354%}.row-fluid .offset4:first-child{margin-left:34.18803418803419%;*margin-left:34.081651209310785%}.row-fluid .offset3{margin-left:28.205128205128204%;*margin-left:28.0987452264048%}.row-fluid .offset3:first-child{margin-left:25.641025641025642%;*margin-left:25.53464266230224%}.row-fluid .offset2{margin-left:19.65811965811966%;*margin-left:19.551736679396257%}.row-fluid .offset2:first-child{margin-left:17.094017094017094%;*mar
gin-left:16.98763411529369%}.row-fluid .offset1{margin-left:11.11111111111111%;*margin-left:11.004728132387708%}.row-fluid .offset1:first-child{margin-left:8.547008547008547%;*margin-left:8.440625568285142%}input,textarea,.uneditable-input{margin-left:0}.controls-row [class*="span"]+[class*="span"]{margin-left:30px}input.span12,textarea.span12,.uneditable-input.span12{width:1156px}input.span11,textarea.span11,.uneditable-input.span11{width:1056px}input.span10,textarea.span10,.uneditable-input.span10{width:956px}input.span9,textarea.span9,.uneditable-input.span9{width:856px}input.span8,textarea.span8,.uneditable-input.span8{width:756px}input.span7,textarea.span7,.uneditable-input.span7{width:656px}input.span6,textarea.span6,.uneditable-input.span6{width:556px}input.span5,textarea.span5,.uneditable-input.span5{width:456px}input.span4,textarea.span4,.uneditable-input.span4{width:356px}input.span3,textarea.span3,.uneditable-input.span3{width:256px}input.span2,textarea.span2,.uneditable-
input.span2{width:156px}input.span1,textarea.span1,.uneditable-input.span1{width:56px}.thumbnails{margin-left:-30px}.thumbnails>li{margin-left:30px}.row-fluid .thumbnails{margin-left:0}}@media(min-width:768px) and (max-width:979px){.row{margin-left:-20px;*zoom:1}.row:before,.row:after{display:table;line-height:0;content:""}.row:after{clear:both}[class*="span"]{float:left;margin-left:20px}.container,.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:724px}.span12{width:724px}.span11{width:662px}.span10{width:600px}.span9{width:538px}.span8{width:476px}.span7{width:414px}.span6{width:352px}.span5{width:290px}.span4{width:228px}.span3{width:166px}.span2{width:104px}.span1{width:42px}.offset12{margin-left:764px}.offset11{margin-left:702px}.offset10{margin-left:640px}.offset9{margin-left:578px}.offset8{margin-left:516px}.offset7{margin-left:454px}.offset6{margin-left:392px}.offset5{margin-left:330px}.offset4{margin-left:268px}.offset3{margin-
left:206px}.offset2{margin-left:144px}.offset1{margin-left:82px}.row-fluid{width:100%;*zoom:1}.row-fluid:before,.row-fluid:after{display:table;line-height:0;content:""}.row-fluid:after{clear:both}.row-fluid [class*="span"]{display:block;float:left;width:100%;min-height:30px;margin-left:2.7624309392265194%;*margin-left:2.709239449864817%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid [class*="span"]:first-child{margin-left:0}.row-fluid .span12{width:100%;*width:99.94680851063829%}.row-fluid .span11{width:91.43646408839778%;*width:91.38327259903608%}.row-fluid .span10{width:82.87292817679558%;*width:82.81973668743387%}.row-fluid .span9{width:74.30939226519337%;*width:74.25620077583166%}.row-fluid .span8{width:65.74585635359117%;*width:65.69266486422946%}.row-fluid .span7{width:57.18232044198895%;*width:57.12912895262725%}.row-fluid .span6{width:48.61878453038674%;*width:48.56559304102504%}.row-fluid .span5{width:40.05524861878453%;*width:40.0
0205712942283%}.row-fluid .span4{width:31.491712707182323%;*width:31.43852121782062%}.row-fluid .span3{width:22.92817679558011%;*width:22.87498530621841%}.row-fluid .span2{width:14.3646408839779%;*width:14.311449394616199%}.row-fluid .span1{width:5.801104972375691%;*width:5.747913483013988%}.row-fluid .offset12{margin-left:105.52486187845304%;*margin-left:105.41847889972962%}.row-fluid .offset12:first-child{margin-left:102.76243093922652%;*margin-left:102.6560479605031%}.row-fluid .offset11{margin-left:96.96132596685082%;*margin-left:96.8549429881274%}.row-fluid .offset11:first-child{margin-left:94.1988950276243%;*margin-left:94.09251204890089%}.row-fluid .offset10{margin-left:88.39779005524862%;*margin-left:88.2914070765252%}.row-fluid .offset10:first-child{margin-left:85.6353591160221%;*margin-left:85.52897613729868%}.row-fluid .offset9{margin-left:79.8342541436464%;*margin-left:79.72787116492299%}.row-fluid .offset9:first-child{margin-left:77.07182320441989%;*margin-left:76.96544
022569647%}.row-fluid .offset8{margin-left:71.2707182320442%;*margin-left:71.16433525332079%}.row-fluid .offset8:first-child{margin-left:68.50828729281768%;*margin-left:68.40190431409427%}.row-fluid .offset7{margin-left:62.70718232044199%;*margin-left:62.600799341718584%}.row-fluid .offset7:first-child{margin-left:59.94475138121547%;*margin-left:59.838368402492065%}.row-fluid .offset6{margin-left:54.14364640883978%;*margin-left:54.037263430116376%}.row-fluid .offset6:first-child{margin-left:51.38121546961326%;*margin-left:51.27483249088986%}.row-fluid .offset5{margin-left:45.58011049723757%;*margin-left:45.47372751851417%}.row-fluid .offset5:first-child{margin-left:42.81767955801105%;*margin-left:42.71129657928765%}.row-fluid .offset4{margin-left:37.01657458563536%;*margin-left:36.91019160691196%}.row-fluid .offset4:first-child{margin-left:34.25414364640884%;*margin-left:34.14776066768544%}.row-fluid .offset3{margin-left:28.45303867403315%;*margin-left:28.346655695309746%}.row-fluid
.offset3:first-child{margin-left:25.69060773480663%;*margin-left:25.584224756083227%}.row-fluid .offset2{margin-left:19.88950276243094%;*margin-left:19.783119783707537%}.row-fluid .offset2:first-child{margin-left:17.12707182320442%;*margin-left:17.02068884448102%}.row-fluid .offset1{margin-left:11.32596685082873%;*margin-left:11.219583872105325%}.row-fluid .offset1:first-child{margin-left:8.56353591160221%;*margin-left:8.457152932878806%}input,textarea,.uneditable-input{margin-left:0}.controls-row [class*="span"]+[class*="span"]{margin-left:20px}input.span12,textarea.span12,.uneditable-input.span12{width:710px}input.span11,textarea.span11,.uneditable-input.span11{width:648px}input.span10,textarea.span10,.uneditable-input.span10{width:586px}input.span9,textarea.span9,.uneditable-input.span9{width:524px}input.span8,textarea.span8,.uneditable-input.span8{width:462px}input.span7,textarea.span7,.uneditable-input.span7{width:400px}input.span6,textarea.span6,.uneditable-input.span6{width:
338px}input.span5,textarea.span5,.uneditable-input.span5{width:276px}input.span4,textarea.span4,.uneditable-input.span4{width:214px}input.span3,textarea.span3,.uneditable-input.span3{width:152px}input.span2,textarea.span2,.uneditable-input.span2{width:90px}input.span1,textarea.span1,.uneditable-input.span1{width:28px}}@media(max-width:767px){body{padding-right:20px;padding-left:20px}.navbar-fixed-top,.navbar-fixed-bottom{margin-right:-20px;margin-left:-20px}.container-fluid{padding:0}.dl-horizontal dt{float:none;width:auto;clear:none;text-align:left}.dl-horizontal dd{margin-left:0}.container{width:auto}.row-fluid{width:100%}.row,.thumbnails{margin-left:0}.thumbnails>li{float:none;margin-left:0}[class*="span"],.row-fluid [class*="span"]{display:block;float:none;width:auto;margin-left:0}.span12,.row-fluid .span12{width:100%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.input-large,.input-xlarge,.input-xxlarge,input[class*="span"],select[class*="span"]
,textarea[class*="span"],.uneditable-input{display:block;width:100%;min-height:30px;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.input-prepend input,.input-append input,.input-prepend input[class*="span"],.input-append input[class*="span"]{display:inline-block;width:auto}.modal{position:fixed;top:20px;right:20px;left:20px;width:auto;margin:0}.modal.fade.in{top:auto}}@media(max-width:480px){.nav-collapse{-webkit-transform:translate3d(0,0,0)}.page-header h1 small{display:block;line-height:20px}input[type="checkbox"],input[type="radio"]{border:1px solid #ccc}.form-horizontal .control-group>label{float:none;width:auto;padding-top:0;text-align:left}.form-horizontal .controls{margin-left:0}.form-horizontal .control-list{padding-top:0}.form-horizontal .form-actions{padding-right:10px;padding-left:10px}.modal{top:10px;right:10px;left:10px}.modal-header .close{padding:10px;margin:-10px}.carousel-caption{position:static}}@media(max-width:979px){body{padding-
top:0}.navbar-fixed-top,.navbar-fixed-bottom{position:static}.navbar-fixed-top{margin-bottom:20px}.navbar-fixed-bottom{margin-top:20px}.navbar-fixed-top .navbar-inner,.navbar-fixed-bottom .navbar-inner{padding:5px}.navbar .container{width:auto;padding:0}.navbar .brand{padding-right:10px;padding-left:10px;margin:0 0 0 -5px}.nav-collapse{clear:both}.nav-collapse .nav{float:none;margin:0 0 10px}.nav-collapse .nav>li{float:none}.nav-collapse .nav>li>a{margin-bottom:2px}.nav-collapse .nav>.divider-vertical{display:none}.nav-collapse .nav .nav-header{color:#555;text-shadow:none}.nav-collapse .nav>li>a,.nav-collapse .dropdown-menu a{padding:9px 15px;font-weight:bold;color:#555;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}.nav-collapse .btn{padding:4px 10px 4px;font-weight:normal;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.nav-collapse .dropdown-menu li+li a{margin-bottom:2px}.nav-collapse .nav>li>a:hover,.nav-collapse .dropdown-menu a:hover{bac
kground-color:#f2f2f2}.navbar-inverse .nav-collapse .nav>li>a:hover,.navbar-inverse .nav-collapse .dropdown-menu a:hover{background-color:#111}.nav-collapse.in .btn-group{padding:0;margin-top:5px}.nav-collapse .dropdown-menu{position:static;top:auto;left:auto;display:block;float:none;max-width:none;padding:0;margin:0 15px;background-color:transparent;border:0;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0;-webkit-box-shadow:none;-moz-box-shadow:none;box-shadow:none}.nav-collapse .dropdown-menu:before,.nav-collapse .dropdown-menu:after{display:none}.nav-collapse .dropdown-menu .divider{display:none}.nav-collapse .navbar-form,.nav-collapse .navbar-search{float:none;padding:10px 15px;margin:10px 0;border-top:1px solid #f2f2f2;border-bottom:1px solid #f2f2f2;-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.1);-moz-box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.1);box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1p
x 0 rgba(255,255,255,0.1)}.navbar .nav-collapse .nav.pull-right{float:none;margin-left:0}.nav-collapse,.nav-collapse.collapse{height:0;overflow:hidden}.navbar .btn-navbar{display:block}.navbar-static .navbar-inner{padding-right:10px;padding-left:10px}}@media(min-width:980px){.nav-collapse.collapse{height:auto!important;overflow:visible!important}}
[08/47] incubator-systemml git commit: initial port of documentation
to md, algorithms reference converted to md, template based on spark docs
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/js/vendor/modernizr-2.6.1-respond-1.1.0.min.js
----------------------------------------------------------------------
diff --git a/js/vendor/modernizr-2.6.1-respond-1.1.0.min.js b/js/vendor/modernizr-2.6.1-respond-1.1.0.min.js
new file mode 100644
index 0000000..8e70894
--- /dev/null
+++ b/js/vendor/modernizr-2.6.1-respond-1.1.0.min.js
@@ -0,0 +1,11 @@
+/* Modernizr 2.6.1 (Custom Build) | MIT & BSD
+ * Build: http://modernizr.com/download/#-fontface-backgroundsize-borderimage-borderradius-boxshadow-flexbox-hsla-multiplebgs-opacity-rgba-textshadow-cssanimations-csscolumns-generatedcontent-cssgradients-cssreflections-csstransforms-csstransforms3d-csstransitions-applicationcache-canvas-canvastext-draganddrop-hashchange-history-audio-video-indexeddb-input-inputtypes-localstorage-postmessage-sessionstorage-websockets-websqldatabase-webworkers-geolocation-inlinesvg-smil-svg-svgclippaths-touch-webgl-shiv-mq-cssclasses-addtest-prefixed-teststyles-testprop-testallprops-hasevent-prefixes-domprefixes-load
+ */
+;window.Modernizr=function(a,b,c){function D(a){j.cssText=a}function E(a,b){return D(n.join(a+";")+(b||""))}function F(a,b){return typeof a===b}function G(a,b){return!!~(""+a).indexOf(b)}function H(a,b){for(var d in a){var e=a[d];if(!G(e,"-")&&j[e]!==c)return b=="pfx"?e:!0}return!1}function I(a,b,d){for(var e in a){var f=b[a[e]];if(f!==c)return d===!1?a[e]:F(f,"function")?f.bind(d||b):f}return!1}function J(a,b,c){var d=a.charAt(0).toUpperCase()+a.slice(1),e=(a+" "+p.join(d+" ")+d).split(" ");return F(b,"string")||F(b,"undefined")?H(e,b):(e=(a+" "+q.join(d+" ")+d).split(" "),I(e,b,c))}function K(){e.input=function(c){for(var d=0,e=c.length;d<e;d++)u[c[d]]=c[d]in k;return u.list&&(u.list=!!b.createElement("datalist")&&!!a.HTMLDataListElement),u}("autocomplete autofocus list placeholder max min multiple pattern required step".split(" ")),e.inputtypes=function(a){for(var d=0,e,f,h,i=a.length;d<i;d++)k.setAttribute("type",f=a[d]),e=k.type!=="text",e&&(k.value=l,k.style.cssText="position:
absolute;visibility:hidden;",/^range$/.test(f)&&k.style.WebkitAppearance!==c?(g.appendChild(k),h=b.defaultView,e=h.getComputedStyle&&h.getComputedStyle(k,null).WebkitAppearance!=="textfield"&&k.offsetHeight!==0,g.removeChild(k)):/^(search|tel)$/.test(f)||(/^(url|email)$/.test(f)?e=k.checkValidity&&k.checkValidity()===!1:e=k.value!=l)),t[a[d]]=!!e;return t}("search tel url email datetime date month week time datetime-local number range color".split(" "))}var d="2.6.1",e={},f=!0,g=b.documentElement,h="modernizr",i=b.createElement(h),j=i.style,k=b.createElement("input"),l=":)",m={}.toString,n=" -webkit- -moz- -o- -ms- ".split(" "),o="Webkit Moz O ms",p=o.split(" "),q=o.toLowerCase().split(" "),r={svg:"http://www.w3.org/2000/svg"},s={},t={},u={},v=[],w=v.slice,x,y=function(a,c,d,e){var f,i,j,k=b.createElement("div"),l=b.body,m=l?l:b.createElement("body");if(parseInt(d,10))while(d--)j=b.createElement("div"),j.id=e?e[d]:h+(d+1),k.appendChild(j);return f=["­",'<style id="s',h,'">',a,"
</style>"].join(""),k.id=h,(l?k:m).innerHTML+=f,m.appendChild(k),l||(m.style.background="",g.appendChild(m)),i=c(k,a),l?k.parentNode.removeChild(k):m.parentNode.removeChild(m),!!i},z=function(b){var c=a.matchMedia||a.msMatchMedia;if(c)return c(b).matches;var d;return y("@media "+b+" { #"+h+" { position: absolute; } }",function(b){d=(a.getComputedStyle?getComputedStyle(b,null):b.currentStyle)["position"]=="absolute"}),d},A=function(){function d(d,e){e=e||b.createElement(a[d]||"div"),d="on"+d;var f=d in e;return f||(e.setAttribute||(e=b.createElement("div")),e.setAttribute&&e.removeAttribute&&(e.setAttribute(d,""),f=F(e[d],"function"),F(e[d],"undefined")||(e[d]=c),e.removeAttribute(d))),e=null,f}var a={select:"input",change:"input",submit:"form",reset:"form",error:"img",load:"img",abort:"img"};return d}(),B={}.hasOwnProperty,C;!F(B,"undefined")&&!F(B.call,"undefined")?C=function(a,b){return B.call(a,b)}:C=function(a,b){return b in a&&F(a.constructor.prototype[b],"undefined")},Function
.prototype.bind||(Function.prototype.bind=function(b){var c=this;if(typeof c!="function")throw new TypeError;var d=w.call(arguments,1),e=function(){if(this instanceof e){var a=function(){};a.prototype=c.prototype;var f=new a,g=c.apply(f,d.concat(w.call(arguments)));return Object(g)===g?g:f}return c.apply(b,d.concat(w.call(arguments)))};return e}),s.flexbox=function(){return J("flexWrap")},s.canvas=function(){var a=b.createElement("canvas");return!!a.getContext&&!!a.getContext("2d")},s.canvastext=function(){return!!e.canvas&&!!F(b.createElement("canvas").getContext("2d").fillText,"function")},s.webgl=function(){return!!a.WebGLRenderingContext},s.touch=function(){var c;return"ontouchstart"in a||a.DocumentTouch&&b instanceof DocumentTouch?c=!0:y(["@media (",n.join("touch-enabled),("),h,")","{#modernizr{top:9px;position:absolute}}"].join(""),function(a){c=a.offsetTop===9}),c},s.geolocation=function(){return"geolocation"in navigator},s.postmessage=function(){return!!a.postMessage},s.webs
qldatabase=function(){return!!a.openDatabase},s.indexedDB=function(){return!!J("indexedDB",a)},s.hashchange=function(){return A("hashchange",a)&&(b.documentMode===c||b.documentMode>7)},s.history=function(){return!!a.history&&!!history.pushState},s.draganddrop=function(){var a=b.createElement("div");return"draggable"in a||"ondragstart"in a&&"ondrop"in a},s.websockets=function(){return"WebSocket"in a||"MozWebSocket"in a},s.rgba=function(){return D("background-color:rgba(150,255,150,.5)"),G(j.backgroundColor,"rgba")},s.hsla=function(){return D("background-color:hsla(120,40%,100%,.5)"),G(j.backgroundColor,"rgba")||G(j.backgroundColor,"hsla")},s.multiplebgs=function(){return D("background:url(https://),url(https://),red url(https://)"),/(url\s*\(.*?){3}/.test(j.background)},s.backgroundsize=function(){return J("backgroundSize")},s.borderimage=function(){return J("borderImage")},s.borderradius=function(){return J("borderRadius")},s.boxshadow=function(){return J("boxShadow")},s.textshadow=
function(){return b.createElement("div").style.textShadow===""},s.opacity=function(){return E("opacity:.55"),/^0.55$/.test(j.opacity)},s.cssanimations=function(){return J("animationName")},s.csscolumns=function(){return J("columnCount")},s.cssgradients=function(){var a="background-image:",b="gradient(linear,left top,right bottom,from(#9f9),to(white));",c="linear-gradient(left top,#9f9, white);";return D((a+"-webkit- ".split(" ").join(b+a)+n.join(c+a)).slice(0,-a.length)),G(j.backgroundImage,"gradient")},s.cssreflections=function(){return J("boxReflect")},s.csstransforms=function(){return!!J("transform")},s.csstransforms3d=function(){var a=!!J("perspective");return a&&"webkitPerspective"in g.style&&y("@media (transform-3d),(-webkit-transform-3d){#modernizr{left:9px;position:absolute;height:3px;}}",function(b,c){a=b.offsetLeft===9&&b.offsetHeight===3}),a},s.csstransitions=function(){return J("transition")},s.fontface=function(){var a;return y('@font-face {font-family:"font";src:url("h
ttps://")}',function(c,d){var e=b.getElementById("smodernizr"),f=e.sheet||e.styleSheet,g=f?f.cssRules&&f.cssRules[0]?f.cssRules[0].cssText:f.cssText||"":"";a=/src/i.test(g)&&g.indexOf(d.split(" ")[0])===0}),a},s.generatedcontent=function(){var a;return y(['#modernizr:after{content:"',l,'";visibility:hidden}'].join(""),function(b){a=b.offsetHeight>=1}),a},s.video=function(){var a=b.createElement("video"),c=!1;try{if(c=!!a.canPlayType)c=new Boolean(c),c.ogg=a.canPlayType('video/ogg; codecs="theora"').replace(/^no$/,""),c.h264=a.canPlayType('video/mp4; codecs="avc1.42E01E"').replace(/^no$/,""),c.webm=a.canPlayType('video/webm; codecs="vp8, vorbis"').replace(/^no$/,"")}catch(d){}return c},s.audio=function(){var a=b.createElement("audio"),c=!1;try{if(c=!!a.canPlayType)c=new Boolean(c),c.ogg=a.canPlayType('audio/ogg; codecs="vorbis"').replace(/^no$/,""),c.mp3=a.canPlayType("audio/mpeg;").replace(/^no$/,""),c.wav=a.canPlayType('audio/wav; codecs="1"').replace(/^no$/,""),c.m4a=(a.canPlayTyp
e("audio/x-m4a;")||a.canPlayType("audio/aac;")).replace(/^no$/,"")}catch(d){}return c},s.localstorage=function(){try{return localStorage.setItem(h,h),localStorage.removeItem(h),!0}catch(a){return!1}},s.sessionstorage=function(){try{return sessionStorage.setItem(h,h),sessionStorage.removeItem(h),!0}catch(a){return!1}},s.webworkers=function(){return!!a.Worker},s.applicationcache=function(){return!!a.applicationCache},s.svg=function(){return!!b.createElementNS&&!!b.createElementNS(r.svg,"svg").createSVGRect},s.inlinesvg=function(){var a=b.createElement("div");return a.innerHTML="<svg/>",(a.firstChild&&a.firstChild.namespaceURI)==r.svg},s.smil=function(){return!!b.createElementNS&&/SVGAnimate/.test(m.call(b.createElementNS(r.svg,"animate")))},s.svgclippaths=function(){return!!b.createElementNS&&/SVGClipPath/.test(m.call(b.createElementNS(r.svg,"clipPath")))};for(var L in s)C(s,L)&&(x=L.toLowerCase(),e[x]=s[L](),v.push((e[x]?"":"no-")+x));return e.input||K(),e.addTest=function(a,b){if(ty
peof a=="object")for(var d in a)C(a,d)&&e.addTest(d,a[d]);else{a=a.toLowerCase();if(e[a]!==c)return e;b=typeof b=="function"?b():b,f&&(g.className+=" "+(b?"":"no-")+a),e[a]=b}return e},D(""),i=k=null,function(a,b){function k(a,b){var c=a.createElement("p"),d=a.getElementsByTagName("head")[0]||a.documentElement;return c.innerHTML="x<style>"+b+"</style>",d.insertBefore(c.lastChild,d.firstChild)}function l(){var a=r.elements;return typeof a=="string"?a.split(" "):a}function m(a){var b=i[a[g]];return b||(b={},h++,a[g]=h,i[h]=b),b}function n(a,c,f){c||(c=b);if(j)return c.createElement(a);f||(f=m(c));var g;return f.cache[a]?g=f.cache[a].cloneNode():e.test(a)?g=(f.cache[a]=f.createElem(a)).cloneNode():g=f.createElem(a),g.canHaveChildren&&!d.test(a)?f.frag.appendChild(g):g}function o(a,c){a||(a=b);if(j)return a.createDocumentFragment();c=c||m(a);var d=c.frag.cloneNode(),e=0,f=l(),g=f.length;for(;e<g;e++)d.createElement(f[e]);return d}function p(a,b){b.cache||(b.cache={},b.createElem=a.creat
eElement,b.createFrag=a.createDocumentFragment,b.frag=b.createFrag()),a.createElement=function(c){return r.shivMethods?n(c,a,b):b.createElem(c)},a.createDocumentFragment=Function("h,f","return function(){var n=f.cloneNode(),c=n.createElement;h.shivMethods&&("+l().join().replace(/\w+/g,function(a){return b.createElem(a),b.frag.createElement(a),'c("'+a+'")'})+");return n}")(r,b.frag)}function q(a){a||(a=b);var c=m(a);return r.shivCSS&&!f&&!c.hasCSS&&(c.hasCSS=!!k(a,"article,aside,figcaption,figure,footer,header,hgroup,nav,section{display:block}mark{background:#FF0;color:#000}")),j||p(a,c),a}var c=a.html5||{},d=/^<|^(?:button|map|select|textarea|object|iframe|option|optgroup)$/i,e=/^<|^(?:a|b|button|code|div|fieldset|form|h1|h2|h3|h4|h5|h6|i|iframe|img|input|label|li|link|ol|option|p|param|q|script|select|span|strong|style|table|tbody|td|textarea|tfoot|th|thead|tr|ul)$/i,f,g="_html5shiv",h=0,i={},j;(function(){try{var a=b.createElement("a");a.innerHTML="<xyz></xyz>",f="hidden"in a,j=a.
childNodes.length==1||function(){b.createElement("a");var a=b.createDocumentFragment();return typeof a.cloneNode=="undefined"||typeof a.createDocumentFragment=="undefined"||typeof a.createElement=="undefined"}()}catch(c){f=!0,j=!0}})();var r={elements:c.elements||"abbr article aside audio bdi canvas data datalist details figcaption figure footer header hgroup mark meter nav output progress section summary time video",shivCSS:c.shivCSS!==!1,supportsUnknownElements:j,shivMethods:c.shivMethods!==!1,type:"default",shivDocument:q,createElement:n,createDocumentFragment:o};a.html5=r,q(b)}(this,b),e._version=d,e._prefixes=n,e._domPrefixes=q,e._cssomPrefixes=p,e.mq=z,e.hasEvent=A,e.testProp=function(a){return H([a])},e.testAllProps=J,e.testStyles=y,e.prefixed=function(a,b,c){return b?J(a,b,c):J(a,"pfx")},g.className=g.className.replace(/(^|\s)no-js(\s|$)/,"$1$2")+(f?" js "+v.join(" "):""),e}(this,this.document),function(a,b,c){function d(a){return o.call(a)=="[object Function]"}function e(a)
{return typeof a=="string"}function f(){}function g(a){return!a||a=="loaded"||a=="complete"||a=="uninitialized"}function h(){var a=p.shift();q=1,a?a.t?m(function(){(a.t=="c"?B.injectCss:B.injectJs)(a.s,0,a.a,a.x,a.e,1)},0):(a(),h()):q=0}function i(a,c,d,e,f,i,j){function k(b){if(!o&&g(l.readyState)&&(u.r=o=1,!q&&h(),l.onload=l.onreadystatechange=null,b)){a!="img"&&m(function(){t.removeChild(l)},50);for(var d in y[c])y[c].hasOwnProperty(d)&&y[c][d].onload()}}var j=j||B.errorTimeout,l={},o=0,r=0,u={t:d,s:c,e:f,a:i,x:j};y[c]===1&&(r=1,y[c]=[],l=b.createElement(a)),a=="object"?l.data=c:(l.src=c,l.type=a),l.width=l.height="0",l.onerror=l.onload=l.onreadystatechange=function(){k.call(this,r)},p.splice(e,0,u),a!="img"&&(r||y[c]===2?(t.insertBefore(l,s?null:n),m(k,j)):y[c].push(l))}function j(a,b,c,d,f){return q=0,b=b||"j",e(a)?i(b=="c"?v:u,a,b,this.i++,c,d,f):(p.splice(this.i++,0,a),p.length==1&&h()),this}function k(){var a=B;return a.loader={load:j,i:0},a}var l=b.documentElement,m=a.setTi
meout,n=b.getElementsByTagName("script")[0],o={}.toString,p=[],q=0,r="MozAppearance"in l.style,s=r&&!!b.createRange().compareNode,t=s?l:n.parentNode,l=a.opera&&o.call(a.opera)=="[object Opera]",l=!!b.attachEvent&&!l,u=r?"object":l?"script":"img",v=l?"script":u,w=Array.isArray||function(a){return o.call(a)=="[object Array]"},x=[],y={},z={timeout:function(a,b){return b.length&&(a.timeout=b[0]),a}},A,B;B=function(a){function b(a){var a=a.split("!"),b=x.length,c=a.pop(),d=a.length,c={url:c,origUrl:c,prefixes:a},e,f,g;for(f=0;f<d;f++)g=a[f].split("="),(e=z[g.shift()])&&(c=e(c,g));for(f=0;f<b;f++)c=x[f](c);return c}function g(a,e,f,g,i){var j=b(a),l=j.autoCallback;j.url.split(".").pop().split("?").shift(),j.bypass||(e&&(e=d(e)?e:e[a]||e[g]||e[a.split("/").pop().split("?")[0]]||h),j.instead?j.instead(a,e,f,g,i):(y[j.url]?j.noexec=!0:y[j.url]=1,f.load(j.url,j.forceCSS||!j.forceJS&&"css"==j.url.split(".").pop().split("?").shift()?"c":c,j.noexec,j.attrs,j.timeout),(d(e)||d(l))&&f.load(functio
n(){k(),e&&e(j.origUrl,i,g),l&&l(j.origUrl,i,g),y[j.url]=2})))}function i(a,b){function c(a,c){if(a){if(e(a))c||(j=function(){var a=[].slice.call(arguments);k.apply(this,a),l()}),g(a,j,b,0,h);else if(Object(a)===a)for(n in m=function(){var b=0,c;for(c in a)a.hasOwnProperty(c)&&b++;return b}(),a)a.hasOwnProperty(n)&&(!c&&!--m&&(d(j)?j=function(){var a=[].slice.call(arguments);k.apply(this,a),l()}:j[n]=function(a){return function(){var b=[].slice.call(arguments);a&&a.apply(this,b),l()}}(k[n])),g(a[n],j,b,n,h))}else!c&&l()}var h=!!a.test,i=a.load||a.both,j=a.callback||f,k=j,l=a.complete||f,m,n;c(h?a.yep:a.nope,!!i),i&&c(i)}var j,l,m=this.yepnope.loader;if(e(a))g(a,0,m,0);else if(w(a))for(j=0;j<a.length;j++)l=a[j],e(l)?g(l,0,m,0):w(l)?B(l):Object(l)===l&&i(l,m);else Object(a)===a&&i(a,m)},B.addPrefix=function(a,b){z[a]=b},B.addFilter=function(a){x.push(a)},B.errorTimeout=1e4,b.readyState==null&&b.addEventListener&&(b.readyState="loading",b.addEventListener("DOMContentLoaded",A=function(
){b.removeEventListener("DOMContentLoaded",A,0),b.readyState="complete"},0)),a.yepnope=k(),a.yepnope.executeStack=h,a.yepnope.injectJs=function(a,c,d,e,i,j){var k=b.createElement("script"),l,o,e=e||B.errorTimeout;k.src=a;for(o in d)k.setAttribute(o,d[o]);c=j?h:c||f,k.onreadystatechange=k.onload=function(){!l&&g(k.readyState)&&(l=1,c(),k.onload=k.onreadystatechange=null)},m(function(){l||(l=1,c(1))},e),i?k.onload():n.parentNode.insertBefore(k,n)},a.yepnope.injectCss=function(a,c,d,e,g,i){var e=b.createElement("link"),j,c=i?h:c||f;e.href=a,e.rel="stylesheet",e.type="text/css";for(j in d)e.setAttribute(j,d[j]);g||(n.parentNode.insertBefore(e,n),m(c,0))}}(this,document),Modernizr.load=function(){yepnope.apply(window,[].slice.call(arguments,0))};
+
+/*! matchMedia() polyfill - Test a CSS media type/query in JS. Authors & copyright (c) 2012: Scott Jehl, Paul Irish, Nicholas Zakas. Dual MIT/BSD license */
+/*! NOTE: If you're already including a window.matchMedia polyfill via Modernizr or otherwise, you don't need this part */
+window.matchMedia=window.matchMedia||(function(e,f){var c,a=e.documentElement,b=a.firstElementChild||a.firstChild,d=e.createElement("body"),g=e.createElement("div");g.id="mq-test-1";g.style.cssText="position:absolute;top:-100em";d.style.background="none";d.appendChild(g);return function(h){g.innerHTML='­<style media="'+h+'"> #mq-test-1 { width: 42px; }</style>';a.insertBefore(d,b);c=g.offsetWidth==42;a.removeChild(d);return{matches:c,media:h}}})(document);
+
+/*! Respond.js v1.1.0: min/max-width media query polyfill. (c) Scott Jehl. MIT/GPLv2 Lic. j.mp/respondjs */
+(function(e){e.respond={};respond.update=function(){};respond.mediaQueriesSupported=e.matchMedia&&e.matchMedia("only all").matches;if(respond.mediaQueriesSupported){return}var w=e.document,s=w.documentElement,i=[],k=[],q=[],o={},h=30,f=w.getElementsByTagName("head")[0]||s,g=w.getElementsByTagName("base")[0],b=f.getElementsByTagName("link"),d=[],a=function(){var D=b,y=D.length,B=0,A,z,C,x;for(;B<y;B++){A=D[B],z=A.href,C=A.media,x=A.rel&&A.rel.toLowerCase()==="stylesheet";if(!!z&&x&&!o[z]){if(A.styleSheet&&A.styleSheet.rawCssText){m(A.styleSheet.rawCssText,z,C);o[z]=true}else{if((!/^([a-zA-Z:]*\/\/)/.test(z)&&!g)||z.replace(RegExp.$1,"").split("/")[0]===e.location.host){d.push({href:z,media:C})}}}}u()},u=function(){if(d.length){var x=d.shift();n(x.href,function(y){m(y,x.href,x.media);o[x.href]=true;u()})}},m=function(I,x,z){var G=I.match(/@media[^\{]+\{([^\{\}]*\{[^\}\{]*\})+/gi),J=G&&G.length||0,x=x.substring(0,x.lastIndexOf("/")),y=function(K){return K.replace(/(url\()['"]?([^\/\)'"
][^:\)'"]+)['"]?(\))/g,"$1"+x+"$2$3")},A=!J&&z,D=0,C,E,F,B,H;if(x.length){x+="/"}if(A){J=1}for(;D<J;D++){C=0;if(A){E=z;k.push(y(I))}else{E=G[D].match(/@media *([^\{]+)\{([\S\s]+?)$/)&&RegExp.$1;k.push(RegExp.$2&&y(RegExp.$2))}B=E.split(",");H=B.length;for(;C<H;C++){F=B[C];i.push({media:F.split("(")[0].match(/(only\s+)?([a-zA-Z]+)\s?/)&&RegExp.$2||"all",rules:k.length-1,hasquery:F.indexOf("(")>-1,minw:F.match(/\(min\-width:[\s]*([\s]*[0-9\.]+)(px|em)[\s]*\)/)&&parseFloat(RegExp.$1)+(RegExp.$2||""),maxw:F.match(/\(max\-width:[\s]*([\s]*[0-9\.]+)(px|em)[\s]*\)/)&&parseFloat(RegExp.$1)+(RegExp.$2||"")})}}j()},l,r,v=function(){var z,A=w.createElement("div"),x=w.body,y=false;A.style.cssText="position:absolute;font-size:1em;width:1em";if(!x){x=y=w.createElement("body");x.style.background="none"}x.appendChild(A);s.insertBefore(x,s.firstChild);z=A.offsetWidth;if(y){s.removeChild(x)}else{x.removeChild(A)}z=p=parseFloat(z);return z},p,j=function(I){var x="clientWidth",B=s[x],H=w.compatMode==="
CSS1Compat"&&B||w.body[x]||B,D={},G=b[b.length-1],z=(new Date()).getTime();if(I&&l&&z-l<h){clearTimeout(r);r=setTimeout(j,h);return}else{l=z}for(var E in i){var K=i[E],C=K.minw,J=K.maxw,A=C===null,L=J===null,y="em";if(!!C){C=parseFloat(C)*(C.indexOf(y)>-1?(p||v()):1)}if(!!J){J=parseFloat(J)*(J.indexOf(y)>-1?(p||v()):1)}if(!K.hasquery||(!A||!L)&&(A||H>=C)&&(L||H<=J)){if(!D[K.media]){D[K.media]=[]}D[K.media].push(k[K.rules])}}for(var E in q){if(q[E]&&q[E].parentNode===f){f.removeChild(q[E])}}for(var E in D){var M=w.createElement("style"),F=D[E].join("\n");M.type="text/css";M.media=E;f.insertBefore(M,G.nextSibling);if(M.styleSheet){M.styleSheet.cssText=F}else{M.appendChild(w.createTextNode(F))}q.push(M)}},n=function(x,z){var y=c();if(!y){return}y.open("GET",x,true);y.onreadystatechange=function(){if(y.readyState!=4||y.status!=200&&y.status!=304){return}z(y.responseText)};if(y.readyState==4){return}y.send(null)},c=(function(){var x=false;try{x=new XMLHttpRequest()}catch(y){x=new ActiveX
Object("Microsoft.XMLHTTP")}return function(){return x}})();a();respond.update=a;function t(){j(true)}if(e.addEventListener){e.addEventListener("resize",t,false)}else{if(e.attachEvent){e.attachEvent("onresize",t)}}})(this);
\ No newline at end of file
[07/47] incubator-systemml git commit: [SYSML-249] Removed old RTC
metadata files
Posted by du...@apache.org.
[SYSML-249] Removed old RTC metadata files
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/a6e038a8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/a6e038a8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/a6e038a8
Branch: refs/heads/gh-pages
Commit: a6e038a89b59f94c0e6fe0b73be4d7a80252c1b2
Parents: 9bb58c6
Author: frreiss <fr...@us.ibm.com>
Authored: Fri Aug 21 09:45:12 2015 -0700
Committer: frreiss <fr...@us.ibm.com>
Committed: Fri Aug 21 09:45:12 2015 -0700
----------------------------------------------------------------------
Algorithms Reference/.jazzignore | 35 -----------------------------------
1 file changed, 35 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a6e038a8/Algorithms Reference/.jazzignore
----------------------------------------------------------------------
diff --git a/Algorithms Reference/.jazzignore b/Algorithms Reference/.jazzignore
deleted file mode 100644
index d782c85..0000000
--- a/Algorithms Reference/.jazzignore
+++ /dev/null
@@ -1,35 +0,0 @@
-### Jazz Ignore 0
-# Ignored files and folders will not be committed, but may be modified during
-# accept or update.
-# - Ignore properties should contain a space separated list of filename patterns.
-# - Each pattern is case sensitive and surrounded by braces ('{' and '}').
-# - "*" matches zero or more characters.
-# - "?" matches a single character.
-# - The pattern list may be split across lines by ending the line with a
-# backslash and starting the next line with a tab.
-# - Patterns in core.ignore prevent matching resources in the same
-# directory from being committed.
-# - Patterns in core.ignore.recursive matching resources in the current
-# directory and all subdirectories from being committed.
-# - The default value of core.ignore.recursive is *.class
-# - The default value for core.ignore is bin
-#
-# To ignore shell scripts and hidden files in this subtree:
-# e.g: core.ignore.recursive = {*.sh} {\.*}
-#
-# To ignore resources named 'bin' in the current directory (but allow
-# them in any sub directorybelow):
-# e.g: core.ignore.recursive = {*.sh} {\.*}
-#
-# NOTE: modifying ignore files will not change the ignore status of
-# Eclipse derived resources.
-
-core.ignore.recursive=
-
-core.ignore= \
- {SystemML_Algorithms_Reference.aux} \
- {SystemML_Algorithms_Reference.bbl} \
- {SystemML_Algorithms_Reference.blg} \
- {SystemML_Algorithms_Reference.dvi} \
- {SystemML_Algorithms_Reference.out} \
- {SystemML_Algorithms_Reference.synctex.gz}
\ No newline at end of file
[16/47] incubator-systemml git commit: initial port of documentation
to md, algorithms reference converted to md, template based on spark docs
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/algorithms-descriptive-statistics.md
----------------------------------------------------------------------
diff --git a/algorithms-descriptive-statistics.md b/algorithms-descriptive-statistics.md
new file mode 100644
index 0000000..dd276af
--- /dev/null
+++ b/algorithms-descriptive-statistics.md
@@ -0,0 +1,1321 @@
+---
+layout: global
+title: SystemML Algorithms Reference - Descriptive Statistics
+displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
+---
+
+
+# 1. Descriptive Statistics
+
+Descriptive statistics are used to quantitatively describe the main
+characteristics of the data. They provide meaningful summaries computed
+over different observations or data records collected in a study. These
+summaries typically form the basis of the initial data exploration as
+part of a more extensive statistical analysis. Such a quantitative
+analysis assumes that every variable (also known as, attribute, feature,
+or column) in the data has a specific *level of
+measurement* [[Stevens1946]](algorithms-bibliography.html).
+
+The measurement level of a variable, often called as **variable
+type**, can either be *scale* or *categorical*. A *scale*
+variable represents the data measured on an interval scale or ratio
+scale. Examples of scale variables include ‘Height’, ‘Weight’, ‘Salary’,
+and ‘Temperature’. Scale variables are also referred to as
+*quantitative* or *continuous* variables. In contrast, a *categorical*
+variable has a fixed limited number of distinct values or categories.
+Examples of categorical variables include ‘Gender’, ‘Region’, ‘Hair
+color’, ‘Zipcode’, and ‘Level of Satisfaction’. Categorical variables
+can further be classified into two types, *nominal* and *ordinal*,
+depending on whether the categories in the variable can be ordered via
+an intrinsic ranking. For example, there is no meaningful ranking among
+distinct values in ‘Hair color’ variable, while the categories in ‘Level
+of Satisfaction’ can be ranked from highly dissatisfied to highly
+satisfied.
+
+The input dataset for descriptive statistics is provided in the form of
+a matrix, whose rows are the records (data points) and whose columns are
+the features (i.e. variables). Some scripts allow this matrix to be
+vertically split into two or three matrices. Descriptive statistics are
+computed over the specified features (columns) in the matrix. Which
+statistics are computed depends on the types of the features. It is
+important to keep in mind the following caveats and restrictions:
+
+1. Given a finite set of data records, i.e. a *sample*, we take their
+feature values and compute their *sample statistics*. These statistics
+will vary from sample to sample even if the underlying distribution of
+feature values remains the same. Sample statistics are accurate for the
+given sample only. If the goal is to estimate the *distribution
+statistics* that are parameters of the (hypothesized) underlying
+distribution of the features, the corresponding sample statistics may
+sometimes be used as approximations, but their accuracy will vary.
+
+2. In particular, the accuracy of the estimated distribution statistics
+will be low if the number of values in the sample is small. That is, for
+small samples, the computed statistics may depend on the randomness of
+the individual sample values more than on the underlying distribution of
+the features.
+
+3. The accuracy will also be low if the sample records cannot be assumed
+mutually independent and identically distributed (i.i.d.), that is,
+sampled at random from the same underlying distribution. In practice,
+feature values in one record often depend on other features and other
+records, including unknown ones.
+
+4. Most of the computed statistics will have low estimation accuracy in the
+presence of extreme values (outliers) or if the underlying distribution
+has heavy tails, for example obeys a power law. However, a few of the
+computed statistics, such as the median and Spearman’s rank
+correlation coefficient, are *robust* to outliers.
+
+5. Some sample statistics are reported with their *sample standard errors*
+in an attempt to quantify their accuracy as distribution parameter
+estimators. But these sample standard errors, in turn, only estimate the
+underlying distribution’s standard errors and will have low accuracy for
+small or samples, outliers in samples, or heavy-tailed distributions.
+
+6. We assume that the quantitative (scale) feature columns do not contain
+missing values, infinite values, `NaN`s, or coded non-numeric values,
+unless otherwise specified. We assume that each categorical feature
+column contains positive integers from 1 to the number of categories;
+for ordinal features, the natural order on the integers should coincide
+with the order on the categories.
+
+* * *
+
+## 1.1. Univariate Statistics
+
+### Description
+
+*Univariate statistics* are the simplest form of descriptive statistics
+in data analysis. They are used to quantitatively describe the main
+characteristics of each feature in the data. For a given dataset matrix,
+script `Univar-Stats.dml` computes certain univariate
+statistics for each feature column in the matrix. The feature type
+governs the exact set of statistics computed for that feature. For
+example, the statistic *mean* can only be computed on a quantitative
+(scale) feature like ‘Height’ and ‘Temperature’. It does not make sense
+to compute the mean of a categorical attribute like ‘Hair Color’.
+
+
+### Usage
+
+ hadoop jar SystemML.jar -f Univar-Stats.dml -nvargs X=file TYPES=file STATS=file
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read the data matrix $X$ whose columns we want to
+analyze as the features.
+
+**TYPES**: Location (on HDFS) to read the single-row matrix whose $i^{\textrm{th}}$
+column-cell contains the type of the $i^{\textrm{th}}$ feature column
+`X[,i]` in the data matrix. Feature types must be encoded by integer
+numbers: 1 = scale, 2 = nominal, 3 = ordinal.
+
+**STATS**: Location (on HDFS) where the output matrix of computed statistics will
+be stored. The format of the output matrix is defined by
+[**Table 1**](algorithms-descriptive-statistics.html#table1).
+
+
+### Examples
+
+ hadoop jar SystemML.jar -f Univar-Stats.dml -nvargs X=/user/ml/X.mtx TYPES=/user/ml/types.mtx STATS=/user/ml/stats.mtx
+
+
+* * *
+
+<a name="table1" />
+**Table 1**: The output matrix of `Univar-Stats.dml` has one row per
+each univariate statistic and one column per input feature. This table
+lists the meaning of each row. Signs “+” show applicability to scale
+or/and to categorical features.
+
+
+| Row | Name of Statistic | Scale | Category |
+| --- | -------------------------- | :---: | :------: |
+| 1 | Minimum | + | |
+| 2 | Maximum | + | |
+| 3 | Range | + | |
+| 4 | Mean | + | |
+| 5 | Variance | + | |
+| 6 | Standard deviation | + | |
+| 7 | Standard error of mean | + | |
+| 8 | Coefficient of variation | + | |
+| 9 | Skewness | + | |
+| 10 | Kurtosis | + | |
+| 11 | Standard error of skewness | + | |
+| 12 | Standard error of kurtosis | + | |
+| 13 | Median | + | |
+| 14 | Interquartile mean | + | |
+| 15 | Number of categories | | + |
+| 16 | Mode | | + |
+| 17 | Number of modes | | + |
+
+* * *
+
+### Details
+
+Given an input matrix `X`, this script computes the set of all relevant
+univariate statistics for each feature column `X[,i]` in `X`. The list
+of statistics to be computed depends on the *type*, or *measurement
+level*, of each column. The command-line argument points to a vector
+containing the types of all columns. The types must be provided as per
+the following convention: 1 = scale, 2 = nominal,
+3 = ordinal.
+
+Below we list all univariate statistics computed by script
+`Univar-Stats.dml`. The statistics are collected by
+relevance into several groups, namely: central tendency, dispersion,
+shape, and categorical measures. The first three groups contain
+statistics computed for a quantitative (also known as: numerical, scale,
+or continuous) feature; the last group contains the statistics for a
+categorical (either nominal or ordinal) feature.
+
+Let $n$ be the number of data records (rows) with feature values. In
+what follows we fix a column index `idx` and consider sample statistics
+of feature column `X[`$\,$`,`$\,$`idx]`. Let
+$v = (v_1, v_2, \ldots, v_n)$ be the values of `X[`$\,$`,`$\,$`idx]` in
+their original unsorted order:
+$v_i = \texttt{X[}i\texttt{,}\,\texttt{idx]}$. Let
+$v^s = (v^s_1, v^s_2, \ldots, v^s_n)$ be the same values in the sorted
+order, preserving duplicates: $v^s_1 \leq v^s_2 \leq \ldots \leq v^s_n$.
+
+<a name="figure1" />
+**Figure 1**: The computation of quartiles, median, and interquartile mean from the
+empirical distribution function of the 10-point
+sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8}. Each vertical step in
+the graph has height $1{/}n = 0.1$. Values $q_{25\%}$, $q_{50\%}$, and $q_{75\%}$ denote
+the $1^{\textrm{st}}$, $2^{\textrm{nd}}$, and $3^{\textrm{rd}}$ quartiles correspondingly;
+value $\mu$ denotes the median. Values $\phi_1$ and $\phi_2$ show the partial contribution
+of border points (quartiles) $v_3=3.7$ and $v_8=6.4$ into the interquartile mean.
+
+
+
+#### Central Tendency Measures
+
+Sample statistics that describe the location of the quantitative (scale)
+feature distribution, represent it with a single value.
+
+**Mean** (output row 4): The arithmetic average over a sample of a quantitative
+feature. Computed as the ratio between the sum of values and the number
+of values: $\left(\sum_{i=1}^n v_i\right)\!/n$. Example: the mean of
+sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8}
+equals 5.2.
+
+Note that the mean is significantly affected by extreme values in the
+sample and may be misleading as a central tendency measure if the
+feature varies on exponential scale. For example, the mean of {0.01,
+0.1, 1.0, 10.0, 100.0} is 22.222, greater than all the sample values
+except the largest.
+
+**Median** (output row 13): The “middle” value that separates the higher half of the
+sample values (in a sorted order) from the lower half. To compute the
+median, we sort the sample in the increasing order, preserving
+duplicates: $v^s_1 \leq v^s_2 \leq \ldots \leq v^s_n$. If $n$ is odd,
+the median equals $v^s_i$ where $i = (n\,{+}\,1)\,{/}\,2$, same as the
+$50^{\textrm{th}}$ percentile of the sample. If $n$ is even, there are
+two “middle” values $v^s_{n/2}$ and $v^s_{n/2\,+\,1}$, so we compute the
+median as the mean of these two values. (For even $n$ we compute the
+$50^{\textrm{th}}$ percentile as $v^s_{n/2}$, not as the median.)
+Example: the median of sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1,
+6.4, 7.2, 7.8} equals $(5.3\,{+}\,5.7)\,{/}\,2$ ${=}$ 5.5, see
+[**Figure 1**](algorithms-descriptive-statistics.html#figure1).
+
+Unlike the mean, the median is not sensitive to extreme values in the
+sample, i.e. it is robust to outliers. It works better as a measure of
+central tendency for heavy-tailed distributions and features that vary
+on exponential scale. However, the median is sensitive to small sample
+size.
+
+**Interquartile mean** (output row 14): For a sample of a quantitative feature, this is
+the mean of the values greater than or equal to the $1^{\textrm{st}}$
+quartile and less than or equal the $3^{\textrm{rd}}$ quartile. In other
+words, it is a “truncated mean” where the lowest 25$\%$ and the highest
+25$\%$ of the sorted values are omitted in its computation. The two
+“border values”, i.e. the $1^{\textrm{st}}$ and the $3^{\textrm{rd}}$
+quartiles themselves, contribute to this mean only partially. This
+measure is occasionally used as the “robust” version of the mean that is
+less sensitive to the extreme values.*
+
+To compute the measure, we sort the sample in the increasing order,
+preserving duplicates: $v^s_1 \leq v^s_2 \leq \ldots \leq v^s_n$. We set
+$j = \lceil n{/}4 \rceil$ for the $1^{\textrm{st}}$ quartile index and
+$k = \lceil 3n{/}4 \rceil$ for the $3^{\textrm{rd}}$ quartile index,
+then compute the following weighted mean:
+
+$$\frac{1}{3{/}4 - 1{/}4} \left[
+\left(\frac{j}{n} - \frac{1}{4}\right) v^s_j \,\,+
+\sum_{j<i<k} \left(\frac{i}{n} - \frac{i\,{-}\,1}{n}\right) v^s_i
+\,\,+\,\, \left(\frac{3}{4} - \frac{k\,{-}\,1}{n}\right) v^s_k\right]$$
+
+In other words, all sample values between the $1^{\textrm{st}}$ and the
+$3^{\textrm{rd}}$ quartile enter the sum with weights $2{/}n$, times
+their number of duplicates, while the two quartiles themselves enter the
+sum with reduced weights. The weights are proportional to the vertical
+steps in the empirical distribution function of the sample, see
+Figure \[fig:example\_quartiles\] for an illustration. Example: the
+interquartile mean of sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4,
+7.2, 7.8} equals the sum
+$0.1 (3.7\,{+}\,6.4) + 0.2 (4.4\,{+}\,5.3\,{+}\,5.7\,{+}\,6.1)$, which
+equals 5.31.
+
+
+#### Dispersion Measures
+
+Statistics that describe the amount of variation or spread in a
+quantitative (scale) data feature.
+
+**Variance** (output row 5): A measure of dispersion, or spread-out, of sample values
+around their mean, expressed in units that are the square of those of
+the feature itself. Computed as the sum of squared differences between
+the values in the sample and their mean, divided by one less than the
+number of values: $\sum_{i=1}^n (v_i - \bar{v})^2\,/\,(n\,{-}\,1)$ where
+$\bar{v}=\left(\sum_{i=1}^n v_i\right)/n$. Example: the variance of
+sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8}
+equals 3.24. Note that at least two values ($n\geq 2$) are required to
+avoid division by zero. Sample variance is sensitive to outliers, even
+more than the mean.
+
+**Standard deviation** (output row 6): A measure of dispersion around the mean, the
+square root of variance. Computed by taking the square root of the
+sample variance; see *Variance* above on computing the variance.
+Example: the standard deviation of sample {2.2, 3.2, 3.7, 4.4, 5.3,
+5.7, 6.1, 6.4, 7.2, 7.8} equals 1.8. At least two values are required
+to avoid division by zero. Note that standard deviation is sensitive to
+outliers.
+
+Standard deviation is used in conjunction with the mean to determine an
+interval containing a given percentage of the feature values, assuming
+the normal distribution. In a large sample from a normal distribution,
+around 68% of the cases fall within one standard deviation and around
+95% of cases fall within two standard deviations of the mean. For
+example, if the mean age is 45 with a standard deviation of 10, around
+95% of the cases would be between 25 and 65 in a normal distribution.
+
+**Coefficient of variation** (output row 8): The ratio of the standard deviation to the
+mean, i.e. the *relative* standard deviation, of a quantitative feature
+sample. Computed by dividing the sample *standard deviation* by the
+sample *mean*, see above for their computation details. Example: the
+coefficient of variation for sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7,
+6.1, 6.4, 7.2, 7.8} equals 1.8$\,{/}\,$5.2 ${\approx}$ 0.346.
+
+This metric is used primarily with non-negative features such as
+financial or population data. It is sensitive to outliers. Note: zero
+mean causes division by zero, returning infinity or `NaN`. At least two
+values (records) are required to compute the standard deviation.
+
+**Minimum** (output row 1): The smallest value of a quantitative sample, computed as
+$\min v = v^s_1$. Example: the minimum of sample {2.2, 3.2, 3.7, 4.4,
+5.3, 5.7, 6.1, 6.4, 7.2, 7.8} equals 2.2.
+
+**Maximum** (output row 2): The largest value of a quantitative sample, computed as
+$\max v = v^s_n$. Example: the maximum of sample {2.2, 3.2, 3.7, 4.4,
+5.3, 5.7, 6.1, 6.4, 7.2, 7.8} equals 7.8.
+
+**Range** (output row 3): The difference between the largest and the smallest value of
+a quantitative sample, computed as $\max v - \min v = v^s_n - v^s_1$. It
+provides information about the overall spread of the sample values.
+Example: the range of sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4,
+7.2, 7.8} equals 7.8$\,{-}\,$2.2 ${=}$ 5.6.
+
+**Standard error of the mean** (output row 7): A measure of how much the value of the
+sample mean may vary from sample to sample taken from the same
+(hypothesized) distribution of the feature. It helps to roughly bound
+the distribution mean, i.e.the limit of the sample mean as the sample
+size tends to infinity. Under certain assumptions (e.g. normality and
+large sample), the difference between the distribution mean and the
+sample mean is unlikely to exceed 2 standard errors.
+
+The measure is computed by dividing the sample standard deviation by the
+square root of the number of values $n$; see *standard deviation* for
+its computation details. Ensure $n\,{\geq}\,2$ to avoid division by 0.
+Example: for sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2,
+7.8} with the mean of 5.2 the standard error of the mean equals
+1.8$\,{/}\sqrt{10}$ ${\approx}$ 0.569.
+
+Note that the standard error itself is subject to sample randomness. Its
+accuracy as an error estimator may be low if the sample size is small or
+non-i.i.d., if there are outliers, or if the distribution has heavy tails.
+
+
+#### Shape Measures
+
+Statistics that describe the shape and symmetry of the quantitative
+(scale) feature distribution estimated from a sample of its values.
+
+**Skewness** (output row 9): It measures how symmetrically the values of a feature are
+spread out around the mean. A significant positive skewness implies a
+longer (or fatter) right tail, i.e. feature values tend to lie farther
+away from the mean on the right side. A significant negative skewness
+implies a longer (or fatter) left tail. The normal distribution is
+symmetric and has a skewness value of 0; however, its sample skewness is
+likely to be nonzero, just close to zero. As a guideline, a skewness
+value more than twice its standard error is taken to indicate a
+departure from symmetry.
+
+Skewness is computed as the $3^{\textrm{rd}}$ central moment divided by
+the cube of the standard deviation. We estimate the
+$3^{\textrm{rd}}$ central moment as the sum of cubed differences between
+the values in the feature column and their sample mean, divided by the
+number of values: $\sum_{i=1}^n (v_i - \bar{v})^3 / n$ where
+$\bar{v}=\left(\sum_{i=1}^n v_i\right)/n$. The standard deviation is
+computed as described above in *standard deviation*. To avoid division
+by 0, at least two different sample values are required. Example: for
+sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8} with the
+mean of 5.2 and the standard deviation of 1.8 skewness is estimated as
+$-1.0728\,{/}\,1.8^3 \approx -0.184$. Note: skewness is sensitive to
+outliers.
+
+**Standard error in skewness** (output row 11): A measure of how much the sample
+skewness may vary from sample to sample, assuming that the feature is
+normally distributed, which makes its distribution skewness equal 0.
+Given the number $n$ of sample values, the standard error is computed as
+
+$$\sqrt{\frac{6n\,(n-1)}{(n-2)(n+1)(n+3)}}$$
+
+This measure can tell us, for example:
+
+ * If the sample skewness lands within two standard errors from 0, its
+positive or negative sign is non-significant, may just be accidental.
+ * If the sample skewness lands outside this interval, the feature is
+unlikely to be normally distributed.
+
+At least 3 values ($n\geq 3$) are required to avoid arithmetic failure.
+Note that the standard error is inaccurate if the feature distribution
+is far from normal or if the number of samples is small.
+
+**Kurtosis** (output row 10): As a distribution parameter, kurtosis is a measure of the
+extent to which feature values cluster around a central point. In other
+words, it quantifies “peakedness” of the distribution: how tall and
+sharp the central peak is relative to a standard bell curve.
+
+Positive kurtosis (*leptokurtic* distribution) indicates that, relative
+to a normal distribution:
+
+ * Observations cluster more about the center (peak-shaped)
+ * The tails are thinner at non-extreme values
+ * The tails are thicker at extreme values
+
+Negative kurtosis (*platykurtic* distribution) indicates that, relative
+to a normal distribution:
+
+ * Observations cluster less about the center (box-shaped)
+ * The tails are thicker at non-extreme values
+ * The tails are thinner at extreme values
+
+Kurtosis of a normal distribution is zero; however, the sample kurtosis
+(computed here) is likely to deviate from zero.
+
+Sample kurtosis is computed as the $4^{\textrm{th}}$ central moment
+divided by the $4^{\textrm{th}}$ power of the standard deviation,
+minus 3. We estimate the $4^{\textrm{th}}$ central moment as the sum of
+the $4^{\textrm{th}}$ powers of differences between the values in the
+feature column and their sample mean, divided by the number of values:
+$\sum_{i=1}^n (v_i - \bar{v})^4 / n$ where
+$\bar{v}=\left(\sum_{i=1}^n v_i\right)/n$. The standard deviation is
+computed as described above, see *standard deviation*.
+
+Note that kurtosis is sensitive to outliers, and requires at least two
+different sample values. Example: for sample {2.2, 3.2, 3.7, 4.4,
+5.3, 5.7, 6.1, 6.4, 7.2, 7.8} with the mean of 5.2 and the standard
+deviation of 1.8, sample kurtosis equals
+$16.6962\,{/}\,1.8^4 - 3 \approx -1.41$.
+
+**Standard error in kurtosis** (output row 12): A measure of how much the sample
+kurtosis may vary from sample to sample, assuming that the feature is
+normally distributed, which makes its distribution kurtosis equal 0.
+Given the number $n$ of sample values, the standard error is computed as
+
+$$\sqrt{\frac{24n\,(n-1)^2}{(n-3)(n-2)(n+3)(n+5)}}$$
+
+This measure can tell us, for example:
+
+ * If the sample kurtosis lands within two standard errors from 0, its
+positive or negative sign is non-significant, may just be accidental.
+ * If the sample kurtosis lands outside this interval, the feature is
+unlikely to be normally distributed.
+
+At least 4 values ($n\geq 4$) are required to avoid arithmetic failure.
+Note that the standard error is inaccurate if the feature distribution
+is far from normal or if the number of samples is small.
+
+
+#### Categorical Measures
+
+Statistics that describe the sample of a categorical feature, either
+nominal or ordinal. We represent all categories by integers from 1 to
+the number of categories; we call these integers *category IDs*.
+
+**Number of categories** (output row 15): The maximum category ID that occurs in the
+sample. Note that some categories with IDs *smaller* than this
+maximum ID may have no occurrences in the sample, without reducing the
+number of categories. However, any categories with IDs *larger* than the
+maximum ID with no occurrences in the sample will not be counted.
+Example: in sample {1, 3, 3, 3, 3, 4, 4, 5, 7, 7, 7, 7, 8, 8, 8}
+the number of categories is reported as 8. Category IDs 2 and 6, which
+have zero occurrences, are still counted; but if there is a category
+with ID${}=9$ and zero occurrences, it is not counted.
+
+**Mode** (output row 16): The most frequently occurring category value. If several
+values share the greatest frequency of occurrence, then each of them is
+a mode; but here we report only the smallest of these modes. Example: in
+sample {1, 3, 3, 3, 3, 4, 4, 5, 7, 7, 7, 7, 8, 8, 8} the modes are
+3 and 7, with 3 reported.
+
+Computed by counting the number of occurrences for each category, then
+taking the smallest category ID that has the maximum count. Note that
+the sample modes may be different from the distribution modes, i.e. the
+categories whose (hypothesized) underlying probability is the maximum
+over all categories.
+
+**Number of modes** (output row 17): The number of category values that each have the
+largest frequency count in the sample. Example: in sample {1, 3, 3,
+3, 3, 4, 4, 5, 7, 7, 7, 7, 8, 8, 8} there are two category IDs (3
+and 7) that occur the maximum count of 4 times; hence, we return 2.
+
+Computed by counting the number of occurrences for each category, then
+counting how many categories have the maximum count. Note that the
+sample modes may be different from the distribution modes, i.e. the
+categories whose (hypothesized) underlying probability is the maximum
+over all categories.
+
+
+### Returns
+
+The output matrix containing all computed statistics is of size
+$17$ rows and as many columns as in the input matrix `X`. Each row
+corresponds to a particular statistic, according to the convention
+specified in Table \[table:univars\]. The first $14$ statistics are
+applicable for *scale* columns, and the last $3$ statistics are
+applicable for categorical, i.e. nominal and ordinal, columns.
+
+
+* * *
+
+## 1.2. Bivariate Statistics
+
+
+### Description
+
+Bivariate statistics are used to quantitatively describe the association
+between two features, such as test their statistical (in-)dependence or
+measure the accuracy of one data feature predicting the other feature,
+in a sample. The `bivar-stats.dml` script computes common
+bivariate statistics, such as Pearson’s correlation
+coefficient and Pearson’s $\chi^2$, in parallel for
+many pairs of data features. For a given dataset matrix, script
+`bivar-stats.dml` computes certain bivariate statistics for
+the given feature (column) pairs in the matrix. The feature types govern
+the exact set of statistics computed for that pair. For example,
+Pearson’s correlation coefficient can only be computed on
+two quantitative (scale) features like ‘Height’ and ‘Temperature’. It
+does not make sense to compute the linear correlation of two categorical
+attributes like ‘Hair Color’.
+
+
+### Usage
+
+ hadoop jar SystemML.jar -f bivar-stats.dml -nvargs X=file index1=file index2=file types1=file types2=file OUTDIR=directory
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read the data matrix $X$ whose columns are the
+features that we want to compare and correlate with bivariate
+statistics.
+
+**index1**: Location (on HDFS) to read the single-row matrix that lists the column
+indices of the *first-argument* features in pairwise statistics. Its
+$i^{\textrm{th}}$ entry (i.e. $i^{\textrm{th}}$ column-cell) contains
+the index $k$ of column `X[,k]` in the data matrix whose bivariate
+statistics need to be computed.
+
+**index2**: Location (on HDFS) to read the single-row matrix that lists the column
+indices of the *second-argument* features in pairwise statistics. Its
+$j^{\textrm{th}}$ entry (i.e. $j^{\textrm{th}}$ column-cell) contains
+the index $l$ of column `X[,l]` in the data matrix whose bivariate
+statistics need to be computed.
+
+**types1**: Location (on HDFS) to read the single-row matrix that lists the *types*
+of the *first-argument* features in pairwise statistics. Its
+$i^{\textrm{th}}$ entry (i.e. $i^{\textrm{th}}$ column-cell) contains
+the type of column `X[,k]` in the data matrix, where $k$ is the
+$i^{\textrm{th}}$ entry in the index1 matrix. Feature types
+must be encoded by integer numbers:1 = scale, 2 = nominal,
+3 = ordinal.
+
+**types2**: Location (on HDFS) to read the single-row matrix that lists the *types*
+of the *second-argument* features in pairwise statistics. Its
+$j^{\textrm{th}}$ entry (i.e. $j^{\textrm{th}}$ column-cell) contains
+the type of column `X[,l]` in the data matrix, where $l$ is the
+$j^{\textrm{th}}$ entry in the index2 matrix. Feature types
+must be encoded by integer numbers: 1 = scale, 2 = nominal,
+3 = ordinal.
+
+**OUTDIR**: Location path (on HDFS) where the output matrices with computed
+bivariate statistics will be stored. The matrices’ file names and format
+are defined in [**Table 2**](algorithms-descriptive-statistics.html#table2).
+
+
+### Examples
+
+ hadoop jar SystemML.jar -f bivar-stats.dml -nvargs X=/user/ml/X.mtx index1=/user/ml/S1.mtx index2=/user/ml/S2.mtx types1=/user/ml/K1.mtx types2=/user/ml/K2.mtx OUTDIR=/user/ml/stats.mtx
+
+
+* * *
+
+<a name="table2" />
+**Table 2**: The output matrices of `bivar-stats.dml` have one row per one bivariate
+statistic and one column per one pair of input features. This table lists
+the meaning of each matrix and each row.
+
+
+| Output File / Matrix | Row | Name of Statistic |
+| --------------------------- | ----| ------------------|
+| All Files | 1 | 1-st feature column |
+| " | 2 | 2-nd feature column |
+| bivar.scale.scale.stats | 3 | Pearson’s correlation coefficient |
+| bivar.nominal.nominal.stats | 3 | Pearson’s $\chi^2$ |
+| " | 4 | Degrees of freedom |
+| " | 5 | $P\textrm{-}$value of Pearson’s $\chi^2$ |
+| " | 6 | Cramér’s $V$ |
+| bivar.nominal.scale.stats | 3 | Eta statistic |
+| " | 4 | $F$ statistic |
+| bivar.ordinal.ordinal.stats | 3 | Spearman’s rank correlation coefficient |
+
+* * *
+
+
+### Details
+
+Script `bivar-stats.dml` takes an input matrix `X` whose
+columns represent the features and whose rows represent the records of a
+data sample. Given `X`, the script computes certain relevant bivariate
+statistics for specified pairs of feature columns `X[,i]` and
+`X[,j]`. Command-line parameters `index1` and `index2` specify the
+files with column pairs of interest to the user. Namely, the file given
+by `index1` contains the vector of the 1st-attribute column indices and
+the file given by `index2` has the vector of the 2nd-attribute column
+indices, with “1st” and “2nd” referring to their places in bivariate
+statistics. Note that both `index1` and `index2` files should contain a
+1-row matrix of positive integers.
+
+The bivariate statistics to be computed depend on the *types*, or
+*measurement levels*, of the two columns. The types for each pair are
+provided in the files whose locations are specified by `types1` and
+`types2` command-line parameters. These files are also 1-row matrices,
+i.e. vectors, that list the 1st-attribute and the 2nd-attribute column
+types in the same order as their indices in the `index1` and `index2`
+files. The types must be provided as per the following convention:
+1 = scale, 2 = nominal, 3 = ordinal.
+
+The script orgainizes its results into (potentially) four output
+matrices, one per each type combination. The types of bivariate
+statistics are defined using the types of the columns that were used for
+their arguments, with “ordinal” sometimes retrogressing to “nominal.”
+[**Table 2**](algorithms-descriptive-statistics.html#table2)
+describes what each column in each output matrix
+contains. In particular, the script includes the following statistics:
+
+ * For a pair of scale (quantitative) columns, Pearson’s correlation coefficient.
+ * For a pair of nominal columns (with finite-sized, fixed, unordered
+domains), the Pearson’s $\chi^2$ and its p-value.
+ * For a pair of one scale column and one nominal column, $F$ statistic.
+ * For a pair of ordinal columns (ordered domains depicting ranks),
+Spearman’s rank correlation coefficient.
+
+Note that, as shown in [**Table 2**](algorithms-descriptive-statistics.html#table2), the output matrices
+contain the column indices of the features involved in each statistic.
+Moreover, if the output matrix does not contain a value in a certain
+cell then it should be interpreted as a $0$ (sparse matrix
+representation).
+
+Below we list all bivariate statistics computed by script
+`bivar-stats.dml`. The statistics are collected into
+several groups by the type of their input features. We refer to the two
+input features as $v_1$ and $v_2$ unless specified otherwise; the value
+pairs are $(v_{1,i}, v_{2,i})$ for $i=1,\ldots,n$, where $n$ is the
+number of rows in `X`, i.e. the sample size.
+
+
+#### Scale-vs-Scale Statistics
+
+Sample statistics that describe association between two quantitative
+(scale) features. A scale feature has numerical values, with the natural
+ordering relation.
+
+*Pearson’s correlation coefficient*: A measure of linear
+dependence between two numerical features:
+
+$$r
+= \frac{Cov(v_1, v_2)}{\sqrt{Var v_1 Var v_2}}
+= \frac{\sum_{i=1}^n (v_{1,i} - \bar{v}_1) (v_{2,i} - \bar{v}_2)}{\sqrt{\sum_{i=1}^n (v_{1,i} - \bar{v}_1)^{2\mathstrut} \cdot \sum_{i=1}^n (v_{2,i} - \bar{v}_2)^{2\mathstrut}}}
+$$
+
+Commonly denoted by $r$, correlation ranges between $-1$ and $+1$,
+reaching ${\pm}1$ when all value pairs $(v_{1,i}, v_{2,i})$ lie on the
+same line. Correlation near 0 means that a line is not a good way to
+represent the dependence between the two features; however, this does
+not imply independence. The sign indicates direction of the linear
+association: $r > 0$ ($r < 0$) if one feature tends to linearly increase
+(decrease) when the other feature increases. Nonlinear association, if
+present, may disobey this sign. Pearson’s correlation
+coefficient is symmetric: $r(v_1, v_2) = r(v_2, v_1)$; it does
+not change if we transform $v_1$ and $v_2$ to $a + b v_1$ and
+$c + d v_2$ where $a, b, c, d$ are constants and $b, d > 0$.
+
+Suppose that we use simple linear regression to represent one feature
+given the other, say represent $v_{2,i} \approx \alpha + \beta v_{1,i}$
+by selecting $\alpha$ and $\beta$ to minimize the least-squares error
+$\sum_{i=1}^n (v_{2,i} - \alpha - \beta v_{1,i})^2$. Then the best error
+equals
+
+$$\min_{\alpha, \beta} \,\,\sum_{i=1}^n \big(v_{2,i} - \alpha - \beta v_{1,i}\big)^2 \,\,=\,\,
+(1 - r^2) \,\sum_{i=1}^n \big(v_{2,i} - \bar{v}_2\big)^2$$
+
+In other words, $1\,{-}\,r^2$ is the ratio of the residual sum of squares to the
+total sum of squares. Hence, $r^2$ is an accuracy measure of the linear
+regression.
+
+
+#### Nominal-vs-Nominal Statistics
+
+Sample statistics that describe association between two nominal
+categorical features. Both features’ value domains are encoded with
+positive integers in arbitrary order: nominal features do not order
+their value domains.
+
+*Pearson’s $\chi^2$*: A measure of how much the
+frequencies of value pairs of two categorical features deviate from
+statistical independence. Under independence, the probability of every
+value pair must equal the product of probabilities of each value in the
+pair: $Prob[a, b] - Prob[a]Prob[b] = 0$.
+But we do not know these (hypothesized) probabilities; we only know the
+sample frequency counts. Let $n_{a,b}$ be the frequency count of pair
+$(a, b)$, let $n_a$ and $n_b$ be the frequency counts of $a$ alone and
+of $b$ alone. Under independence, difference
+$n_{a,b}{/}n - (n_a{/}n)(n_b{/}n)$ is unlikely to be exactly 0 due to
+sample randomness, yet it is unlikely to be too far from 0. For some
+pairs $(a,b)$ it may deviate from 0 farther than for other pairs.
+Pearson’s $\chi^2$ is an aggregate measure that combines
+squares of these differences across all value pairs:
+
+$$\chi^2 \,\,=\,\, \sum_{a,\,b} \Big(\frac{n_a n_b}{n}\Big)^{-1} \Big(n_{a,b} - \frac{n_a n_b}{n}\Big)^2
+\,=\,\, \sum_{a,\,b} \frac{(O_{a,b} - E_{a,b})^2}{E_{a,b}}$$
+
+where $O_{a,b} = n_{a,b}$ are the *observed* frequencies and
+$E_{a,b} = (n_a n_b){/}n$ are the *expected* frequencies for all
+pairs $(a,b)$. Under independence (plus other standard assumptions) the
+sample $\chi^2$ closely follows a well-known distribution, making it a
+basis for statistical tests for independence,
+see *$P\textrm{-}$value of Pearson’s $\chi^2$* for details.
+Note that Pearson’s $\chi^2$ does *not* measure the
+strength of dependence: even very weak dependence may result in a
+significant deviation from independence if the counts are large enough.
+Use Cramér’s $V$ instead to measure the strength of
+dependence.
+
+*Degrees of freedom*: An integer parameter required for the
+interpretation of Pearson’s $\chi^2$ measure. Under
+independence (plus other standard assumptions) the sample $\chi^2$
+statistic is approximately distributed as the sum of $d$ squares of
+independent normal random variables with mean 0 and variance 1, where
+$d$ is this integer parameter. For a pair of categorical features such
+that the $1^{\textrm{st}}$ feature has $k_1$ categories and the
+$2^{\textrm{nd}}$ feature has $k_2$ categories, the number of degrees of
+freedom is $d = (k_1 - 1)(k_2 - 1)$.
+
+*$P\textrm{-}$value of Pearson’s $\chi^2$*: A measure of
+how likely we would observe the current frequencies of value pairs of
+two categorical features assuming their statistical independence. More
+precisely, it computes the probability that the sum of $d$ squares of
+independent normal random variables with mean 0 and variance 1 (called
+the $\chi^2$ distribution with $d$ degrees of freedom) generates a value
+at least as large as the current sample Pearson’s $\chi^2$.
+The $d$ parameter is *degrees of freedom*, see above. Under independence
+(plus other standard assumptions) the sample
+Pearson’s $\chi^2$ closely follows the
+$\chi^2$ distribution and is unlikely to land very far into its tail. On
+the other hand, if the two features are dependent, their sample
+Pearson’s $\chi^2$ becomes arbitrarily large as
+$n\to\infty$ and lands extremely far into the tail of the
+$\chi^2$ distribution given a large enough data sample.
+$P\textrm{-}$value of Pearson’s $\chi^2$ returns the tail
+“weight” on the right-hand side of Pearson’s $\chi^2$:
+
+$$P = Prob\big[r \geq \textrm{Pearson’s $\chi^2$} \big|\,\, r \sim \textrm{the $\chi^2$ distribution}\big]$$
+
+As any probability, $P$ ranges between 0 and 1. If $P\leq 0.05$, the
+dependence between the two features may be considered statistically
+significant (i.e. their independence is considered statistically ruled
+out). For highly dependent features, it is not unusual to have
+$P\leq 10^{-20}$ or less, in which case our script will simply return
+$P = 0$. Independent features should have their $P\geq 0.05$ in about
+95% cases.
+
+*Cramér’s $V$*: A measure for the strength of
+association, i.e. of statistical dependence, between two categorical
+features, conceptually similar to Pearson’s correlation
+coefficient. It divides the
+observed Pearson’s $\chi^2$ by the maximum
+possible $\chi^2_{\textrm{max}}$ given $n$ and the number $k_1, k_2$ of
+categories in each feature, then takes the square root. Thus,
+Cramér’s $V$ ranges from 0 to 1, where 0 implies no
+association and 1 implies the maximum possible association (one-to-one
+correspondence) between the two features. See
+*Pearson’s $\chi^2$* for the computation of $\chi^2$; its
+maximum = $n\cdot\min\\{k_1\,{-}\,1, k_2\,{-}\,1\\}$ where the
+$1^{\textrm{st}}$ feature has $k_1$ categories and the
+$2^{\textrm{nd}}$ feature has $k_2$
+categories
+[[AcockStavig1979]](algorithms-bibliography.html), so
+
+$$\textrm{Cramér’s $V$} \,\,=\,\, \sqrt{\frac{\textrm{Pearson’s $\chi^2$}}{n\cdot\min\{k_1\,{-}\,1, k_2\,{-}\,1\}}}$$
+
+As opposed to $P\textrm{-}$value of Pearson’s $\chi^2$,
+which goes to 0 (rapidly) as the features’ dependence increases,
+Cramér’s $V$ goes towards 1 (slowly) as the dependence
+increases. Both Pearson’s $\chi^2$ and
+$P\textrm{-}$value of Pearson’s $\chi^2$ are very sensitive
+to $n$, but in Cramér’s $V$ this is mitigated by taking the
+ratio.
+
+
+#### Nominal-vs-Scale Statistics
+
+Sample statistics that describe association between a categorical
+feature (order ignored) and a quantitative (scale) feature. The values
+of the categorical feature must be coded as positive integers.
+
+*Eta statistic*: A measure for the strength of
+association (statistical dependence) between a nominal feature and a
+scale feature, conceptually similar to Pearson’s correlation
+coefficient. Ranges from 0 to 1, approaching 0 when there is no
+association and approaching 1 when there is a strong association. The
+nominal feature, treated as the independent variable, is assumed to have
+relatively few possible values, all with large frequency counts. The
+scale feature is treated as the dependent variable. Denoting the nominal
+feature by $x$ and the scale feature by $y$, we have:
+
+$$\eta^2 \,=\, 1 - \frac{\sum_{i=1}^{n} \big(y_i - \hat{y}[x_i]\big)^2}{\sum_{i=1}^{n} (y_i - \bar{y})^2},
+\,\,\,\,\textrm{where}\,\,\,\,
+\hat{y}[x] = \frac{1}{\mathop{\mathrm{freq}}(x)}\sum_{i=1}^n
+\,\left\{\!\!\begin{array}{rl} y_i & \textrm{if $x_i = x$}\\ 0 & \textrm{otherwise}\end{array}\right.\!\!\!$$
+
+and $\bar{y} = (1{/}n)\sum_{i=1}^n y_i$ is the mean. Value $\hat{y}[x]$
+is the average of $y_i$ among all records where $x_i = x$; it can also
+be viewed as the “predictor” of $y$ given $x$. Then
+$\sum_{i=1}^{n} (y_i - \hat{y}[x_i])^2$ is the residual error
+sum-of-squares and $\sum_{i=1}^{n} (y_i - \bar{y})^2$ is the total
+sum-of-squares for $y$. Hence, $\eta^2$ measures the accuracy of
+predicting $y$ with $x$, just like the “R-squared” statistic measures
+the accuracy of linear regression. Our output $\eta$ is the square root
+of $\eta^2$.
+
+*$F$ statistic*: A measure of how much the values of the
+scale feature, denoted here by $y$, deviate from statistical
+independence on the nominal feature, denoted by $x$. The same measure
+appears in the one-way analysis of variance (ANOVA). Like
+Pearson’s $\chi^2$, $F$ statistic is used to
+test the hypothesis that $y$ is independent from $x$, given the
+following assumptions:
+
+ * The scale feature $y$ has approximately normal distribution whose mean
+may depend only on $x$ and variance is the same for all $x$.
+ * The nominal feature $x$ has relatively small value domain with large
+frequency counts, the $x_i$-values are treated as fixed (non-random).
+ * All records are sampled independently of each other.
+
+To compute $F$ statistic, we first compute $\hat{y}[x]$ as
+the average of $y_i$ among all records where $x_i = x$. These
+$\hat{y}[x]$ can be viewed as “predictors” of $y$ given $x$; if $y$ is
+independent on $x$, they should “predict” only the global
+mean $\bar{y}$. Then we form two sums-of-squares:
+
+ * *Residual* sum-of-squares of the “predictor” accuracy:
+$y_i - \hat{y}[x_i]$.
+ * *Explained* sum-of-squares of the “predictor” variability:
+$\hat{y}[x_i] - \bar{y}$.
+
+$F$ statistic is the ratio of the explained sum-of-squares
+to the residual sum-of-squares, each divided by their corresponding
+degrees of freedom:
+
+$$F \,\,=\,\,
+\frac{\sum_{x}\, \mathop{\mathrm{freq}}(x) \, \big(\hat{y}[x] - \bar{y}\big)^2 \,\big/\,\, (k\,{-}\,1)}{\sum_{i=1}^{n} \big(y_i - \hat{y}[x_i]\big)^2 \,\big/\,\, (n\,{-}\,k)} \,\,=\,\,
+\frac{n\,{-}\,k}{k\,{-}\,1} \cdot \frac{\eta^2}{1 - \eta^2}$$
+
+Here $k$
+is the domain size of the nominal feature $x$. The $k$ “predictors” lose
+1 freedom due to their linear dependence with $\bar{y}$; similarly, the
+$n$ $y_i$-s lose $k$ freedoms due to the “predictors”.
+
+The statistic can test if the independence hypothesis of $y$ from $x$ is
+reasonable; more generally (with relaxed normality assumptions) it can
+test the hypothesis that *the mean* of $y$ among records with a
+given $x$ is the same for all $x$. Under this hypothesis
+$F$ statistic has, or approximates, the
+$F(k\,{-}\,1, n\,{-}\,k)$-distribution. But if the mean of $y$ given $x$
+depends on $x$, $F$ statistic becomes arbitrarily large as
+$n\to\infty$ (with $k$ fixed) and lands extremely far into the tail of
+the $F(k\,{-}\,1, n\,{-}\,k)$-distribution given a large enough data
+sample.
+
+
+#### Ordinal-vs-Ordinal Statistics
+
+Sample statistics that describe association between two ordinal
+categorical features. Both features’ value domains are encoded with
+positive integers, so that the natural order of the integers coincides
+with the order in each value domain.
+
+*Spearman’s rank correlation coefficient*: A measure for
+the strength of association (statistical dependence) between two ordinal
+features, conceptually similar to Pearson’s correlation
+coefficient. Specifically, it is Pearson’s correlation
+coefficient applied to the feature vectors in which all values
+are replaced by their ranks, i.e. their positions if the vector is
+sorted. The ranks of identical (duplicate) values are replaced with
+their average rank. For example, in vector $(15, 11, 26, 15, 8)$ the
+value “15” occurs twice with ranks 3 and 4 per the sorted order
+$(8_1, 11_2, 15_3, 15_4, 26_5)$; so, both values are assigned their
+average rank of $3.5 = (3\,{+}\,4)\,{/}\,2$ and the vector is replaced
+by $(3.5,\, 2,\, 5,\, 3.5,\, 1)$.
+
+Our implementation of Spearman’s rank correlation
+coefficient is geared towards features having small value domains
+and large counts for the values. Given the two input vectors, we form a
+contingency table $T$ of pairwise frequency counts, as well as a vector
+of frequency counts for each feature: $f_1$ and $f_2$. Here in
+$T_{i,j}$, $f_{1,i}$, $f_{2,j}$ indices $i$ and $j$ refer to the
+order-preserving integer encoding of the feature values. We use prefix
+sums over $f_1$ and $f_2$ to compute the values’ average ranks:
+$r_{1,i} = \sum_{j=1}^{i-1} f_{1,j} + (f_{1,i}\,{+}\,1){/}2$, and
+analogously for $r_2$. Finally, we compute rank variances for $r_1, r_2$
+weighted by counts $f_1, f_2$ and their covariance weighted by $T$,
+before applying the standard formula for Pearson’s correlation
+coefficient:
+
+$$\rho \,\,=\,\, \frac{Cov_T(r_1, r_2)}{\sqrt{Var_{f_1}(r_1)Var_{f_2}(r_2)}}
+\,\,=\,\, \frac{\sum_{i,j} T_{i,j} (r_{1,i} - \bar{r}_1) (r_{2,j} - \bar{r}_2)}{\sqrt{\sum_i f_{1,i} (r_{1,i} - \bar{r}_1)^{2\mathstrut} \cdot \sum_j f_{2,j} (r_{2,j} - \bar{r}_2)^{2\mathstrut}}}$$
+
+where $\bar{r_1} = \sum_i r_{1,i} f_{1,i}{/}n$, analogously
+for $\bar{r}_2$. The value of $\rho$ lies between $-1$ and $+1$, with
+sign indicating the prevalent direction of the association: $\rho > 0$
+($\rho < 0$) means that one feature tends to increase (decrease) when
+the other feature increases. The correlation becomes 1 when the two
+features are monotonically related.
+
+
+### Returns
+
+A collection of (potentially) 4 matrices. Each matrix contains bivariate
+statistics that resulted from a different combination of feature types.
+There is one matrix for scale-scale statistics (which includes
+Pearson’s correlation coefficient), one for nominal-nominal
+statistics (includes Pearson’s $\chi^2$), one for
+nominal-scale statistics (includes $F$ statistic) and one
+for ordinal-ordinal statistics (includes Spearman’s rank
+correlation coefficient). If any of these matrices is not
+produced, then no pair of columns required the corresponding type
+combination. See
+[**Table 2**](algorithms-descriptive-statistics.html#table2)
+for the matrix naming and format
+details.
+
+
+* * *
+
+## 1.3. Stratified Bivariate Statistics
+
+
+### Description
+
+The `stratstats.dml` script computes common bivariate
+statistics, such as correlation, slope, and their p-value, in parallel
+for many pairs of input variables in the presence of a confounding
+categorical variable. The values of this confounding variable group the
+records into strata (subpopulations), in which all bivariate pairs are
+assumed free of confounding. The script uses the same data model as in
+one-way analysis of covariance (ANCOVA), with strata representing
+population samples. It also outputs univariate stratified and bivariate
+unstratified statistics.
+
+To see how data stratification mitigates confounding, consider an
+(artificial) example in
+[**Table 3**](algorithms-descriptive-statistics.html#table3). A highly seasonal
+retail item was marketed with and without a promotion over the final
+3 months of the year. In each month the sale was more likely with the
+promotion than without it. But during the peak holiday season, when
+shoppers came in greater numbers and bought the item more often, the
+promotion was less frequently used. As a result, if the 4-th quarter
+data is pooled together, the promotion’s effect becomes reversed and
+magnified. Stratifying by month restores the positive correlation.
+
+The script computes its statistics in parallel over all possible pairs
+from two specified sets of covariates. The 1-st covariate is a column in
+input matrix $X$ and the 2-nd covariate is a column in input matrix $Y$;
+matrices $X$ and $Y$ may be the same or different. The columns of
+interest are given by their index numbers in special matrices. The
+stratum column, specified in its own matrix, is the same for all
+covariate pairs.
+
+Both covariates in each pair must be numerical, with the 2-nd covariate
+normally distributed given the 1-st covariate (see Details). Missing
+covariate values or strata are represented by “NaN”. Records with NaN’s
+are selectively omitted wherever their NaN’s are material to the output
+statistic.
+
+
+* * *
+
+<a name="table3" />
+**Table 3**: Stratification example: the effect of the promotion on average sales
+becomes reversed and amplified (from $+0.1$ to $-0.5$) if we ignore the months.
+
+<table>
+ <thead>
+ <tr>
+ <th>Month</th>
+ <th colspan="2">Oct</th>
+ <th colspan="2">Nov</th>
+ <th colspan="2">Dec</th>
+ <th colspan="2">Oct - Dec</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td>Customers (millions)</td>
+ <td>0.6</td>
+ <td>1.4</td>
+ <td>1.4</td>
+ <td>0.6</td>
+ <td>3.0</td>
+ <td>1.0</td>
+ <td>5.0</td>
+ <td>3.0</td>
+ </tr>
+ <tr>
+ <td>Promotions (0 or 1)</td>
+ <td>0</td>
+ <td>1</td>
+ <td>0</td>
+ <td>1</td>
+ <td>0</td>
+ <td>1</td>
+ <td>0</td>
+ <td>1</td>
+ </tr>
+ <tr>
+ <td>Avg sales per 1000</td>
+ <td>0.4</td>
+ <td>0.5</td>
+ <td>0.9</td>
+ <td>1.0</td>
+ <td>2.5</td>
+ <td>2.6</td>
+ <td>1.8</td>
+ <td>1.3</td>
+ </tr>
+ </tbody>
+</table>
+
+* * *
+
+
+### Usage
+
+ hadoop jar SystemML.jar -f stratstats.dml -nvargs X=file Xcid=file Y=file Ycid=file S=file Scid=int O=file fmt=format
+
+
+### Arguments
+
+**X**: Location (on HDFS) to read matrix $X$ whose columns we want to use as
+the 1-st covariate (i.e. as the feature variable)
+
+**Xcid**: (default: " ") Location to read the single-row matrix that lists all index
+numbers of the $X$-columns used as the 1-st covariate; the default value
+means “use all $X$-columns”
+
+**Y**: (default: " ") Location to read matrix $Y$ whose columns we want to use as
+the 2-nd covariate (i.e. as the response variable); the default value
+means “use $X$ in place of $Y$”
+
+**Ycid**: (default: " ") Location to read the single-row matrix that lists all index
+numbers of the $Y$-columns used as the 2-nd covariate; the default value
+means “use all $Y$-columns”
+
+**S**: (default: " ") Location to read matrix $S$ that has the stratum column.
+Note: the stratum column must contain small positive integers; all
+fractional values are rounded; stratum IDs of value ${\leq}\,0$ or NaN
+are treated as missing. The default value for S means “use
+$X$ in place of $S$”
+
+**Scid**: (default: 1) The index number of the stratum column in $S$
+
+**O**: Location to store the output matrix defined in
+[**Table 4**](algorithms-descriptive-statistics.html#table4).
+
+**fmt**: (default: `"text"`) Matrix file output format, such as `text`,
+`mm`, or `csv`; see read/write functions in
+SystemML Language Reference for details.
+
+
+* * *
+
+<a name="table4" />
+**Table 4**: The `stratstats.dml` output matrix has one row per each distinct pair of 1-st and 2-nd covariates, and 40 columns with the statistics described here.
+
+<table>
+ <thead>
+ <tr>
+ <th> </th>
+ <th>Col</th>
+ <th>Meaning</th>
+ <th> </th>
+ <th>Col</th>
+ <th>Meaning</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td rowspan="9">1-st covariate</td>
+ <td>01</td>
+ <td>$X$-column number</td>
+ <td rowspan="9">2-nd covariate</td>
+ <td>11</td>
+ <td>$Y$-column number</td>
+ </tr>
+ <tr>
+ <td>02</td>
+ <td>presence count for $x$</td>
+ <td>12</td>
+ <td>presence count for $y$</td>
+ </tr>
+ <tr>
+ <td>03</td>
+ <td>global mean $(x)$</td>
+ <td>13</td>
+ <td>global mean $(y)$</td>
+ </tr>
+ <tr>
+ <td>04</td>
+ <td>global std. dev. $(x)$</td>
+ <td>14</td>
+ <td>global std. dev. $(y)$</td>
+ </tr>
+ <tr>
+ <td>05</td>
+ <td>stratified std. dev. $(x)$</td>
+ <td>15</td>
+ <td>stratified std. dev. $(y)$</td>
+ </tr>
+ <tr>
+ <td>06</td>
+ <td>$R^2$ for $x \sim $ strata</td>
+ <td>16</td>
+ <td>$R^2$ for $y \sim $ strata</td>
+ </tr>
+ <tr>
+ <td>07</td>
+ <td>adjusted $R^2$ for $x \sim $ strata</td>
+ <td>17</td>
+ <td>adjusted $R^2$ for $y \sim $ strata</td>
+ </tr>
+ <tr>
+ <td>08</td>
+ <td>p-value, $x \sim $ strata</td>
+ <td>18</td>
+ <td>p-value, $y \sim $ strata</td>
+ </tr>
+ <tr>
+ <td>09 - 10</td>
+ <td>reserved</td>
+ <td>19 - 20</td>
+ <td>reserved</td>
+ </tr>
+ <tr>
+ <td rowspan="10">$y \sim x$, NO strata</td>
+ <td>21</td>
+ <td>presence count $(x, y)$</td>
+ <td rowspan="10">$y \sim x$ AND strata</td>
+ <td>31</td>
+ <td>presence count $(x, y, s)$</td>
+ </tr>
+ <tr>
+ <td>22</td>
+ <td>regression slope</td>
+ <td>32</td>
+ <td>regression slope</td>
+ </tr>
+ <tr>
+ <td>23</td>
+ <td>regres. slope std. dev.</td>
+ <td>33</td>
+ <td>regres. slope std. dev.</td>
+ </tr>
+ <tr>
+ <td>24</td>
+ <td>correlation $= \pm\sqrt{R^2}$</td>
+ <td>34</td>
+ <td>correlation $= \pm\sqrt{R^2}$</td>
+ </tr>
+ <tr>
+ <td>25</td>
+ <td>residual std. dev.</td>
+ <td>35</td>
+ <td>residual std. dev.</td>
+ </tr>
+ <tr>
+ <td>26</td>
+ <td>$R^2$ in $y$ due to $x$</td>
+ <td>36</td>
+ <td>$R^2$ in $y$ due to $x$</td>
+ </tr>
+ <tr>
+ <td>27</td>
+ <td>adjusted $R^2$ in $y$ due to $x$</td>
+ <td>37</td>
+ <td>adjusted $R^2$ in $y$ due to $x$</td>
+ </tr>
+ <tr>
+ <td>28</td>
+ <td>p-value for “slope = 0”</td>
+ <td>38</td>
+ <td>p-value for “slope = 0”</td>
+ </tr>
+ <tr>
+ <td>29</td>
+ <td>reserved</td>
+ <td>39</td>
+ <td># strata with ${\geq}\,2$ count</td>
+ </tr>
+ <tr>
+ <td>30</td>
+ <td>reserved</td>
+ <td>40</td>
+ <td>reserved</td>
+ </tr>
+ </tbody>
+</table>
+
+* * *
+
+
+### Examples
+
+ hadoop jar SystemML.jar -f stratstats.dml -nvargs X=/user/ml/X.mtx Xcid=/user/ml/Xcid.mtx Y=/user/ml/Y.mtx Ycid=/user/ml/Ycid.mtx S=/user/ml/S.mtx Scid=2 O=/user/ml/Out.mtx fmt=csv
+
+ hadoop jar SystemML.jar -f stratstats.dml -nvargs X=/user/ml/Data.mtx Xcid=/user/ml/Xcid.mtx Ycid=/user/ml/Ycid.mtx Scid=7 O=/user/ml/Out.mtx
+
+
+### Details
+
+Suppose we have $n$ records of format $(i, x, y)$, where
+$i\in\{1,\ldots, k\}$ is a stratum number and $(x, y)$ are two numerical
+covariates. We want to analyze conditional linear relationship between
+$y$ and $x$ conditioned by $i$. Note that $x$, but not $y$, may
+represent a categorical variable if we assign a numerical value to each
+category, for example 0 and 1 for two categories.
+
+We assume a linear regression model for $y$:
+
+$$y_{i,j} \,=\, \alpha_i + \beta x_{i,j} + {\varepsilon}_{i,j}\,, \quad\textrm{where}\,\,\,\,
+{\varepsilon}_{i,j} \sim Normal(0, \sigma^2)$$
+
+Here $i = 1\ldots k$ is a stratum number and
+$j = 1\ldots n_i$ is a record number in stratum $i$; by $n_i$ we denote
+the number of records available in stratum $i$. The noise
+term $\varepsilon_{i,j}$ is assumed to have the same variance in all
+strata. When $n_i\,{>}\,0$, we can estimate the means of $x_{i, j}$ and
+$y_{i, j}$ in stratum $i$ as
+
+$$\bar{x}_i \,= \Big(\sum\nolimits_{j=1}^{n_i} \,x_{i, j}\Big) / n_i\,;\quad
+\bar{y}_i \,= \Big(\sum\nolimits_{j=1}^{n_i} \,y_{i, j}\Big) / n_i$$
+
+If
+$\beta$ is known, the best estimate for $\alpha_i$ is
+$\bar{y}_i - \beta \bar{x}_i$, which gives the prediction error
+sum-of-squares of
+
+$$\sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(y_{i,j} - \beta x_{i,j} - (\bar{y}_i - \beta \bar{x}_i)\big)^2
+\,\,=\,\, \beta^{2\,}V_x \,-\, 2\beta \,V_{x,y} \,+\, V_y
+\label{eqn:stratsumsq}$$
+
+where $V_x$, $V_y$, and $V_{x, y}$ are,
+correspondingly, the “stratified” sample estimates of variance
+$Var(x)$ and
+$Var(y)$ and covariance
+$Cov(x,y)$ computed as
+
+$$\begin{aligned}
+V_x \,&=\, \sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(x_{i,j} - \bar{x}_i\big)^2; \quad
+V_y \,=\, \sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(y_{i,j} - \bar{y}_i\big)^2;\\
+V_{x,y} \,&=\, \sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(x_{i,j} - \bar{x}_i\big)\big(y_{i,j} - \bar{y}_i\big)\end{aligned}$$
+
+They are stratified because we compute the sample (co-)variances in each
+stratum $i$ separately, then combine by summation. The stratified
+estimates for $Var(X)$ and
+$Var(Y)$ tend to be smaller
+than the non-stratified ones (with the global mean instead of
+$\bar{x_i}$ and $\bar{y_i}$) since $\bar{x_i}$ and $\bar{y_i}$ fit
+closer to $x_{i,j}$ and $y_{i,j}$ than the global means. The stratified
+variance estimates the uncertainty in $x_{i,j}$ and $y_{i,j}$ given
+their stratum $i$.
+
+Minimizing over $\beta$ the error sum-of-squares
+gives us the regression slope estimate $\hat{\beta} = V_{x,y} / V_x$,
+with the error sum-of-squares
+becoming the residual sum-of-squares (RSS):
+
+$$\mathrm{RSS} \,\,=\, \,
+\sum\nolimits_{i=1}^k \sum\nolimits_{j=1}^{n_i} \big(y_{i,j} -
+\hat{\beta} x_{i,j} - (\bar{y}_i - \hat{\beta} \bar{x}_i)\big)^2
+\,\,=\,\, V_y \,\big(1 \,-\, V_{x,y}^2 / (V_x V_y)\big)$$
+
+The quantity
+$\hat{R}^2 = V_{x,y}^2 / (V_x V_y)$, called *$R$-squared*, estimates the
+fraction of stratified variance in $y_{i,j}$ explained by covariate
+$x_{i, j}$ in the linear regression model. We
+define *stratified correlation* as the square root of $\hat{R}^2$ taken
+with the sign of $V_{x,y}$. We also use RSS to estimate the residual
+standard deviation $\sigma$ in the linear regression model that models the
+prediction error of $y_{i,j}$ given $x_{i,j}$ and the stratum:
+
+$$\hat{\beta}\, =\, \frac{V_{x,y}}{V_x}; \,\,\,\, \hat{R} \,=\, \frac{V_{x,y}}{\sqrt{V_x V_y}};
+\,\,\,\, \hat{R}^2 \,=\, \frac{V_{x,y}^2}{V_x V_y};
+\,\,\,\, \hat{\sigma} \,=\, \sqrt{\frac{\mathrm{RSS}}{n - k - 1}}\,\,\,\,
+\Big(n = \sum_{i=1}^k n_i\Big)$$
+
+The $t$-test and the $F$-test for the null-hypothesis of “$\beta = 0$”
+are obtained by considering the effect of $\hat{\beta}$ on the residual
+sum-of-squares, measured by the decrease from $V_y$ to RSS. The
+$F$-statistic is the ratio of the “explained” sum-of-squares to the
+residual sum-of-squares, divided by their corresponding degrees of
+freedom. There are $n\,{-}\,k$ degrees of freedom for $V_y$, parameter
+$\beta$ reduces that to $n\,{-}\,k\,{-}\,1$ for RSS, and their
+difference $V_y - {}$RSS has just 1 degree of freedom:
+
+$$F \,=\, \frac{(V_y - \mathrm{RSS})/1}{\mathrm{RSS}/(n\,{-}\,k\,{-}\,1)}
+\,=\, \frac{\hat{R}^2\,(n\,{-}\,k\,{-}\,1)}{1-\hat{R}^2}; \quad
+t \,=\, \hat{R}\, \sqrt{\frac{n\,{-}\,k\,{-}\,1}{1-\hat{R}^2}}.$$
+
+The
+$t$-statistic is simply the square root of the $F$-statistic with the
+appropriate choice of sign. If the null hypothesis and the linear model
+are both true, the $t$-statistic has Student $t$-distribution with
+$n\,{-}\,k\,{-}\,1$ degrees of freedom. We can also compute it if we
+divide $\hat{\beta}$ by its estimated standard deviation:
+
+$$st.dev(\hat{\beta})_{\mathrm{est}} \,=\, \hat{\sigma}\,/\sqrt{V_x} \quad\Longrightarrow\quad
+t \,=\, \hat{R}\sqrt{V_y} \,/\, \hat{\sigma} \,=\, \beta \,/\, st.dev(\hat{\beta})_{\mathrm{est}}$$
+
+The standard deviation estimate for $\beta$ is included in
+`stratstats.dml` output.
+
+### Returns
+
+The output matrix format is defined in
+[**Table 4**](algorithms-descriptive-statistics.html#table4).
+
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/algorithms-matrix-factorization.md
----------------------------------------------------------------------
diff --git a/algorithms-matrix-factorization.md b/algorithms-matrix-factorization.md
new file mode 100644
index 0000000..2fedfce
--- /dev/null
+++ b/algorithms-matrix-factorization.md
@@ -0,0 +1,143 @@
+---
+layout: global
+title: SystemML Algorithms Reference - Matrix Factorization
+displayTitle: <a href="algorithms-reference.html">SystemML Algorithms Reference</a>
+---
+
+
+# 5 Matrix Factorization
+
+
+## 5.1 Principle Component Analysis
+
+### Description
+
+Principle Component Analysis (PCA) is a simple, non-parametric method to
+transform the given data set with possibly correlated columns into a set
+of linearly uncorrelated or orthogonal columns, called *principle
+components*. The principle components are ordered in such a way
+that the first component accounts for the largest possible variance,
+followed by remaining principle components in the decreasing order of
+the amount of variance captured from the data. PCA is often used as a
+dimensionality reduction technique, where the original data is projected
+or rotated onto a low-dimensional space with basis vectors defined by
+top-$K$ (for a given value of $K$) principle components.
+
+
+### Usage
+
+ hadoop jar SystemML.jar -f PCA.dml
+ -nvargs INPUT=file
+ K=int
+ CENTER=int
+ SCALE=int
+ PROJDATA=int
+ OFMT=format
+ MODEL=file
+ OUTPUT=file
+
+
+#### Arguments
+
+**INPUT**: Location (on HDFS) to read the input matrix.
+
+**K**: Indicates dimension of the new vector space constructed from $K$
+ principle components. It must be a value between $1$ and the number
+ of columns in the input data.
+
+**CENTER**: (default: 0) 0 or 1. Indicates whether or not to
+ *center* input data prior to the computation of
+ principle components.
+
+**SCALE**: (default: 0) 0 or 1. Indicates whether or not to
+ *scale* input data prior to the computation of
+ principle components.
+
+**PROJDATA**: 0 or 1. Indicates whether or not the input data must be projected
+ on to new vector space defined over principle components.
+
+**OFMT**: (default: `"csv"`) Specifies the output format.
+ Choice of comma-separated values (`csv`) or as a sparse-matrix (`text`).
+
+**MODEL**: Either the location (on HDFS) where the computed model is
+ stored; or the location of an existing model.
+
+**OUTPUT**: Location (on HDFS) to store the data rotated on to the new
+ vector space.
+
+
+
+#### Examples
+
+ hadoop jar SystemML.jar -f PCA.dml
+ -nvargs INPUT=/user/ml/input.mtx
+ K=10
+ CENTER=1
+ SCALE=1O
+ FMT=csv
+ PROJDATA=1
+ OUTPUT=/user/ml/pca_output/
+
+ hadoop jar SystemML.jar -f PCA.dml
+ -nvargs INPUT=/user/ml/test_input.mtx
+ K=10
+ CENTER=1
+ SCALE=1O
+ FMT=csv
+ PROJDATA=1
+ MODEL=/user/ml/pca_output/
+ OUTPUT=/user/ml/test_output.mtx
+
+
+
+#### Details
+
+Principle Component Analysis (PCA) is a non-parametric procedure for
+orthogonal linear transformation of the input data to a new coordinate
+system, such that the greatest variance by some projection of the data
+comes to lie on the first coordinate (called the first principal
+component), the second greatest variance on the second coordinate, and
+so on. In other words, PCA first selects a normalized direction in
+$m$-dimensional space ($m$ is the number of columns in the input data)
+along which the variance in input data is maximized – this is referred
+to as the first principle component. It then repeatedly finds other
+directions (principle components) in which the variance is maximized. At
+every step, PCA restricts the search for only those directions that are
+perpendicular to all previously selected directions. By doing so, PCA
+aims to reduce the redundancy among input variables. To understand the
+notion of redundancy, consider an extreme scenario with a data set
+comprising of two variables, where the first one denotes some quantity
+expressed in meters, and the other variable represents the same quantity
+but in inches. Both these variables evidently capture redundant
+information, and hence one of them can be removed. In a general
+scenario, keeping solely the linear combination of input variables would
+both express the data more concisely and reduce the number of variables.
+This is why PCA is often used as a dimensionality reduction technique.
+
+The specific method to compute such a new coordinate system is as
+follows – compute a covariance matrix $C$ that measures the strength of
+correlation among all pairs of variables in the input data; factorize
+$C$ according to eigen decomposition to calculate its eigenvalues and
+eigenvectors; and finally, order eigenvectors in the decreasing order of
+their corresponding eigenvalue. The computed eigenvectors (also known as
+*loadings*) define the new coordinate system and the square
+root of eigen values provide the amount of variance in the input data
+explained by each coordinate or eigenvector.
+
+
+#### Returns
+
+When MODEL is not provided, PCA procedure is
+applied on INPUT data to generate MODEL as well as the rotated data
+OUTPUT (if PROJDATA is set to $1$) in the new coordinate system. The
+produced model consists of basis vectors MODEL$/dominant.eigen.vectors$
+for the new coordinate system; eigen values
+MODEL$/dominant.eigen.values$; and the standard deviation
+MODEL$/dominant.eigen.standard.deviations$ of principle components. When
+MODEL is provided, INPUT data is rotated according to the coordinate
+system defined by MODEL$/dominant.eigen.vectors$. The resulting data is
+stored at location OUTPUT.
+
+
+* * *
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d47e76df/algorithms-reference.md
----------------------------------------------------------------------
diff --git a/algorithms-reference.md b/algorithms-reference.md
new file mode 100644
index 0000000..58d4aa0
--- /dev/null
+++ b/algorithms-reference.md
@@ -0,0 +1,30 @@
+---
+layout: global
+title: SystemML Algorithms Reference
+description: SystemML Algorithms Reference
+displayTitle: SystemML Algorithms Reference
+---
+
+
+* [Descriptive Statistics](algorithms-descriptive-statistics.html)
+ * [Univariate Statistics](algorithms-descriptive-statistics.html#univariate-statistics)
+ * [Bivariate Statistics](algorithms-descriptive-statistics.html#bivariate-statistics)
+ * [Stratified Bivariate Statistics](algorithms-descriptive-statistics.html#stratified-bivariate-statistics)
+
+* [Classification](algorithms-classification.html)
+ * [Multinomial Logistic Regression](algorithms-classification.html#multinomial-logistic-regression)
+ * [Support Vector Machines](algorithms-classification.html#support-vector-machines)
+ * [Binary-Class Support Vector Machines](algorithms-classification.html#binary-class-support-vector-machines)
+ * [Multi-Class Support Vector Machines](algorithms-classification.html#multi-class-support-vector-machines)
+ * [Naive Bayes](algorithms-classification.html#naive-bayes)
+
+* [Clustering](algorithms-clustering.html)
+ * [K-Means Clustering](algorithms-clustering.html#k-means-clustering)
+
+* [Regression](algorithms-regression.html)
+ * [Linear Regression](algorithms-regression.html#linear-regression)
+ * [Generalized Linear Models](algorithms-regression.html#generalized-linear-models)
+ * [Regression Scoring and Prediction](algorithms-regression.html#regression-scoring-and-prediction)
+
+* [Matrix Factorization](algorithms-matrix-factorization.html)
+ * [Principle Component Analysis](algorithms-matrix-factorization.html#principle-component-analysis)
[40/47] incubator-systemml git commit: [SYSML-347] Invocation Tabs on
Algorithms Reference
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b8055201/algorithms-regression.md
----------------------------------------------------------------------
diff --git a/algorithms-regression.md b/algorithms-regression.md
index f1da3c9..e35003b 100644
--- a/algorithms-regression.md
+++ b/algorithms-regression.md
@@ -61,6 +61,8 @@ efficient when the number of features $m$ is relatively small
**Linear Regression - Direct Solve**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f LinearRegDS.dml
-nvargs X=<file>
Y=<file>
@@ -69,10 +71,29 @@ efficient when the number of features $m$ is relatively small
icpt=[int]
reg=[double]
fmt=[format]
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f LinearRegDS.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ B=<file>
+ O=[file]
+ icpt=[int]
+ reg=[double]
+ fmt=[format]
+</div>
+</div>
**Linear Regression - Conjugate Gradient**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f LinearRegCG.dml
-nvargs X=<file>
Y=<file>
@@ -84,6 +105,27 @@ efficient when the number of features $m$ is relatively small
tol=[double]
maxi=[int]
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f LinearRegCG.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ B=<file>
+ O=[file]
+ Log=[file]
+ icpt=[int]
+ reg=[double]
+ tol=[double]
+ maxi=[int]
+ fmt=[format]
+</div>
+</div>
### Arguments
@@ -135,30 +177,71 @@ SystemML Language Reference for details.
**Linear Regression - Direct Solve**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f LinearRegDS.dml
- -nvargs X=/user/ml/X.mtx
- Y=/user/ml/Y.mtx
- B=/user/ml/B.mtx
- fmt=csv
- O=/user/ml/stats.csv
- icpt=2
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ fmt=csv
+ O=/user/ml/stats.csv
+ icpt=2
reg=1.0
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f LinearRegDS.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ fmt=csv
+ O=/user/ml/stats.csv
+ icpt=2
+ reg=1.0
+</div>
+</div>
**Linear Regression - Conjugate Gradient**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f LinearRegCG.dml
- -nvargs X=/user/ml/X.mtx
- Y=/user/ml/Y.mtx
- B=/user/ml/B.mtx
- fmt=csv
- O=/user/ml/stats.csv
- icpt=2
- reg=1.0
- tol=0.00000001
- maxi=100
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ fmt=csv
+ O=/user/ml/stats.csv
+ icpt=2
+ reg=1.0
+ tol=0.00000001
+ maxi=100
Log=/user/ml/log.csv
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f LinearRegCG.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ fmt=csv
+ O=/user/ml/stats.csv
+ icpt=2
+ reg=1.0
+ tol=0.00000001
+ maxi=100
+ Log=/user/ml/log.csv
+</div>
+</div>
* * *
@@ -368,6 +451,8 @@ lowest AIC is computed.
### Usage
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f StepLinearRegDS.dml
-nvargs X=<file>
Y=<file>
@@ -377,7 +462,25 @@ lowest AIC is computed.
icpt=[int]
thr=[double]
fmt=[format]
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f StepLinearRegDS.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ B=<file>
+ S=[file]
+ O=[file]
+ icpt=[int]
+ thr=[double]
+ fmt=[format]
+</div>
+</div>
### Arguments
@@ -419,6 +522,8 @@ SystemML Language Reference for details.
### Examples
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f StepLinearRegDS.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/Y.mtx
@@ -428,6 +533,25 @@ SystemML Language Reference for details.
icpt=2
thr=0.05
fmt=csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f StepLinearRegDS.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ S=/user/ml/selected.csv
+ O=/user/ml/stats.csv
+ icpt=2
+ thr=0.05
+ fmt=csv
+</div>
+</div>
### Details
@@ -521,6 +645,8 @@ distributions and link functions, see below for details.
### Usage
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f GLM.dml
-nvargs X=<file>
Y=<file>
@@ -539,7 +665,34 @@ distributions and link functions, see below for details.
disp=[double]
moi=[int]
mii=[int]
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f GLM.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ B=<file>
+ fmt=[format]
+ O=[file]
+ Log=[file]
+ dfam=[int]
+ vpow=[double]
+ link=[int]
+ lpow=[double]
+ yneg=[double]
+ icpt=[int]
+ reg=[double]
+ tol=[double]
+ disp=[double]
+ moi=[int]
+ mii=[int]
+</div>
+</div>
### Arguments
@@ -629,27 +782,53 @@ data
**mii**: (default: `0`) Maximum number of inner (conjugate gradient) iterations, or 0
if no maximum limit provided
-* * *
### Examples
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f GLM.dml
- -nvargs X=/user/ml/X.mtx
- Y=/user/ml/Y.mtx
- B=/user/ml/B.mtx
- fmt=csv
- dfam=2
- link=2
- yneg=-1.0
- icpt=2
- reg=0.01
- tol=0.00000001
- disp=1.0
- moi=100
- mii=10
- O=/user/ml/stats.csv
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ fmt=csv
+ dfam=2
+ link=2
+ yneg=-1.0
+ icpt=2
+ reg=0.01
+ tol=0.00000001
+ disp=1.0
+ moi=100
+ mii=10
+ O=/user/ml/stats.csv
Log=/user/ml/log.csv
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f GLM.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ fmt=csv
+ dfam=2
+ link=2
+ yneg=-1.0
+ icpt=2
+ reg=0.01
+ tol=0.00000001
+ disp=1.0
+ moi=100
+ mii=10
+ O=/user/ml/stats.csv
+ Log=/user/ml/log.csv
+</div>
+</div>
* * *
@@ -944,6 +1123,8 @@ distribution family is supported (see below for details).
### Usage
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f StepGLM.dml
-nvargs X=<file>
Y=<file>
@@ -959,6 +1140,31 @@ distribution family is supported (see below for details).
mii=[int]
thr=[double]
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f StepGLM.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=<file>
+ B=<file>
+ S=[file]
+ O=[file]
+ link=[int]
+ yneg=[double]
+ icpt=[int]
+ tol=[double]
+ disp=[double]
+ moi=[int]
+ mii=[int]
+ thr=[double]
+ fmt=[format]
+</div>
+</div>
### Arguments
@@ -1023,6 +1229,8 @@ SystemML Language Reference for details.
### Examples
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f StepGLM.dml
-nvargs X=/user/ml/X.mtx
Y=/user/ml/Y.mtx
@@ -1037,6 +1245,30 @@ SystemML Language Reference for details.
mii=10
thr=0.05
fmt=csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f StepGLM.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ Y=/user/ml/Y.mtx
+ B=/user/ml/B.mtx
+ S=/user/ml/selected.csv
+ O=/user/ml/stats.csv
+ link=2
+ yneg=-1.0
+ icpt=2
+ tol=0.000001
+ moi=100
+ mii=10
+ thr=0.05
+ fmt=csv
+</div>
+</div>
### Details
@@ -1145,6 +1377,8 @@ this step outside the scope of `GLM-predict.dml` for now.
### Usage
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f GLM-predict.dml
-nvargs X=<file>
Y=[file]
@@ -1157,6 +1391,28 @@ this step outside the scope of `GLM-predict.dml` for now.
lpow=[double]
disp=[double]
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f GLM-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ Y=[file]
+ B=<file>
+ M=[file]
+ O=[file]
+ dfam=[int]
+ vpow=[double]
+ link=[int]
+ lpow=[double]
+ disp=[double]
+ fmt=[format]
+</div>
+</div>
### Arguments
@@ -1258,101 +1514,256 @@ argument is set arbitrarily. The correct dispersion value should be
computed from the training data during model estimation, or omitted if
unknown (which sets it to `1.0`).
-Linear regression example:
+**Linear regression example**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f GLM-predict.dml
- -nvargs dfam=1
- vpow=0.0
- link=1
- lpow=1.0
- disp=5.67
- X=/user/ml/X.mtx
- B=/user/ml/B.mtx
- M=/user/ml/Means.mtx
- fmt=csv
- Y=/user/ml/Y.mtx
+ -nvargs dfam=1
+ vpow=0.0
+ link=1
+ lpow=1.0
+ disp=5.67
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Means.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
O=/user/ml/stats.csv
-
-
-Linear regression example, prediction only (no Y given):
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f GLM-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs dfam=1
+ vpow=0.0
+ link=1
+ lpow=1.0
+ disp=5.67
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Means.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+</div>
+</div>
+
+**Linear regression example, prediction only (no Y given)**:
+
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f GLM-predict.dml
- -nvargs dfam=1
- vpow=0.0
- link=1
- lpow=1.0
- X=/user/ml/X.mtx
- B=/user/ml/B.mtx
- M=/user/ml/Means.mtx
+ -nvargs dfam=1
+ vpow=0.0
+ link=1
+ lpow=1.0
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Means.mtx
fmt=csv
-
-Binomial logistic regression example:
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f GLM-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs dfam=1
+ vpow=0.0
+ link=1
+ lpow=1.0
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Means.mtx
+ fmt=csv
+</div>
+</div>
+
+**Binomial logistic regression example**:
+
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f GLM-predict.dml
- -nvargs dfam=2
- link=2
- disp=3.0004464
- X=/user/ml/X.mtx
- B=/user/ml/B.mtx
- M=/user/ml/Probabilities.mtx
- fmt=csv
- Y=/user/ml/Y.mtx
+ -nvargs dfam=2
+ link=2
+ disp=3.0004464
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Probabilities.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
O=/user/ml/stats.csv
-
-Binomial probit regression example:
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f GLM-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs dfam=2
+ link=2
+ disp=3.0004464
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Probabilities.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+</div>
+</div>
+
+**Binomial probit regression example**:
+
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f GLM-predict.dml
- -nvargs dfam=2
- link=3
- disp=3.0004464
- X=/user/ml/X.mtx
- B=/user/ml/B.mtx
- M=/user/ml/Probabilities.mtx
- fmt=csv
- Y=/user/ml/Y.mtx
+ -nvargs dfam=2
+ link=3
+ disp=3.0004464
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Probabilities.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
O=/user/ml/stats.csv
-
-Multinomial logistic regression example:
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f GLM-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs dfam=2
+ link=3
+ disp=3.0004464
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Probabilities.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+</div>
+</div>
+
+**Multinomial logistic regression example**:
+
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f GLM-predict.dml
-nvargs dfam=3
- X=/user/ml/X.mtx
- B=/user/ml/B.mtx
- M=/user/ml/Probabilities.mtx
- fmt=csv
- Y=/user/ml/Y.mtx
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Probabilities.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
O=/user/ml/stats.csv
-
-Poisson regression with the log link example:
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f GLM-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs dfam=3
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Probabilities.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+</div>
+</div>
+
+**Poisson regression with the log link example**:
+
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f GLM-predict.dml
- -nvargs dfam=1
- vpow=1.0
- link=1
- lpow=0.0
- disp=3.45
- X=/user/ml/X.mtx
- B=/user/ml/B.mtx
- M=/user/ml/Means.mtx
- fmt=csv
- Y=/user/ml/Y.mtx
+ -nvargs dfam=1
+ vpow=1.0
+ link=1
+ lpow=0.0
+ disp=3.45
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Means.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
O=/user/ml/stats.csv
-
-Gamma regression with the inverse (reciprocal) link example:
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f GLM-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs dfam=1
+ vpow=1.0
+ link=1
+ lpow=0.0
+ disp=3.45
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Means.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+</div>
+</div>
+
+**Gamma regression with the inverse (reciprocal) link example**:
+
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f GLM-predict.dml
- -nvargs dfam=1
- vpow=2.0
- link=1
- lpow=-1.0
- disp=1.99118
- X=/user/ml/X.mtx
- B=/user/ml/B.mtx
- M=/user/ml/Means.mtx
- fmt=csv
- Y=/user/ml/Y.mtx
+ -nvargs dfam=1
+ vpow=2.0
+ link=1
+ lpow=-1.0
+ disp=1.99118
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Means.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
O=/user/ml/stats.csv
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f GLM-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs dfam=1
+ vpow=2.0
+ link=1
+ lpow=-1.0
+ disp=1.99118
+ X=/user/ml/X.mtx
+ B=/user/ml/B.mtx
+ M=/user/ml/Means.mtx
+ fmt=csv
+ Y=/user/ml/Y.mtx
+ O=/user/ml/stats.csv
+</div>
+</div>
* * *
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b8055201/algorithms-survival-analysis.md
----------------------------------------------------------------------
diff --git a/algorithms-survival-analysis.md b/algorithms-survival-analysis.md
index b76ab91..e2fa97a 100644
--- a/algorithms-survival-analysis.md
+++ b/algorithms-survival-analysis.md
@@ -23,6 +23,8 @@ censored and uncensored survival times.
### Usage
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f KM.dml
-nvargs X=<file>
TE=<file>
@@ -36,6 +38,29 @@ censored and uncensored survival times.
ctype=[plain|log|log-log]
ttype=[none|log-rank|wilcoxon]
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f KM.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ TE=<file>
+ GI=<file>
+ SI=<file>
+ O=<file>
+ M=<file>
+ T=<file>
+ alpha=[double]
+ etype=[greenwood|peto]
+ ctype=[plain|log|log-log]
+ ttype=[none|log-rank|wilcoxon]
+ fmt=[format]
+</div>
+</div>
### Arguments
@@ -95,6 +120,8 @@ SystemML Language Reference for details.
### Examples
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f KM.dml
-nvargs X=/user/ml/X.mtx
TE=/user/ml/TE
@@ -106,7 +133,30 @@ SystemML Language Reference for details.
etype=greenwood
ctype=plain
fmt=csv
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f KM.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ TE=/user/ml/TE
+ GI=/user/ml/GI
+ SI=/user/ml/SI
+ O=/user/ml/kaplan-meier.csv
+ M=/user/ml/model.csv
+ alpha=0.01
+ etype=greenwood
+ ctype=plain
+ fmt=csv
+</div>
+</div>
+
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f KM.dml
-nvargs X=/user/ml/X.mtx
TE=/user/ml/TE
@@ -120,6 +170,29 @@ SystemML Language Reference for details.
ctype=log
ttype=log-rank
fmt=csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f KM.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ TE=/user/ml/TE
+ GI=/user/ml/GI
+ SI=/user/ml/SI
+ O=/user/ml/kaplan-meier.csv
+ M=/user/ml/model.csv
+ T=/user/ml/test.csv
+ alpha=0.01
+ etype=peto
+ ctype=log
+ ttype=log-rank
+ fmt=csv
+</div>
+</div>
### Details
@@ -350,6 +423,8 @@ may be categorical (ordinal or nominal) as well as continuous-valued.
**Cox**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Cox.dml
-nvargs X=<file>
TE=<file>
@@ -367,9 +442,39 @@ may be categorical (ordinal or nominal) as well as continuous-valued.
moi=[int]
mii=[int]
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Cox.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ TE=<file>
+ F=<file>
+ R=[file]
+ M=<file>
+ S=[file]
+ T=[file]
+ COV=<file>
+ RT=<file>
+ XO=<file>
+ MF=<file>
+ alpha=[double]
+ tol=[double]
+ moi=[int]
+ mii=[int]
+ fmt=[format]
+</div>
+</div>
+
**Cox Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Cox-predict.dml
-nvargs X=<file>
RT=<file>
@@ -379,6 +484,26 @@ may be categorical (ordinal or nominal) as well as continuous-valued.
MF=<file>
P=<file>
fmt=[format]
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Cox-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=<file>
+ RT=<file>
+ M=<file>
+ Y=<file>
+ COV=<file>
+ MF=<file>
+ P=<file>
+ fmt=[format]
+</div>
+</div>
+
### Arguments - Cox Model Fitting/Prediction
@@ -456,6 +581,8 @@ SystemML Language Reference for details.
**Cox**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Cox.dml
-nvargs X=/user/ml/X.mtx
TE=/user/ml/TE
@@ -466,7 +593,29 @@ SystemML Language Reference for details.
COV=/user/ml/var-covar.csv
XO=/user/ml/X-sorted.mtx
fmt=csv
-
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Cox.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ TE=/user/ml/TE
+ F=/user/ml/F
+ R=/user/ml/R
+ M=/user/ml/model.csv
+ T=/user/ml/test.csv
+ COV=/user/ml/var-covar.csv
+ XO=/user/ml/X-sorted.mtx
+ fmt=csv
+</div>
+</div>
+
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Cox.dml
-nvargs X=/user/ml/X.mtx
TE=/user/ml/TE
@@ -483,9 +632,37 @@ SystemML Language Reference for details.
moi=100
mii=20
fmt=csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Cox.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X.mtx
+ TE=/user/ml/TE
+ F=/user/ml/F
+ R=/user/ml/R
+ M=/user/ml/model.csv
+ T=/user/ml/test.csv
+ COV=/user/ml/var-covar.csv
+ RT=/user/ml/recoded-timestamps.csv
+ XO=/user/ml/X-sorted.csv
+ MF=/user/ml/baseline.csv
+ alpha=0.01
+ tol=0.000001
+ moi=100
+ mii=20
+ fmt=csv
+</div>
+</div>
**Cox Prediction**:
+<div class="codetabs">
+<div data-lang="Hadoop" markdown="1">
hadoop jar SystemML.jar -f Cox-predict.dml
-nvargs X=/user/ml/X-sorted.mtx
RT=/user/ml/recoded-timestamps.csv
@@ -495,6 +672,25 @@ SystemML Language Reference for details.
MF=/user/ml/baseline.csv
P=/user/ml/predictions.csv
fmt=csv
+</div>
+<div data-lang="Spark" markdown="1">
+ $SPARK_HOME/bin/spark-submit --master yarn-cluster
+ --conf spark.driver.maxResultSize=0
+ --conf spark.akka.frameSize=128
+ SystemML.jar
+ -f Cox-predict.dml
+ -config=SystemML-config.xml
+ -exec hybrid_spark
+ -nvargs X=/user/ml/X-sorted.mtx
+ RT=/user/ml/recoded-timestamps.csv
+ M=/user/ml/model.csv
+ Y=/user/ml/Y.mtx
+ COV=/user/ml/var-covar.csv
+ MF=/user/ml/baseline.csv
+ P=/user/ml/predictions.csv
+ fmt=csv
+</div>
+</div>
### Details
@@ -671,7 +867,7 @@ follows. File `S` stores the following information
* Line 5: Cox & Snell Rsquare
* Line 6: maximum possible Rsquare.
-Above, the AIC is computed as in [[Stepwise Linear Regression]](algorithms-regression.html#stepwise-linear-regression), the Cox & Snell Rsquare
+Above, the AIC is computed as in [Stepwise Linear Regression](algorithms-regression.html#stepwise-linear-regression), the Cox & Snell Rsquare
is equal to $$1-\exp\{ -l/n \}$$, where $l$ is the log-rank test statistic
as discussed above and $n$ is total number of observations, and the
maximum possible Rsquare computed as $$1-\exp\{ -2 L(\textbf{0})/n \}$$,
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b8055201/dml-and-pydml-programming-guide.md
----------------------------------------------------------------------
diff --git a/dml-and-pydml-programming-guide.md b/dml-and-pydml-programming-guide.md
index 2fae62a..d8dc836 100644
--- a/dml-and-pydml-programming-guide.md
+++ b/dml-and-pydml-programming-guide.md
@@ -52,7 +52,7 @@ The `DMLScript` class serves as the main entrypoint to SystemML. Executing
In Eclipse, a Debug Configuration can be created with `DMLScript` as the Main class and any arguments specified as
Program arguments. A PyDML script requires the addition of a `-python` switch.
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="Eclipse Debug Configuration - Main" markdown="1">
@@ -81,7 +81,7 @@ based on its value. Mathematical operations typically operate on
doubles/floats, whereas integers/ints are typically useful for tasks such as
iteration and accessing elements in a matrix.
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="DML" markdown="1">
{% highlight r %}
@@ -161,7 +161,7 @@ function. In the example below, a matrix element is still considered to be of th
so the value is cast to a scalar in order to print it. Matrix element values are of type **double**/**float**.
*Note that matrices index from 1 in both DML and PyDML.*
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="DML" markdown="1">
{% highlight r %}
@@ -212,7 +212,7 @@ PyDML Language Reference (Matrix Construction).
A matrix can be saved using the **`write()`** function in DML and the **`save()`** function in PyDML. SystemML supports four
different formats: **`text`** (`i,j,v`), **`mm`** (`Matrix Market`), **`csv`** (`delimiter-separated values`), and **`binary`**.
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="DML" markdown="1">
{% highlight r %}
@@ -239,7 +239,7 @@ save(m, "m.binary", format="binary")
Saving a matrix automatically creates a metadata file for each format except for Matrix Market, since Matrix Market contains
metadata within the *.mm file. All formats are text-based except binary. The contents of the resulting files are shown here.
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="m.txt" markdown="1">
1 1 1.0
@@ -321,7 +321,7 @@ A matrix can be loaded using the **`read()`** function in DML and the **`load()`
formats: **`text`** (`i,j,v`), **`mm`** (`Matrix Market`), **`csv`** (`delimiter-separated values`), and **`binary`**. To read a file, a corresponding
metadata file is required, except for the Matrix Market format.
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="DML" markdown="1">
{% highlight r %}
@@ -387,7 +387,7 @@ by 2 and assign the resulting matrix to D.
This example also shows a user-defined function called `printMatrix()`, which takes a string and matrix as arguments and returns
nothing.
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="DML" markdown="1">
{% highlight r %}
@@ -471,7 +471,7 @@ specifying row 2 and leaving the column blank. We obtain a column slice (vector)
column 3. After that, we obtain a submatrix via range indexing, where we specify rows 2 to 3, separated by a colon, and columns
1 to 2, separated by a colon.
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="DML" markdown="1">
{% highlight r %}
@@ -557,7 +557,7 @@ DML and PyDML feature 3 loop statements: `while`, `for`, and `parfor` (parallel
sequentially as in a regular `for` loop. The `parfor` statement can include several optional parameters, as described
in the DML Language Reference ([ParFor Statement](dml-language-reference.html#parfor-statement)) and PyDML Language Reference (ParFor Statement).
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="DML" markdown="1">
{% highlight r %}
@@ -623,7 +623,7 @@ Functions encapsulate useful functionality in SystemML. In addition to built-in
Functions take 0 or more parameters and return 0 or more values.
Currently, if a function returns nothing, it still needs to be assigned to a variable.
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="DML" markdown="1">
{% highlight r %}
@@ -669,7 +669,7 @@ is appended to the matrix. A column consisting of the row sums is appended to th
matrix is returned to variable B. Matrix A is output to the `A.csv` file and matrix B is saved as the `B.csv` file.
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="A.csv" markdown="1">
1.6091961493071,0.7088614208099939
@@ -698,7 +698,7 @@ In the example below, a matrix is read from the file system using named argument
using the `rowsToPrint` argument, which defaults to 2 if no argument is supplied. Likewise, the number of columns is
specified using `colsToPrint` with a default value of 2.
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="DML" markdown="1">
{% highlight r %}
@@ -780,7 +780,7 @@ for (i in 1:numRowsToPrint):
Here, we see identical functionality but with positional arguments.
-<div class="codetabs">
+<div class="codetabs2">
<div data-lang="DML" markdown="1">
{% highlight r %}
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b8055201/js/main.js
----------------------------------------------------------------------
diff --git a/js/main.js b/js/main.js
index 28e7187..2ea6053 100644
--- a/js/main.js
+++ b/js/main.js
@@ -18,6 +18,7 @@
/* Custom JavaScript code in the MarkDown docs */
// Enable language-specific code tabs
+// similar to spark code tabs, except can have spaces and periods
function codeTabs() {
var counter = 0;
var langImages = {
@@ -39,13 +40,79 @@ function codeTabs() {
var lang = $(this).data("lang");
var image = $(this).data("image");
var notabs = $(this).data("notabs");
+ var capitalizedLang = lang.substr(0, 1).toUpperCase() + lang.substr(1);
+
+ // for flexibility, allow spaces and periods
var langConv = lang.replace(/\./g, "_");
langConv = langConv.replace(/ /g, "_");
+
var id = "tab_" + langConv + "_" + counter;
$(this).attr("id", id);
if (image != null && langImages[lang]) {
var buttonLabel = "<img src='" +langImages[lang] + "' alt='" + capitalizedLang + "' />";
} else if (notabs == null) {
+ var buttonLabel = "<b>" + capitalizedLang + "</b>";
+ } else {
+ var buttonLabel = ""
+ }
+ tabBar.append(
+ '<li><a class="tab_' + langConv + '" href="#' + id + '">' + buttonLabel + '</a></li>'
+ );
+ });
+
+ codeSamples.first().addClass("active");
+ tabBar.children("li").first().addClass("active");
+ counter++;
+ });
+ $("ul.nav-tabs a").click(function (e) {
+ // Toggling a tab should switch all tabs corresponding to the same language
+ // while retaining the scroll position
+ e.preventDefault();
+ var scrollOffset = $(this).offset().top - $(document).scrollTop();
+ $("." + $(this).attr('class')).tab('show');
+ $(document).scrollTop($(this).offset().top - scrollOffset);
+ });
+}
+
+// Code tabs that don't switch all tabs
+//
+// Note that codeTabs2() should be called after codeTabs(). If codeTabs2()
+// were called first, 2 click events would get attached to the tabs since
+// "ul.nav-tabs a" and "ul.nav-tabs2 a" would both apply. Note that
+// nav-tabs and nav-tabs2 are both used because css doesn't support
+// inheritance, and nav-tabs has a significant amount of styling in
+// bootstrap.css.
+function codeTabs2() {
+ var counter = 0;
+ var langImages = {
+ "scala": "img/scala-sm.png",
+ "python": "img/python-sm.png",
+ "java": "img/java-sm.png"
+ };
+ $("div.codetabs2").each(function() {
+ $(this).addClass("tab-content");
+
+ // Insert the tab bar
+ var tabBar = $('<ul class="nav nav-tabs nav-tabs2" data-tabs="tabs"></ul>');
+ $(this).before(tabBar);
+
+ // Add each code sample to the tab bar:
+ var codeSamples = $(this).children("div");
+ codeSamples.each(function() {
+ $(this).addClass("tab-pane");
+ var lang = $(this).data("lang");
+ var image = $(this).data("image");
+ var notabs = $(this).data("notabs");
+
+ // for flexibility, allow spaces and periods
+ var langConv = lang.replace(/\./g, "_");
+ langConv = langConv.replace(/ /g, "_");
+
+ var id = "tab_" + langConv + "_" + counter;
+ $(this).attr("id", id);
+ if (image != null && langImages[lang]) {
+ var buttonLabel = "<img src='" +langImages[lang] + "' alt='" + lang + "' />";
+ } else if (notabs == null) {
var buttonLabel = "<b>" + lang + "</b>";
} else {
var buttonLabel = ""
@@ -59,7 +126,7 @@ function codeTabs() {
tabBar.children("li").first().addClass("active");
counter++;
});
- $("ul.nav-tabs a").click(function (e) {
+ $("ul.nav-tabs2 a").click(function (e) {
e.preventDefault();
var scrollOffset = $(this).offset().top - $(document).scrollTop();
$(this).tab('show'); // only switch current tab - makes comparisons feel slightly better on this page
@@ -79,6 +146,7 @@ function maybeScrollToHash() {
$(function() {
codeTabs();
+ codeTabs2();
// Display anchor links when hovering over headers. For documentation of the
// configuration options, see the AnchorJS documentation.
anchors.options = {
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b8055201/quick-start-guide.md
----------------------------------------------------------------------
diff --git a/quick-start-guide.md b/quick-start-guide.md
index c8d5f15..5080a49 100644
--- a/quick-start-guide.md
+++ b/quick-start-guide.md
@@ -17,7 +17,7 @@ about running SystemML in distributed execution mode (Hadoop, Spark) will
be added soon.
For more in depth information, please refer to the
[Algorithms Reference](algorithms-reference.html) and the
-[SystemML Language Reference](dml-language-reference.html).
+[DML Language Reference](dml-language-reference.html).
<br/>
@@ -122,9 +122,9 @@ data file `<filename>` must be `<filename>.mtd`.
# Example 1 - Univariate Statistics
-Let's start with a simple example, computing certain [univariate statistics](systemml-algorithms-descriptive-statistics.html#univariate-statistics)
+Let's start with a simple example, computing certain [univariate statistics](algorithms-descriptive-statistics.html#univariate-statistics)
for each feature column using the algorithm `Univar-Stats.dml` which requires 3
-[arguments](systemml-algorithms-descriptive-statistics.html#arguments):
+[arguments](algorithms-descriptive-statistics.html#arguments):
* `X`: location of the input data file to analyze
* `TYPES`: location of the file that contains the feature column types encoded by integer numbers: `1` = scale, `2` = nominal, `3` = ordinal
@@ -225,7 +225,7 @@ features.
# Example 2 - Binary-class Support Vector Machines
Let's take the same `haberman.data` to explore the
-[binary-class support vector machines](systemml-algorithms-classification.html#binary-class-support-vector-machines) algorithm `l2-svm.dml`.
+[binary-class support vector machines](algorithms-classification.html#binary-class-support-vector-machines) algorithm `l2-svm.dml`.
This example also illustrates how to use of the sampling algorithm `sample.dml`
and the data split algorithm `spliXY.dml`.
@@ -278,7 +278,7 @@ the `splitXY.dml` algorithm on our training and test data sets.
Now we need to train our model using the `l2-svm.dml` algorithm.
-[Parameters](systemml-algorithms-classification.html#arguments-1):
+[Parameters](algorithms-classification.html#arguments-1):
* `X` : (input) filename of training data features
* `Y` : (input) filename of training data labels
@@ -335,7 +335,7 @@ If the confusion matrix looks like this ..
<br/>
-Refer to the *SystemML Algorithms Reference* (`docs/SystemML_Algorithms_Reference.pdf`) for more details.
+Refer to the [Algorithms Reference](algorithms-reference.html) for more details.
<br/>
@@ -353,46 +353,8 @@ the memory available to the JVM, i.e:
# Next Steps
-Check out the [SystemML Algorithm Reference](systemml-algorithms.html) and run
+Check out the [Algorithms Reference](algorithms-reference.html) and run
some of the other pre-packaged algorithms.
-<style type="text/css">
-<!--
- ol { list-style-type: none; counter-reset: item; margin: 1em; }
- ol > li { display: table; counter-increment: item; margin-bottom: 0.1em; }
- ol > li:before { content: counters(item, ".") ". "; display: table-cell; padding-right: 0.6em; }
- li ol > li { margin: 0; }
- li ol > li:before { content: counters(item, ".") " "; }
--->
-</style>
-
-1. Descriptive Statistics
- - [Univariate Statistics](systemml-algorithms-descriptive-statistics.html#univariate-statistics) (`Univar-Stats.dml`)
- - [Bivariate Statistics](systemml-algorithms-descriptive-statistics.html#bivariate-statistics) (`bivar-stats.dml`)
- - [Stratified Bivariate Statistics](systemml-algorithms-descriptive-statistics.html#stratified-bivariate-statistics) (`stratstats.dml`)
-2. Classification
- - [Multinomial Logistic Regression](systemml-algorithms-classification.html#multinomial-logistic-regression) (`MultiLogReg.dml`)
- - [Binary-Class Support Vector Machines](systemml-algorithms-classification.html#binary-class-support-vector-machines) (`l2-svm.dml`, `l2-svm-predict.dml`)
- - [Multi-Class Support Vector Machines](systemml-algorithms-classification.html#multi-class-support-vector-machines) (`m-svm.dml`, `m-svm-predict.dml`)
- - [Naive Bayes](systemml-algorithms-classification.html#naive-bayes) (`naive-bayes.dml`, `naive-bayes-predict.dml`)
-3. Clustering
- - [K-Means Clustering](systemml-algorithms-clustering.html#k-means-clustering) (`Kmeans.dml`, `Kmeans-predict.dml`)
-4. Regression
- - [Linear Regression](systemml-algorithms-regression.html#linear-regression) (`LinearRegDS.dml`, `LinearRegCG.dml`)
- - [Generalized Linear Models](systemml-algorithms-regression.html#generalized-linear-models) (`GLM.dml`)
- - [Regression Scoring and Prediction](systemml-algorithms-regression.html#regression-scoring-and-prediction) (`GLM-predict.dml`)
-5. Matrix Factorization
- - [Principle Component Analysis](systemml-algorithms-matrix-factorization.html#principle-component-analysis) (`PCA.dml`)
-6. Interpolation
- - [Cubic Spline Interpolation](systemml-algorithms-interpolations.html#cubic-spline)
-
-
-{% comment %} TODO: update the algorithms in the standalone package {% endcomment %}
-
-{% comment %} TODO: update the algorithms reference doc to include all algorithms, currently missing decision-tree.dml {% endcomment %}
-
-<br/>
-
-Follow the [Programming Guide](systemml-programming-guide.html) to implement
-your own SystemML algorithms.
+Follow the [DML and PyDML Programming Guide](dml-and-pydml-programming-guide.html) to become familiar with DML and PyDML.
[22/47] incubator-systemml git commit: Fix md errors related to
anchors in Eclipse editor.
Posted by du...@apache.org.
Fix md errors related to anchors in Eclipse editor.
Anchors referencing headers in md can show up as errors
in Eclipse md editor, so prepended anchors with page name
to avoid the sporadic error messages that can show up
in the Eclipse Problems view.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/82895b13
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/82895b13
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/82895b13
Branch: refs/heads/gh-pages
Commit: 82895b13106ffe8b2b136dd25df2e6512e48ecf9
Parents: f4b41e0
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Tue Sep 8 18:22:52 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Tue Sep 8 18:22:52 2015 -0700
----------------------------------------------------------------------
dml-language-reference.md | 66 +++++++++++++++++++++---------------------
1 file changed, 33 insertions(+), 33 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/82895b13/dml-language-reference.md
----------------------------------------------------------------------
diff --git a/dml-language-reference.md b/dml-language-reference.md
index 2824f97..c553a58 100644
--- a/dml-language-reference.md
+++ b/dml-language-reference.md
@@ -7,39 +7,39 @@ description: Declarative Machine Learning (DML) Language Reference
### Table of Contents
- * [Introduction](#introduction)
- * [Variables](#variables)
- * [Identifier Names](#identifier-names)
- * [Data Types](#data-types)
- * [Comments](#comments)
- * [Expressions](#expressions)
- * [Operators](#operators)
- * [Matrix-Vector Operations](#matrix-vector-operations)
- * [Matrix Indexing](#matrix-indexing)
- * [Statements](#statements)
- * [Assignment Statement](#assignment-statement)
- * [Control Statements](#control-statements)
- * [While Statement](#while-statement)
- * [If Statement](#if-statement)
- * [For Statement](#for-statement)
- * [ParFor Statement](#parfor-statement)
- * [User-Defined Function (UDF)](#user-defined-function-udf)
- * [Variable Scoping](#variable-scoping)
- * [Command-Line Arguments](#command-line-arguments)
- * [Built-in Functions](#built-in-functions)
- * [Matrix Construction, Manipulation, and Aggregation Built-In Functions](#matrix-construction-manipulation-and-aggregation-built-in-functions)
- * [Matrix and/or Scalar Comparison Built-In Functions](#matrix-andor-scalar-comparison-built-in-functions)
- * [Casting Built-In Functions](#casting-built-in-functions)
- * [Statistical Built-In Functions](#statistical-built-in-functions)
- * [Mathematical and Trigonometric Built-In Functions](#mathematical-and-trigonometric-built-in-functions)
- * [Linear Algebra Built-In Functions](#linear-algebra-built-in-functions)
- * [Read/Write Built-In Functions](#readwrite-built-in-functions)
- * [Data Pre-Processing Built-In Functions](#data-pre-processing-built-in-functions)
- * [Other Built-In Functions](#other-built-in-functions)
- * [Modules](#modules)
- * [Reserved Keywords](#reserved-keywords)
- * [Invocation of SystemML](#invocation-of-systemml)
- * [MLContext API](#mlcontext-api)
+ * [Introduction](dml-language-reference.html#introduction)
+ * [Variables](dml-language-reference.html#variables)
+ * [Identifier Names](dml-language-reference.html#identifier-names)
+ * [Data Types](dml-language-reference.html#data-types)
+ * [Comments](dml-language-reference.html#comments)
+ * [Expressions](dml-language-reference.html#expressions)
+ * [Operators](dml-language-reference.html#operators)
+ * [Matrix-Vector Operations](dml-language-reference.html#matrix-vector-operations)
+ * [Matrix Indexing](dml-language-reference.html#matrix-indexing)
+ * [Statements](dml-language-reference.html#statements)
+ * [Assignment Statement](dml-language-reference.html#assignment-statement)
+ * [Control Statements](dml-language-reference.html#control-statements)
+ * [While Statement](dml-language-reference.html#while-statement)
+ * [If Statement](dml-language-reference.html#if-statement)
+ * [For Statement](dml-language-reference.html#for-statement)
+ * [ParFor Statement](dml-language-reference.html#parfor-statement)
+ * [User-Defined Function (UDF)](dml-language-reference.html#user-defined-function-udf)
+ * [Variable Scoping](dml-language-reference.html#variable-scoping)
+ * [Command-Line Arguments](dml-language-reference.html#command-line-arguments)
+ * [Built-in Functions](dml-language-reference.html#built-in-functions)
+ * [Matrix Construction, Manipulation, and Aggregation Built-In Functions](dml-language-reference.html#matrix-construction-manipulation-and-aggregation-built-in-functions)
+ * [Matrix and/or Scalar Comparison Built-In Functions](dml-language-reference.html#matrix-andor-scalar-comparison-built-in-functions)
+ * [Casting Built-In Functions](dml-language-reference.html#casting-built-in-functions)
+ * [Statistical Built-In Functions](dml-language-reference.html#statistical-built-in-functions)
+ * [Mathematical and Trigonometric Built-In Functions](dml-language-reference.html#mathematical-and-trigonometric-built-in-functions)
+ * [Linear Algebra Built-In Functions](dml-language-reference.html#linear-algebra-built-in-functions)
+ * [Read/Write Built-In Functions](dml-language-reference.html#readwrite-built-in-functions)
+ * [Data Pre-Processing Built-In Functions](dml-language-reference.html#data-pre-processing-built-in-functions)
+ * [Other Built-In Functions](dml-language-reference.html#other-built-in-functions)
+ * [Modules](dml-language-reference.html#modules)
+ * [Reserved Keywords](dml-language-reference.html#reserved-keywords)
+ * [Invocation of SystemML](dml-language-reference.html#invocation-of-systemml)
+ * [MLContext API](dml-language-reference.html#mlcontext-api)
## Introduction
[42/47] incubator-systemml git commit: [SYSML-363] Built-in functions
cbind and rbind added to documentation
Posted by du...@apache.org.
[SYSML-363] Built-in functions cbind and rbind added to documentation
New built-in functions cbind and rbind added to DML Language Reference.
Use of matrix append discouraged. Replaced append example with cbind
in DML and PyDML Programming Guide.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/56c997a7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/56c997a7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/56c997a7
Branch: refs/heads/gh-pages
Commit: 56c997a75606f06f8401c166da09552e8bf0299b
Parents: b805520
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Fri Oct 30 12:54:33 2015 -0300
Committer: Luciano Resende <lr...@apache.org>
Committed: Fri Oct 30 12:54:33 2015 -0300
----------------------------------------------------------------------
dml-and-pydml-programming-guide.md | 22 +++++++++++-----------
dml-language-reference.md | 4 +++-
2 files changed, 14 insertions(+), 12 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/56c997a7/dml-and-pydml-programming-guide.md
----------------------------------------------------------------------
diff --git a/dml-and-pydml-programming-guide.md b/dml-and-pydml-programming-guide.md
index d8dc836..85367e5 100644
--- a/dml-and-pydml-programming-guide.md
+++ b/dml-and-pydml-programming-guide.md
@@ -629,10 +629,10 @@ Currently, if a function returns nothing, it still needs to be assigned to a var
{% highlight r %}
doSomething = function(matrix[double] mat) return (matrix[double] ret) {
additionalCol = matrix(1, rows=nrow(mat), cols=1) # 1x3 matrix with 1 values
- ret = append(mat, additionalCol) # append column to matrix
- ret = append(ret, seq(0, 2, 1)) # append column (0,1,2) to matrix
- ret = append(ret, rowMaxs(ret)) # append column of max row values to matrix
- ret = append(ret, rowSums(ret)) # append column of row sums to matrix
+ ret = cbind(mat, additionalCol) # concatenate column to matrix
+ ret = cbind(ret, seq(0, 2, 1)) # concatenate column (0,1,2) to matrix
+ ret = cbind(ret, rowMaxs(ret)) # concatenate column of max row values to matrix
+ ret = cbind(ret, rowSums(ret)) # concatenate column of row sums to matrix
}
A = rand(rows=3, cols=2, min=0, max=2) # random 3x2 matrix with values 0 to 2
@@ -646,10 +646,10 @@ write(B, "B.csv", format="csv")
{% highlight python %}
def doSomething(mat: matrix[float]) -> (ret: matrix[float]):
additionalCol = full(1, rows=nrow(mat), cols=1) # 1x3 matrix with 1 values
- ret = append(mat, additionalCol) # append column to matrix
- ret = append(ret, seq(0, 2, 1)) # append column (0,1,2) to matrix
- ret = append(ret, rowMaxs(ret)) # append column of max row values to matrix
- ret = append(ret, rowSums(ret)) # append column of row sums to matrix
+ ret = cbind(mat, additionalCol) # concatenate column to matrix
+ ret = cbind(ret, seq(0, 2, 1)) # concatenate column (0,1,2) to matrix
+ ret = cbind(ret, rowMaxs(ret)) # concatenate column of max row values to matrix
+ ret = cbind(ret, rowSums(ret)) # concatenate column of row sums to matrix
A = rand(rows=3, cols=2, min=0, max=2) # random 3x2 matrix with values 0 to 2
B = doSomething(A)
@@ -663,9 +663,9 @@ save(B, "B.csv", format="csv")
In the above example, a 3x2 matrix of random doubles between 0 and 2 is created using the **`rand()`** function.
Additional parameters can be passed to **`rand()`** to control sparsity and other matrix characteristics.
-Matrix A is passed to the `doSomething` function. A column of 1 values is appended to the matrix. A column
-consisting of the values `(0, 1, 2)` is appended to the matrix. Next, a column consisting of the maximum row values
-is appended to the matrix. A column consisting of the row sums is appended to the matrix, and this resulting
+Matrix A is passed to the `doSomething` function. A column of 1 values is concatenated to the matrix. A column
+consisting of the values `(0, 1, 2)` is concatenated to the matrix. Next, a column consisting of the maximum row values
+is concatenated to the matrix. A column consisting of the row sums is concatenated to the matrix, and this resulting
matrix is returned to variable B. Matrix A is output to the `A.csv` file and matrix B is saved as the `B.csv` file.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/56c997a7/dml-language-reference.md
----------------------------------------------------------------------
diff --git a/dml-language-reference.md b/dml-language-reference.md
index de73ab2..f6915a3 100644
--- a/dml-language-reference.md
+++ b/dml-language-reference.md
@@ -526,7 +526,8 @@ Table 3. Matrix Construction, Manipulation, and Aggregation Built-In Functions
Function | Description | Parameters | Example
-------- | ----------- | ---------- | -------
-append() | Adds the second argument as additional columns to the first argument (note that the first argument is not over-written). Append is meant to be used in situations where one cannot use left-indexing. | Input : (X <matrix>, Y <matrix>) <br/>Output : <matrix> <br/> X and Y are matrices (with possibly multiple columns), where the number of rows in X and Y must be the same. Output is a matrix with exactly the same number of rows as X and Y. Let n1 and n2 denote the number of columns of matrix X and Y, respectively. The returned matrix has n1+n2 columns, where the first n1 columns contain X and the last n2 columns contain Y. | A = matrix(1, rows=2,cols=5) <br/> B = matrix(1, rows=2,cols=3) <br/> C = append(A,B) <br/> print( "Dimensions of C:" + nrow(C) + " X " + ncol(C)) <br/> The output of above example is: <br/> Dimensions of C:2 X 8
+append() | Adds the second argument as additional columns to the first argument (note that the first argument is not over-written). Append is meant to be used in situations where one cannot use left-indexing. <br/> **NOTE: append() has been replaced by cbind(), so its use is discouraged.** | Input: (X <matrix>, Y <matrix>) <br/>Output: <matrix> <br/> X and Y are matrices (with possibly multiple columns), where the number of rows in X and Y must be the same. Output is a matrix with exactly the same number of rows as X and Y. Let n1 and n2 denote the number of columns of matrix X and Y, respectively. The returned matrix has n1+n2 columns, where the first n1 columns contain X and the last n2 columns contain Y. | A = matrix(1, rows=2,cols=5) <br/> B = matrix(1, rows=2,cols=3) <br/> C = append(A,B) <br/> print("Dimensions of C: " + nrow(C) + " X " + ncol(C)) <br/> The output of above example is: <br/> Dimensions of C: 2 X 8
+cbind() | Column-wise matrix concatenation. Concatenates the second matrix as additional columns to the first matrix | Input: (X <matrix>, Y <matrix>) <br/>Output: <matrix> <br/> X and Y are matrices, where the number of rows in X and the number of rows in Y are the same. | A = matrix(1, rows=2,cols=3) <br/> B = matrix(2, rows=2,cols=3) <br/> C = cbind(A,B) <br/> print("Dimensions of C: " + nrow(C) + " X " + ncol(C)) <br/> Output: <br/> Dimensions of C: 2 X 6
matrix() | Matrix constructor (assigning all the cells to numeric literals). | Input: (<init>, rows=<value>, cols=<value>) <br/> init: numeric literal; <br/> rows/cols: number of rows/cols (expression) <br/> Output: matrix | # 10x10 matrix initialized to 0 <br/> A = matrix (0, rows=10, cols=10)
| Matrix constructor (reshaping an existing matrix). | Input: (<existing matrix>, rows=<value>, cols=<value>, byrow=TRUE) <br/> Output: matrix | A = matrix (0, rows=10, cols=10) <br/> B = matrix (A, rows=100, cols=1)
| Matrix constructor (initializing using string). | Input: (<initialization string>, rows=<value>, cols=<value>) <br/> Output: matrix | A = matrix("4 3 2 5 7 8", rows=3, cols=2) <br/> Creates a matrix: [ [4, 3], [2, 5], [7, 8] ]
@@ -535,6 +536,7 @@ min() <br/> max() | Return the minimum/maximum cell values of two matrices, matr
nrow(), <br/> ncol(), <br/> length() | Return the number of rows, number of columns, or number of cells in matrix respectively. | Input: matrix <br/> Output: scalar | nrow(X)
prod() | Return the product of all cells in matrix | Input: matrix <br/> Output: scalarj | prod(X)
rand() | Generates a random matrix | Input: (rows=<value>, cols=<value>, min=<value>, max=<value>, sparsity=<value>, pdf=<string>, seed=<value>) <br/> rows/cols: Number of rows/cols (expression) <br/> min/max: Min/max value for cells (either constant value, or variable that evaluates to constant value) <br/> sparsity: fraction of non-zero cells (constant value) <br/> pdf: “uniform” (min, max) distribution, or “normal” (0,1) distribution; or “poisson” (lambda=1) distribution. string; default value is “uniform”. Note that, for the Poisson distribution, users can provide the mean/lambda parameter as follows: <br/> rand(rows=1000,cols=1000, pdf=”poisson”, lambda=2.5). <br/> The default value for lambda is 1. <br/> seed: Every invocation of rand() internally generates a random seed with which the cell values are generated. One can optionally provide a seed when repeatability is desired. <br/> Output: matrix | X = rand(rows=1
0, cols=20, min=0, max=1, pdf="uniform", sparsity=0.2) <br/> The example generates a 10 x 20 matrix, with cell values uniformly chosen at random between 0 and 1, and approximately 20% of cells will have non-zero values.
+rbind() | Row-wise matrix concatenation. Concatenates the second matrix as additional rows to the first matrix | Input: (X <matrix>, Y <matrix>) <br/>Output: <matrix> <br/> X and Y are matrices, where the number of columns in X and the number of columns in Y are the same. | A = matrix(1, rows=2,cols=3) <br/> B = matrix(2, rows=2,cols=3) <br/> C = rbind(A,B) <br/> print("Dimensions of C: " + nrow(C) + " X " + ncol(C)) <br/> Output: <br/> Dimensions of C: 4 X 3
removeEmpty() | Removes all empty rows or columns from the input matrix target X according to the specified margin. | Input : (target= X <matrix>, margin="...") <br/> Output : <matrix> <br/> Valid values for margin are "rows" or "cols". | A = removeEmpty(target=X, margin="rows")
replace() | Creates a copy of input matrix X, where all values that are equal to the scalar pattern s1 are replaced with the scalar replacement s2. | Input : (target= X <matrix>, pattern=<scalar>, replacement=<scalar>) <br/> Output : <matrix> <br/> If s1 is NaN, then all NaN values of X are treated as equal and hence replaced with s2. Positive and negative infinity are treated as different values. | A = replace(target=X, pattern=s1, replacement=s2)
seq() | Creates a single column vector with values starting from <from>, to <to>, in increments of <increment> | Input: (<from>, <to>, <increment>) <br/> Output: <matrix> | S = seq (10, 200, 10)
[36/47] incubator-systemml git commit: [SYSML-327] Add additional
algorithm tex files to Algorithms Reference
Posted by du...@apache.org.
[SYSML-327] Add additional algorithm tex files to Algorithms Reference
Documents the following Algorithms in Algorithms Reference:
ALS.tex, Cox.tex, KaplanMeier.tex, RandomForest.tex,
StepGLM.tex, StepLinRegDS.tex
Also provides updates to the following Algorithms:
SystemML_Algorithms_Reference.bib, SystemML_Algorithms_Reference.pdf,
and SystemML_Algorithms_Reference.tex
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/53e814f8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/53e814f8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/53e814f8
Branch: refs/heads/gh-pages
Commit: 53e814f89604dab94d5d614dccdb3c4c20d728e6
Parents: 3bae2a6
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Sun Oct 4 08:06:31 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Sun Oct 4 08:06:31 2015 -0700
----------------------------------------------------------------------
Algorithms Reference/ALS.tex | 277 +++++++++++++++
Algorithms Reference/Cox.tex | 319 +++++++++++++++++
Algorithms Reference/DecisionTrees.tex | 354 +++++++++++--------
Algorithms Reference/GLM.tex | 1 +
Algorithms Reference/KaplanMeier.tex | 268 ++++++++++++++
Algorithms Reference/LinReg.tex | 1 +
Algorithms Reference/RandomForest.tex | 194 ++++++++++
Algorithms Reference/StepGLM.tex | 111 ++++++
Algorithms Reference/StepLinRegDS.tex | 101 ++++++
.../SystemML_Algorithms_Reference.bib | 77 ++++
.../SystemML_Algorithms_Reference.pdf | Bin 620293 -> 1266909 bytes
.../SystemML_Algorithms_Reference.tex | 29 +-
12 files changed, 1571 insertions(+), 161 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/53e814f8/Algorithms Reference/ALS.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/ALS.tex b/Algorithms Reference/ALS.tex
new file mode 100644
index 0000000..669af33
--- /dev/null
+++ b/Algorithms Reference/ALS.tex
@@ -0,0 +1,277 @@
+\subsection{Matrix Completion via Alternating Minimizations}
+\label{matrix_completion}
+
+\noindent{\bf Description}
+\smallskip
+
+Low-rank matrix completion is an effective technique for statistical data analysis widely used in the data mining and machine learning applications.
+Matrix completion is a variant of low-rank matrix factorization with the goal of recovering a partially observed and potentially noisy matrix from a subset of its revealed entries.
+Perhaps the most popular applications in which matrix completion has been successfully applied is in the context of collaborative filtering in recommender systems.
+In this setting, the rows in the data matrix correspond to users,
+the columns to items such as movies, and entries to feedback provided by users for items.
+The goal is to predict missing entries of the rating matrix.
+This implementation uses the alternating least-squares (ALS) technique for solving large-scale matrix completion problems.\\
+
+
+\smallskip
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+ {\tt{}-f }path/\/{\tt{}ALS.dml}
+ {\tt{} -nvargs}
+ {\tt{} V=}path/file
+ {\tt{} L=}path/file
+ {\tt{} R=}path/file
+% {\tt{} VO=}path/file
+ {\tt{} rank=}int
+ {\tt{} reg=}L2$\mid$wL2%regularization
+ {\tt{} lambda=}double
+ {\tt{} fmt=}format
+
+}
+
+
+\smallskip
+\noindent{\bf Arguments}
+\begin{Description}
+ \item[{\tt V}:]
+ Location (on HDFS) to read the input (user-item) matrix $V$ to be factorized
+ \item[{\tt L}:]
+ Location (on HDFS) to write the left (user) factor matrix $L$
+ \item[{\tt R}:]
+ Location (on HDFS) to write the right (item) factor matrix $R$
+% \item[{\tt VO}:]
+% Location (on HDFS) to write the input matrix $VO$ with empty rows and columns removed (if there are any)
+ \item[{\tt rank}:] (default:\mbox{ }{\tt 10})
+ Rank of the factorization
+ \item[{\tt reg}] (default:\mbox{ }{\tt L2})
+ Regularization:\\
+ {\tt L2} = L2 regularization;\\
+ {\tt wL2} = weighted L2 regularization;\\
+ if {\tt reg} is not provided no regularization will be performed.
+ \item[{\tt lambda}:] (default:\mbox{ }{\tt 0.000001})
+ Regularization parameter
+ \item[{\tt maxi}:] (default:\mbox{ }{\tt 50})
+ Maximum number of iterations
+ \item[{\tt check}:] (default:\mbox{ }{\tt FALSE})
+ Check for convergence after every iteration, i.e., updating $L$ and $R$ once
+ \item[{\tt thr}:] (default:\mbox{ }{\tt 0.0001})
+ Assuming {\tt check=TRUE}, the algorithm stops and convergence is declared
+ if the decrease in loss in any two consecutive iterations falls below threshold {\tt thr};
+ if {\tt check=FALSE} parameter {\tt thr} is ignored.
+ \item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+ Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv}
+\end{Description}
+
+ \smallskip
+ \noindent{\bf Usage: ALS Prediction/Top-K Prediction}
+ \smallskip
+
+ {\hangindent=\parindent\noindent\it%
+ {\tt{}-f }path/\/{\tt{}ALS\_predict.dml}
+ {\tt{} -nvargs}
+ {\tt{} X=}path/file
+ {\tt{} Y=}path/file
+ {\tt{} L=}path/file
+ {\tt{} R=}path/file
+ {\tt{} Vrows=}int
+ {\tt{} Vcols=}int
+ {\tt{} fmt=}format
+
+ }\smallskip
+
+
+ \smallskip
+ {\hangindent=\parindent\noindent\it%
+ {\tt{}-f }path/\/{\tt{}ALS\_topk\_predict.dml}
+ {\tt{} -nvargs}
+ {\tt{} X=}path/file
+ {\tt{} Y=}path/file
+ {\tt{} L=}path/file
+ {\tt{} R=}path/file
+ {\tt{} V=}path/file
+ {\tt{} K=}int
+ {\tt{} fmt=}format
+
+ }\smallskip
+
+% \noindent{\bf Arguments --- Prediction}
+% \begin{Description}
+% \item[{\tt X}:]
+% Location (on HDFS) to read the input matrix $X$ containing user-ids (first column) and item-ids (second column)
+% \item[{\tt L}:]
+% Location (on HDFS) to read the left (user) factor matrix $L$
+% \item[{\tt R}:]
+% Location (on HDFS) to read the right (item) factor matrix $R$
+% \item[{\tt Y}:]
+% Location (on HDFS) to write the output matrix $Y$ containing user-ids (first column), item-ids (second column) and predicted ratings (third column)
+% \item[{\tt Vrows}:]
+% Number of rows of the user-item matrix $V$
+% \item[{\tt Vcols}]
+% Number of columns of the user-item matrix $V$
+% \item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+% Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv}
+% \end{Description}
+
+
+ \noindent{\bf Arguments --- Prediction/Top-K Prediction}
+ \begin{Description}
+ \item[{\tt V}:]
+ Location (on HDFS) to read the user-item matrix $V$
+ \item[{\tt X}:]
+ Location (on HDFS) to read the input matrix $X$ with following format:
+ \begin{itemize}
+ \item for {ALS\_predict.dml}: a 2-column matrix that contains the user-ids (first column) and the item-ids (second column),
+ \item for {ALS\_topk\_predict.dml}: a 1-column matrix that contains the user-ids.
+ \end{itemize}
+ \item[{\tt Y}:]
+ Location (on HDFS) to write the output of prediction with the following format:
+ \begin{itemize}
+ \item for {ALS\_predict.dml}: a 3-column matrix that contains the user-ids (first column), the item-ids (second column) and the predicted ratings (third column),
+ \item for {ALS\_topk\_predict.dml}: a ($K+1$)-column matrix that contains the user-ids in the first column and the top-K item-ids in the remaining $K$ columns will be stored at {\tt Y}.
+ Additionally, a matrix with the same dimensions that contains the corresponding actual top-K ratings will be stored at {\tt Y.ratings}; see below for details.
+ \end{itemize}
+% Note the following output format in predicting top-K items.
+% For a user with no available ratings in $V$ no
+% top-K items will be provided, i.e., the corresponding row in $Y$ will contains 0s.
+% Moreover, $K'<K$ items with highest predicted ratings will be provided for a user $i$
+% if the number of missing ratings $K'$ (i.e., those with 0 value in $V$) for $i$ is less than $K$.
+ \item[{\tt L}:]
+ Location (on HDFS) to read the left (user) factor matrix $L$
+ \item[{\tt R}:]
+ Location (on HDFS) to write the right (item) factor matrix $R$
+ \item[{\tt Vrows}:]
+ Number of rows of $V$ (i.e., number of users)
+ \item[{\tt Vcols}]
+ Number of columns of $V$ (i.e., number of items)
+ \item[{\tt K}:] (default:\mbox{ }{\tt 5})
+ Number of top-K items for top-K prediction
+ \item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+ Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv}
+ \end{Description}
+
+ \noindent{\bf Details}
+ \smallskip
+
+ Given an $m \times n$ input matrix $V$ and a rank parameter $r \ll \min{(m,n)}$, low-rank matrix factorization seeks to find an $m \times r$ matrix $L$ and an $r \times n$ matrix $R$ such that $V \approx LR$, i.e., we aim to approximate $V$ by the low-rank matrix $LR$.
+ The quality of the approximation is determined by an application-dependent loss function $\mathcal{L}$. We aim at finding the loss-minimizing factor matrices, i.e.,
+ \begin{equation}\label{eq:problem}
+ (L^*, R^*) = \textrm{argmin}_{L,R}{\mathcal{L}(V,L,R)}.
+ \end{equation}
+ In the context of collaborative filtering in the recommender systems it is often the case that the input matrix $V$ contains several missing entries. Such entries are coded with the 0 value and the loss function is computed only based on the nonzero entries in $V$, i.e.,
+ \begin{equation*} %\label{eq:loss}
+ \mathcal{L}=\sum_{(i,j)\in\Omega}l(V_{ij},L_{i*},R_{*j}),
+ \end{equation*}
+ where $L_{i*}$ and $R_{*j}$, respectively, denote the $i$th row of $L$ and the $j$th column of $R$, $\Omega=\{\omega_1,\dots,\omega_N\}$ denotes the training set containing the observed (nonzero) entries in $V$, and $l$ is some local loss function.
+ %for some training set $\Omega$ that contains the observed (nonzero) entries in $V$ and some local loss function $l$. In the above formula,
+
+ ALS is an optimization technique that can be used to solve quadratic problems.
+ For matrix completion, the algorithm repeatedly keeps one of the unknown matrices ($L$ or $R$) fixed and optimizes the other one. In particular, ALS alternates between recomputing the rows of $L$ in one step and the columns of $R$ in the subsequent step.
+ Our implementation of the ALS algorithm supports the loss functions summarized in Table~\ref{tab:loss_functions} commonly used for matrix completion~\cite{ZhouWSP08:als}.
+ %
+ \begin{table}[t]
+ \centering
+ \label{tab:loss_functions}
+ \begin{tabular}{|ll|} \hline
+ Loss & Definition \\ \hline
+% $\mathcal{L}_\text{Sl}$ & $\sum_{i,j} (V_{ij} - [LR]_{ij})^2$ \\
+% $\mathcal{L}_\text{Sl+L2}$ & $\mathcal{L}_\text{Sl} + \lambda \Bigl( \sum_{ik} L_{ik}^2 + \sum_{kj} R_{kj}^2 \Bigr)$ \\
+ $\mathcal{L}_\text{Nzsl}$ & $\sum_{i,j:V_{ij}\neq 0} (V_{ij} - [LR]_{ij})^2$ \\
+ $\mathcal{L}_\text{Nzsl+L2}$ & $\mathcal{L}_\text{Nzsl} + \lambda \Bigl( \sum_{ik} L_{ik}^2 + \sum_{kj} R_{kj}^2 \Bigr)$ \\
+ $\mathcal{L}_\text{Nzsl+wL2}$ & $\mathcal{L}_\text{Nzsl} + \lambda \Bigl(\sum_{ik}N_{i*} L_{ik}^2 + \sum_{kj}N_{*j} R_{kj}^2 \Bigr)$ \\ \hline
+ \end{tabular}
+ \caption{Popular loss functions supported by our ALS implementation; $N_{i*}$ and $N_{*j}$, respectively, denote the number of nonzero entries in row $i$ and column $j$ of $V$.}
+ \end{table}
+
+ Note that the matrix completion problem as defined in (\ref{eq:problem}) is a non-convex problem for all loss functions from Table~\ref{tab:loss_functions}.
+ However, when fixing one of the matrices $L$ or $R$, we get a least-squares problem with a globally optimal solution.
+ For example, for the case of $\mathcal{L}_\text{Nzsl+wL2}$ we have the following closed form solutions
+ \begin{align*}
+ L^\top_{n+1,i*} &\leftarrow (R^{(i)}_n {[R^{(i)}_n]}^\top + \lambda N_2 I)^{-1} R_n V^\top_{i*}, \\
+ R_{n+1,*j} &\leftarrow ({[L^{(j)}_{n+1}]}^\top L^{(j)}_{n+1} + \lambda N_1 I)^{-1} L^\top_{n+1} V_{*j},
+ \end{align*}
+ where $L_{n+1,i*}$ (resp. $R_{n+1,*j}$) denotes the $i$th row of $L_{n+1}$ (resp. $j$th column of $R_{n+1}$), $\lambda$ denotes
+ the regularization parameter, $I$ is the identity matrix of appropriate dimensionality,
+ $V_{i*}$ (resp. $V_{*j}$) denotes the revealed entries in row $i$ (column $j$),
+ $R^{(i)}_n$ (resp. $L^{(j)}_{n+1}$) refers to the corresponding columns of $R_n$ (rows of $L_{n+1}$),
+ and $N_1$ (resp. $N_2$) denotes a diagonal matrix that contains the number of nonzero entries in row $i$ (column $j$) of $V$.
+
+% For example, for the case of $\mathcal{L}_\text{Sl-L2}$ we have the following closed form solutions
+% \begin{align*}
+% L^\top_{n+1,i*} &\leftarrow (R_n {[R_n]}^\top + \lambda I)^{-1} R_n V^\top_{i*}, \\
+% R_{n+1,*j} &\leftarrow ({[L_{n+1}]}^\top L_{n+1} + \lambda I)^{-1} L^\top_{n+1} V_{*j},
+% \end{align*}
+% where $L_{n+1,i*}$ (resp. $R_{n+1,*j}$) denotes the $i$th row of $L_{n+1}$ (resp. $j$th column of $R_{n+1}$), $\lambda$ denotes
+% the regularization parameter and $I$ is the identity matrix of appropriate dimensionality.
+% For the case of $\mathcal{L}_\text{Nzsl}$ we need to remove the equation that correspond to zero entries of $V$ from the least-squares problems.
+% With wL2 we get the following equations
+% \begin{align*}
+% L^\top_{n+1,i*} &\leftarrow (R^{(i)}_n {[R^{(i)}_n]}^\top + \lambda N_2 I)^{-1} R_n V^\top_{i*}, \\
+% R_{n+1,*j} &\leftarrow ({[L^{(j)}_{n+1}]}^\top L^{(j)}_{n+1} + \lambda N_1 I)^{-1} L^\top_{n+1} V_{*j},
+% \end{align*}
+% where $V_{i*}$ (resp. $V_{*j}$) denotes the revealed entries in row $i$ (column $j$),
+% $R^{(i)}_n$ (resp. $L^{(j)}_{n+1}$) refers to the corresponding columns of $R_n$ (rows of $L_{n+1}$),
+% and $N_1$ (resp. $N_2$) denotes a diagonal matrix that contains the number of nonzero entries in row $i$ (column $j$) of $V$.
+
+ \textbf{Prediction.}
+ Based on the factor matrices computed by ALS we provide two prediction scripts:
+ \begin{Enumerate}
+ \item {\tt ALS\_predict.dml} computes the predicted ratings for a given list of users and items;
+ \item {\tt ALS\_topk\_predict.dml} computes top-K item (where $K$ is given as input) with highest predicted ratings together with their corresponding ratings for a given list of users.
+ \end{Enumerate}
+
+ \smallskip
+ \noindent{\bf Returns}
+ \smallskip
+
+ We output the factor matrices $L$ and $R$ after the algorithm has converged. The algorithm is declared as converged if one of the two criteria is meet:
+ (1) the decrease in the value of loss function falls below {\tt thr}
+ given as an input parameter (if parameter {\tt check=TRUE}), or (2) the maximum number of iterations (defined as parameter {\tt maxi}) is reached.
+ Note that for a given user $i$ prediction is possible only if user $i$ has rated at least one item, i.e., row $i$ in matrix $V$ has at least one nonzero entry.
+ In case, some users have not rated any items the corresponding factor in $L$ will be all 0s.
+ Similarly if some items have not been rated at all the corresponding factors in $R$ will contain only 0s.
+ Our prediction scripts output the predicted ratings for a given list of users and items as well as the top-K items with highest predicted ratings together with the predicted ratings for a given list of users. Note that the predictions will only be provided for the users who have rated at least one item, i.e., the corresponding rows contain at least one nonzero entry.
+% Moreover in the case of top-K prediction, if the number of predicted ratings---i.e., missing entries--- for some user $i$ is less than the input parameter $K$, all the predicted ratings for user $i$ will be provided.
+
+
+
+
+
+
+ \smallskip
+ \noindent{\bf Examples}
+ \smallskip
+
+% {\hangindent=\parindent\noindent\tt
+% \hml -f ALS.dml -nvargs V=/user/biadmin/V L=/user/biadmin/L R=/user/biadmin/R rank=10 reg="L2" lambda=0.0001 fmt=csv
+%
+% }
+
+ {\hangindent=\parindent\noindent\tt
+ \hml -f ALS.dml -nvargs V=/user/biadmin/V L=/user/biadmin/L R=/user/biadmin/R rank=10 reg="wL2" lambda=0.0001 maxi=50 check=TRUE thr=0.001 fmt=csv
+
+ }
+
+ \noindent To compute predicted ratings for a given list of users and items:
+
+ {\hangindent=\parindent\noindent\tt
+ \hml -f ALS-predict.dml -nvargs X=/user/biadmin/X Y=/user/biadmin/Y L=/user/biadmin/L R=/user/biadmin/R Vrows=100000 Vcols=10000 fmt=csv
+
+ }
+
+ \noindent To compute top-K items with highest predicted ratings together with the predicted ratings for a given list of users:
+
+ {\hangindent=\parindent\noindent\tt
+ \hml -f ALS-top-predict.dml -nvargs X=/user/biadmin/X Y=/user/biadmin/Y L=/user/biadmin/L R=/user/biadmin/R V=/user/biadmin/V K=10 fmt=csv
+
+ }
+
+
+%
+%\begin{itemize}
+% \item Y. Zhou, D. K. Wilkinson, R. Schreiber, and R. Pan. \newblock{Large-scale parallel collaborative flitering for the Netflix prize}. In Proceedings of the International
+% Conference on Algorithmic Aspects in Information and Management (AAIM), 2008, 337-348.
+%\end{itemize}
+
+
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/53e814f8/Algorithms Reference/Cox.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/Cox.tex b/Algorithms Reference/Cox.tex
new file mode 100644
index 0000000..48613b7
--- /dev/null
+++ b/Algorithms Reference/Cox.tex
@@ -0,0 +1,319 @@
+\subsection{Cox Proportional Hazard Regression Model}
+
+\noindent{\bf Description}
+\smallskip
+
+
+The Cox (proportional hazard or PH) is a semi-parametric statistical approach commonly used for analyzing survival data.
+Unlike non-parametric approaches, e.g., the Kaplan-Meier estimates (Section \ref{sec:kaplan-meier}), which can be used to analyze single sample of survival data or to compare between groups of survival times, the Cox PH models the dependency of the survival times on the values of {\it explanatory variables} (i.e., covariates) recorded for each individual at the time origin. Our focus is on covariates that do not change value over time, i.e., time-independent covariates, and that may be categorical (ordinal or nominal) as well as continuous-valued. \\
+
+
+\smallskip
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+{\tt{}-f }path/\/{\tt{}Cox.dml}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} TE=}path/file
+{\tt{} F=}path/file
+{\tt{} R=}path/file
+{\tt{} M=}path/file
+{\tt{} S=}path/file
+{\tt{} T=}path/file
+{\tt{} COV=}path/file
+{\tt{} RT=}path/file
+{\tt{} XO=}path/file
+{\tt{} MF=}path/file
+{\tt{} alpha=}double
+{\tt{} fmt=}format
+
+}
+
+\smallskip
+\noindent{\bf Arguments --- Model Fitting/Prediction}
+\begin{Description}
+\item[{\tt X}:]
+Location (on HDFS) to read the input matrix of the survival data containing:
+\begin{Itemize}
+ \item timestamps,
+ \item whether event occurred (1) or data is censored (0),
+ \item feature vectors
+\end{Itemize}
+\item[{\tt Y}:]
+Location (on HDFS) to the read matrix used for prediction
+\item[{\tt TE}:]
+Location (on HDFS) to read the 1-column matrix $TE$ that contains the column indices of the input matrix $X$ corresponding to timestamps (first entry) and event information (second entry)
+\item[{\tt F}:]
+Location (on HDFS) to read the 1-column matrix $F$ that contains the column indices of the input matrix $X$ corresponding to the features to be used for fitting the Cox model
+\item[{\tt R}:] (default:\mbox{ }{\tt " "})
+If factors (i.e., categorical features) are available in the input matrix $X$, location (on HDFS) to read matrix $R$ containing the start (first column) and end (second column) indices of each factor in $X$;
+alternatively, user can specify the indices of the baseline level of each factor which needs to be removed from $X$. If $R$ is not provided by default all variables are considered to be continuous-valued.
+\item[{\tt M}:]
+Location (on HDFS) to store the results of Cox regression analysis including regression coefficients $\beta_j$s, their standard errors, confidence intervals, and $P$-values
+\item[{\tt S}:] (default:\mbox{ }{\tt " "})
+Location (on HDFS) to store a summary of some statistics of the fitted model including number of records, number of events, log-likelihood, AIC, Rsquare (Cox \& Snell), and maximum possible Rsquare
+\item[{\tt T}:] (default:\mbox{ }{\tt " "})
+Location (on HDFS) to store the results of Likelihood ratio test, Wald test, and Score (log-rank) test of the fitted model
+\item[{\tt COV}:]
+Location (on HDFS) to store the variance-covariance matrix of $\beta_j$s; note that parameter {\tt COV} needs to provided as input to prediction.
+\item[{\tt RT}:]
+Location (on HDFS) to store matrix $RT$ containing the order-preserving recoded timestamps from $X$; note that parameter {\tt RT} needs to provided as input for prediction.
+\item[{\tt XO}:]
+Location (on HDFS) to store the input matrix $X$ ordered by the timestamps; note that parameter {\tt XO} needs to provided as input for prediction.
+\item[{\tt MF}:]
+Location (on HDFS) to store column indices of $X$ excluding the baseline factors if available; note that parameter {\tt MF} needs to provided as input for prediction.
+\item[{\tt P}]
+Location (on HDFS) to store matrix $P$ containing the results of prediction
+\item[{\tt alpha}](default:\mbox{ }{\tt 0.05})
+Parameter to compute a $100(1-\alpha)\%$ confidence interval for $\beta_j$s
+\item[{\tt tol}](default:\mbox{ }{\tt 0.000001})
+Tolerance (epsilon) used in the convergence criterion
+\item[{\tt moi}:] (default:\mbox{ }{\tt 100})
+Maximum number of outer (Fisher scoring) iterations
+\item[{\tt mii}:] (default:\mbox{ }{\tt 0})
+Maximum number of inner (conjugate gradient) iterations, or~0 if no maximum
+limit provided
+\item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+see read/write functions in SystemML Language Reference for details.
+\end{Description}
+
+
+ \smallskip
+ \noindent{\bf Usage: Cox Prediction}
+ \smallskip
+
+ {\hangindent=\parindent\noindent\it%
+ {\tt{}-f }path/\/{\tt{}Cox-predict.dml}
+ {\tt{} -nvargs}
+ {\tt{} X=}path/file
+ {\tt{} RT=}path/file
+ {\tt{} M=}path/file
+ {\tt{} Y=}path/file
+ {\tt{} COV=}path/file
+ {\tt{} MF=}path/file
+ {\tt{} P=}path/file
+ {\tt{} fmt=}format
+
+ }\smallskip
+
+% \noindent{\bf Arguments --- Prediction}
+% \begin{Description}
+% \item[{\tt X}:]
+% Location (on HDFS) to read the input matrix of the survival data sorted by the timestamps including:
+% \begin{Itemize}
+% \item timestamps,
+% \item whether event occurred (1) or data is censored (0),
+% \item feature vectors
+% \end{Itemize}
+% \item[{\tt RT}:]
+% Location to read column matrix $RT$ containing the (order preserving) recoded timestamps from X (output by {\tt Cox.dml})
+% \item[{\tt M}:]
+% Location to read matrix $M$ containing the fitted Cox model (see below for the schema)
+% \item[{\tt Y}:]
+% Location to the read matrix used for prediction
+% \item[{\tt COV}:]
+% Location to read the variance-covariance matrix of the regression coefficients (output by {\tt Cox.dml})
+% \item[{\tt MF}]
+% Location to store column indices of $X$ excluding the baseline factors if available (output by {\tt Cox.dml})
+% \item[{\tt P}]
+% Location to store matrix $P$ containing the results of prediction
+% \item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+% Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv}
+% \end{Description}
+
+
+
+\noindent{\bf Details}
+\smallskip
+
+
+In Cox PH regression model the relationship between the hazard function---i.e., the probability of event occurrence at a given time---and the covariates is described as
+\begin{equation}
+h_i(t)=h_0(t)\exp\Bigl\{ \sum_{j=1}^{p} \beta_jx_{ij} \Bigr\}, \label{eq:coxph}
+\end{equation}
+where the hazard function for the $i$th individual ($i\in\{1,2,\ldots,n\}$) depends on a set of $p$ covariates $x_i=(x_{i1},x_{i2},\ldots,x_{ip})$, whose importance is measured by the magnitude of the corresponding coefficients
+$\beta=(\beta_1,\beta_2,\ldots,\beta_p)$. The term $h_0(t)$ is the baseline hazard and is related to a hazard value if all covariates equal 0.
+In the Cox PH model the hazard function for the individuals may vary over time, however the baseline hazard is estimated non-parametrically and can take any form.
+Note that re-writing~(\ref{eq:coxph}) we have
+\begin{equation*}
+\log\biggl\{ \frac{h_i(t)}{h_0(t)} \biggr\} = \sum_{j=1}^{p} \beta_jx_{ij}.
+\end{equation*}
+Thus, the Cox PH model is essentially a linear model for the logarithm of the hazard ratio and the hazard of event for any individual is a constant multiple of the hazard of any other.
+%Consequently, the Cox model is a proportional hazard model.
+We follow similar notation and methodology as in~\cite[Sec.~3]{collett2003:kaplanmeier}.
+For completeness we briefly discuss the equations used in our implementation.
+
+
+\textbf{Factors in the model.}
+Note that if some of the feature variables are factors they need to {\it dummy code} as follows.
+Let $\alpha$ be such a variable (i.e., a factor) with $a$ levels.
+We introduce $a-1$ indicator (or dummy coded) variables $X_2,X_3\ldots,X_a$ with $X_j=1$ if $\alpha=j$ and 0 otherwise, for $j\in\{ 2,3,\ldots,a\}$.
+In particular, one of $a$ levels of $\alpha$ will be considered as the baseline and is not included in the model.
+In our implementation, user can specify a baseline level for each of the factor (as selecting the baseline level for each factor is arbitrary).
+On the other hand, if for a given factor $\alpha$ no baseline is specified by the user, the most frequent level of $\alpha$ will be considered as the baseline.
+
+
+\textbf{Fitting the model.}
+We estimate the coefficients of the Cox model via negative log-likelihood method.
+In particular the Cox PH model is fitted by using trust region Newton method with conjugate gradient~\cite{Nocedal2006:Optimization}.
+%The likelihood for the PH hazard model is given by
+%\begin{equation*}
+%\prod_{i=1}^{n} {\Bigg\{ \frac{\exp(\vec{\beta}^\top\vec{x_i})}{\sum_{l\in %R(t_i)\exp(\vec{\beta}\vec{x}_l)}} \Biggr\}}^\delta_i,
+%\end{equation*}
+%where $\delta_i$ is an event indicator, which is 0 if the $i$th survival time is censored or 1 otherwise, and $R(t_i)$ is the risk set defined as the set of individuals who die at time $t_i$ or later.
+Define the risk set $R(t_j)$ at time $t_j$ to be the set of individuals who die at time $t_i$ or later.
+The PH model assumes that survival times are distinct. In order to handle tied observations
+we use the \emph{Breslow} approximation of the likelihood function
+\begin{equation*}
+\mathcal{L}=\prod_{j=1}^{r} \frac{\exp(\beta^\top s_j)}{{\bigg\{ \sum_{l\in R(t_j)} \exp(\beta^\top x_l) \biggr\}}^{d_j}},
+\end{equation*}
+where $d_j$ is number individuals who die at time $t_j$ and $s_j$ denotes the element-wise sum of the covariates for those individuals who die at time $t_j$, $j=1,2,\ldots,r$, i.e.,
+the $h$th element of $s_j$ is given by $s_{hj}=\sum_{k=1}^{d_j}x_{hjk}$, where $x_{hjk}$ is the value of $h$th variable ($h\in \{1,2,\ldots,p\}$) for the $k$th of the $d_j$ individuals ($k\in\{ 1,2,\ldots,d_j \}$) who die at the $j$th death time ($j\in\{ 1,2,\ldots,r \}$).
+
+\textbf{Standard error and confidence interval for coefficients.}
+Note that the variance-covariance matrix of the estimated coefficients $\hat{\beta}$ can be approximated by the inverse of the Hessian evaluated at $\hat{\beta}$. The square root of the diagonal elements of this matrix are the standard errors of estimated coefficients.
+Once the standard errors of the coefficients $se(\hat{\beta})$ is obtained we can compute a $100(1-\alpha)\%$ confidence interval using $\hat{\beta}\pm z_{\alpha/2}se(\hat{\beta})$, where $z_{\alpha/2}$ is the upper $\alpha/2$-point of the standard normal distribution.
+In {\tt Cox.dml}, we utilize the build-in function {\tt inv()} to compute the inverse of the Hessian. Note that this build-in function can be used only if the Hessian fits in the main memory of a single machine.
+
+
+\textbf{Wald test, likelihood ratio test, and log-rank test.}
+In order to test the {\it null hypothesis} that all of the coefficients $\beta_j$s are 0, our implementation provides three statistical test: {\it Wald test}, {\it likelihood ratio test}, the {\it log-rank test} (also known as the {\it score test}).
+Let $p$ be the number of coefficients.
+The Wald test is based on the test statistic ${\hat{\beta}}^2/{se(\hat{\beta})}^2$, which is compared to percentage points of the Chi-squared distribution to obtain the $P$-value.
+The likelihood ratio test relies on the test statistic $-2\log\{ {L}(\textbf{0})/{L}(\hat{\beta}) \}$ ($\textbf{0}$ denotes a zero vector of size $p$ ) which has an approximate Chi-squared distribution with $p$ degrees of freedom under the null hypothesis that all $\beta_j$s are 0.
+The Log-rank test is based on the test statistic
+$l=\nabla^\top L(\textbf{0}) {\mathcal{H}}^{-1}(\textbf{0}) \nabla L(\textbf{0})$,
+where $\nabla L(\textbf{0})$ is the gradient of $L$ and $\mathcal{H}(\textbf{0})$ is the Hessian of $L$ evaluated at \textbf{0}. Under the null hypothesis that $\beta=\textbf{0}$, $l$ has a Chi-squared distribution on $p$ degrees of freedom.
+
+
+% Scoring
+\textbf{Prediction.}
+Once the parameters of the model are fitted, we compute the following predictions together with their standard errors
+\begin{itemize}
+ \item linear predictors,
+ \item risk, and
+ \item estimated cumulative hazard.
+\end{itemize}
+Given feature vector $X_i$ for individual $i$, we obtain the above predictions at time $t$ as follows.
+The linear predictors (denoted as $\mathcal{LP}$) as well as the risk (denoted as $\mathcal{R}$) are computed relative to a baseline whose feature values are the mean of the values in the corresponding features.
+Let $X_i^\text{rel} = X_i - \mu$, where $\mu$ is a row vector that contains the mean values for each feature.
+We have $\mathcal{LP}=X_i^\text{rel} \hat{\beta}$ and $\mathcal{R}=\exp\{ X_i^\text{rel}\hat{\beta} \}$.
+The standard errors of the linear predictors $se\{\mathcal{LP} \}$ are computed as the square root of ${(X_i^\text{rel})}^\top V(\hat{\beta}) X_i^\text{rel}$ and the standard error of the risk $se\{ \mathcal{R} \}$ are given by the square root of
+${(X_i^\text{rel} \odot \mathcal{R})}^\top V(\hat{\beta}) (X_i^\text{rel} \odot \mathcal{R})$, where $V(\hat{\beta})$ is the variance-covariance matrix of the coefficients and $\odot$ is the element-wise multiplication.
+
+We estimate the cumulative hazard function for individual $i$ by
+\begin{equation*}
+\hat{H}_i(t) = \exp(\hat{\beta}^\top X_i) \hat{H}_0(t),
+\end{equation*}
+where $\hat{H}_0(t)$ is the \emph{Breslow estimate} of the cumulative baseline hazard given by
+\begin{equation*}
+\hat{H}_0(t) = \sum_{j=1}^{k} \frac{d_j}{\sum_{l\in R(t_{(j)})} \exp(\hat{\beta}^\top X_l)}.
+\end{equation*}
+In the equation above, as before, $d_j$ is the number of deaths, and $R(t_{(j)})$ is the risk set at time $t_{(j)}$, for $t_{(k)} \leq t \leq t_{(k+1)}$, $k=1,2,\ldots,r-1$.
+The standard error of $\hat{H}_i(t)$ is obtained using the estimation
+\begin{equation*}
+se\{ \hat{H}_i(t) \} = \sum_{j=1}^{k} \frac{d_j}{ {\left[ \sum_{l\in R(t_{(j)})} \exp(X_l\hat{\beta}) \right]}^2 } + J_i^\top(t) V(\hat{\beta}) J_i(t),
+\end{equation*}
+where
+\begin{equation*}
+J_i(t) = \sum_{j-1}^{k} d_j \frac{\sum_{l\in R(t_{(j)})} (X_l-X_i)\exp \{ (X_l-X_i)\hat{\beta} \}}{ {\left[ \sum_{l\in R(t_{(j)})} \exp\{(X_l-X_i)\hat{\beta}\} \right]}^2 },
+\end{equation*}
+for $t_{(k)} \leq t \leq t_{(k+1)}$, $k=1,2,\ldots,r-1$.
+
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+
+Blow we list the results of fitting a Cox regression model stored in matrix {\tt M} with the following schema:
+\begin{itemize}
+ \item Column 1: estimated regression coefficients $\hat{\beta}$
+ \item Column 2: $\exp(\hat{\beta})$
+ \item Column 3: standard error of the estimated coefficients $se\{\hat{\beta}\}$
+ \item Column 4: ratio of $\hat{\beta}$ to $se\{\hat{\beta}\}$ denoted by $Z$
+ \item Column 5: $P$-value of $Z$
+ \item Column 6: lower bound of $100(1-\alpha)\%$ confidence interval for $\hat{\beta}$
+ \item Column 7: upper bound of $100(1-\alpha)\%$ confidence interval for $\hat{\beta}$.
+\end{itemize}
+Note that above $Z$ is the Wald test statistic which is asymptotically standard normal under the hypothesis that $\beta=\textbf{0}$.
+
+Moreover, {\tt Cox.dml} outputs two log files {\tt S} and {\tt T} containing a summary statistics of the fitted model as follows.
+File {\tt S} stores the following information
+\begin{itemize}
+ \item Line 1: total number of observations
+ \item Line 2: total number of events
+ \item Line 3: log-likelihood (of the fitted model)
+ \item Line 4: AIC
+ \item Line 5: Cox \& Snell Rsquare
+ \item Line 6: maximum possible Rsquare.
+\end{itemize}
+Above, the AIC is computed as in (\ref{eq:AIC}),
+the Cox \& Snell Rsquare is equal to $1-\exp\{ -l/n \}$, where $l$ is the log-rank test statistic as discussed above and $n$ is total number of observations,
+and the maximum possible Rsquare computed as $1-\exp\{ -2 L(\textbf{0})/n \}$ , where $L(\textbf{0})$ denotes the initial likelihood.
+
+
+File {\tt T} contains the following information
+\begin{itemize}
+ \item Line 1: Likelihood ratio test statistic, degree of freedom of the corresponding Chi-squared distribution, $P$-value
+ \item Line 2: Wald test statistic, degree of freedom of the corresponding Chi-squared distribution, $P$-value
+ \item Line 3: Score (log-rank) test statistic, degree of freedom of the corresponding Chi-squared distribution, $P$-value.
+\end{itemize}
+
+Additionally, the following matrices will be stored. Note that these matrices are required for prediction.
+\begin{itemize}
+ \item Order-preserving recoded timestamps $RT$, i.e., contiguously numbered from 1 $\ldots$ \#timestamps
+ \item Feature matrix ordered by the timestamps $XO$
+ \item Variance-covariance matrix of the coefficients $COV$
+ \item Column indices of the feature matrix with baseline factors removed (if available) $MF$.
+\end{itemize}
+
+
+\textbf{Prediction}
+Finally, the results of prediction is stored in Matrix $P$ with the following schema
+\begin{itemize}
+ \item Column 1: linear predictors
+ \item Column 2: standard error of the linear predictors
+ \item Column 3: risk
+ \item Column 4: standard error of the risk
+ \item Column 5: estimated cumulative hazard
+ \item Column 6: standard error of the estimated cumulative hazard.
+\end{itemize}
+
+
+
+
+\smallskip
+\noindent{\bf Examples}
+\smallskip
+
+{\hangindent=\parindent\noindent\tt
+ \hml -f Cox.dml -nvargs X=/user/biadmin/X.mtx TE=/user/biadmin/TE
+ F=/user/biadmin/F R=/user/biadmin/R M=/user/biadmin/model.csv
+ T=/user/biadmin/test.csv COV=/user/biadmin/var-covar.csv XO=/user/biadmin/X-sorted.mtx fmt=csv
+
+}\smallskip
+
+{\hangindent=\parindent\noindent\tt
+ \hml -f Cox.dml -nvargs X=/user/biadmin/X.mtx TE=/user/biadmin/TE
+ F=/user/biadmin/F R=/user/biadmin/R M=/user/biadmin/model.csv
+ T=/user/biadmin/test.csv COV=/user/biadmin/var-covar.csv
+ RT=/user/biadmin/recoded-timestamps.csv XO=/user/biadmin/X-sorted.csv
+ MF=/user/biadmin/baseline.csv alpha=0.01 tol=0.000001 moi=100 mii=20 fmt=csv
+
+}\smallskip
+
+\noindent To compute predictions:
+
+{\hangindent=\parindent\noindent\tt
+ \hml -f Cox-predict.dml -nvargs X=/user/biadmin/X-sorted.mtx
+ RT=/user/biadmin/recoded-timestamps.csv
+ M=/user/biadmin/model.csv Y=/user/biadmin/Y.mtx COV=/user/biadmin/var-covar.csv
+ MF=/user/biadmin/baseline.csv P=/user/biadmin/predictions.csv fmt=csv
+
+}
+
+
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/53e814f8/Algorithms Reference/DecisionTrees.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/DecisionTrees.tex b/Algorithms Reference/DecisionTrees.tex
index f404dfc..01418c7 100644
--- a/Algorithms Reference/DecisionTrees.tex
+++ b/Algorithms Reference/DecisionTrees.tex
@@ -1,120 +1,143 @@
\subsection{Decision Trees}
-\label{decision_trees}
+\label{sec:decision_trees}
\noindent{\bf Description}
-Decision trees (for classification) is a classifier that is considered
+\smallskip
+
+
+Decision tree (for classification) is a classifier that is considered
more interpretable than other statistical classifiers. This implementation
is well-suited to handle large-scale data and builds a (binary) decision
tree in parallel.\\
+\smallskip
\noindent{\bf Usage}
-\begin{tabbing}
-\texttt{-f} \textit{path}/\texttt{decision-tree.dml -nvargs}
-\=\texttt{X=}\textit{path}/\textit{file}
- \texttt{Y=}\textit{path}/\textit{file}
- \texttt{types=}\textit{path}/\textit{file}\\
-\>\texttt{model=}\textit{path}/\textit{file}
- \texttt{bins=}\textit{int}
- \texttt{depth=}\textit{int}\\
-\>\texttt{num\_leaf=}\textit{int}
- \texttt{num\_samples=}\textit{int}\\
-\>\texttt{Log=}\textit{path}/\textit{file}
- \texttt{fmt=}\textit{csv}$\vert$\textit{text}
-\end{tabbing}
-
-\begin{tabbing}
-\texttt{-f} \textit{path}/\texttt{decision-tree-predict.dml -nvargs}
-\=\texttt{X=}\textit{path}/\textit{file}
- \texttt{Y=}\textit{path}/\textit{file}
- \texttt{model=}\textit{path}/\textit{file}\\
-\>\texttt{fmt=}\textit{csv}$\vert$\textit{text}
- \texttt{accuracy=}\textit{path}/\textit{file}\\
-\>\texttt{confusion=}\textit{path}/\textit{file}
- \texttt{predictions=}\textit{path}/\textit{file}
-\end{tabbing}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+ {\tt{}-f }path/\/{\tt{}decision-tree.dml}
+ {\tt{} -nvargs}
+ {\tt{} X=}path/file
+ {\tt{} Y=}path/file
+ {\tt{} R=}path/file
+ {\tt{} bins=}integer
+ {\tt{} depth=}integer
+ {\tt{} num\_leaf=}integer
+ {\tt{} num\_samples=}integer
+ {\tt{} impurity=}Gini$\mid$entropy
+ {\tt{} M=}path/file
+ {\tt{} O=}path/file
+ {\tt{} S\_map=}path/file
+ {\tt{} C\_map=}path/file
+ {\tt{} fmt=}format
+
+}
+ \smallskip
+ \noindent{\bf Usage: Prediction}
+ \smallskip
+
+ {\hangindent=\parindent\noindent\it%
+ {\tt{}-f }path/\/{\tt{}decision-tree-predict.dml}
+ {\tt{} -nvargs}
+ {\tt{} X=}path/file
+ {\tt{} Y=}path/file
+ {\tt{} R=}path/file
+ {\tt{} M=}path/file
+ {\tt{} P=}path/file
+ {\tt{} A=}path/file
+ {\tt{} CM=}path/file
+ {\tt{} fmt=}format
+
+ }\smallskip
+
+
\noindent{\bf Arguments}
+\begin{Description}
+ \item[{\tt X}:]
+ Location (on HDFS) to read the matrix of feature vectors;
+ each row constitutes one feature vector. Note that categorical features in $X$ need to be both recoded and dummy coded.
+ \item[{\tt Y}:]
+ Location (on HDFS) to read the matrix of (categorical)
+ labels that correspond to feature vectors in $X$. Note that class labels are assumed to be both recoded and dummy coded.
+ This argument is optional for prediction.
+ \item[{\tt R}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to read matrix $R$ which for each feature in $X$ contains column-ids (first column), start indices (second column), and end indices (third column).
+ If $R$ is not provided by default all features are assumed to be continuous-valued.
+ \item[{\tt bins}:] (default:\mbox{ }{\tt 20})
+ Number of thresholds to choose for each continuous-valued feature (determined by equi-height binning).
+ \item[{\tt depth}:] (default:\mbox{ }{\tt 25})
+ Maximum depth of the learned tree
+ \item[{\tt num\_leaf}:] (default:\mbox{ }{\tt 10})
+ Parameter that controls pruning. The tree
+ is not expanded if a node receives less than {\tt num\_leaf} training examples.
+ \item[{\tt num\_samples}:] (default:\mbox{ }{\tt 3000})
+ Parameter that decides when to switch to in-memory building of subtrees. If a node $v$ receives less than {\tt num\_samples}
+ training examples then this implementation switches to an in-memory subtree
+ building procedure to build the subtree under $v$ in its entirety.
+ \item[{\tt impurity}:] (default:\mbox{ }{\tt "Gini"})
+ Impurity measure used at internal nodes of the tree for selecting which features to split on. Possible value are entropy or Gini.
+ \item[{\tt M}:]
+ Location (on HDFS) to write matrix $M$ containing the learned decision tree (see below for the schema)
+ \item[{\tt O}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to store the training accuracy (\%). Note that this argument is optional.
+ \item[{\tt A}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to store the testing accuracy (\%) from a
+ held-out test set during prediction. Note that this argument is optional.
+ \item[{\tt P}:]
+ Location (on HDFS) to store predictions for a held-out test set
+ \item[{\tt CM}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to store the confusion matrix computed using a held-out test set. Note that this argument is optional.
+ \item[{\tt S\_map}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to write the mappings from the continuous-valued feature-ids to the global feature-ids in $X$ (see below for details). Note that this argument is optional.
+ \item[{\tt C\_map}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to write the mappings from the categorical feature-ids to the global feature-ids in $X$ (see below for details). Note that this argument is optional.
+ \item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+ Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+ see read/write functions in SystemML Language Reference for details.
+\end{Description}
-\begin{itemize}
-\item X: Location (on HDFS) to read the matrix of feature vectors;
-each row constitutes one feature vector.
-\item Y: Location (on HDFS) to read the one-column matrix of (categorical)
-labels that correspond to feature vectors in X. Classes are assumed to be
-contiguously labeled beginning from 1. Note that, this argument is optional
-for prediction.
-\item model: Location (on HDFS) that contains the learnt decision tree.
-\item types: Location (on HDFS) that contains each feature's type. 1 denotes
-continuous-valued (scale) and 2 denotes categorical.
-\item bins (default: {\tt 50}): Number of thresholds to choose for each
-continuous-valued feature (deterimined by equi-height binning).
-\item depth (default: {\tt 10}): Maximum depth of the learnt decision tree.
-\item num\_leaf (default: {\tt 1}): Parameter that controls pruning. The tree
-is not expanded if a node receives less than num\_leaf training examples.
-\item num\_samples (default: {\tt 10}): Parameter that decides when to switch
-to in-memory building of subtrees. If a node $v$ receives less than num\_samples
-training examples then this implementation switches to an in-memory subtree
-building procedure to build the subtree under $v$ in its entirety.
-\item Log: Location (on HDFS) that collects various useful metrics that indicate
-training progress.
-\item predictions: Location (on HDFS) to store predictions for a held-out test set.
-Note that, this is an optional argument.
-\item fmt (default: {\tt text}): Specifies the output format. Choice of
-comma-separated values (csv) or as a sparse-matrix (text).
-\item accuracy: Location (on HDFS) to store the testing accuracy from a
-held-out test set during prediction. Note that, this is an optional argument.
-\item confusion: Location (on HDFS) to store the confusion matrix
-computed using a held-out test set. Note that, this is an optional
-argument.
-\end{itemize}
-\noindent{\bf Details}
+ \noindent{\bf Details}
+ \smallskip
+
-Decision trees (Breiman et al, 1984) are simple models of
+Decision trees~\cite{BreimanFOS84:dtree} are simple models of
classification that, due to their structure, are easy to
-interpret. Given an example feature vector, each node in the learnt
+interpret. Given an example feature vector, each node in the learned
tree runs a simple test on it. Based on the result of the test, the
example is either diverted to the left subtree or to the right
subtree. Once the example reaches a leaf, then the label stored at the
leaf is returned as the prediction for the example.
-\par
Building a decision tree from a fully labeled training set entails
-choosing appropriate tests for each internal node in the tree and this
-is usually performed in a top-down manner. Choosing a test requires
+choosing appropriate splitting tests for each internal node in the tree and this is usually performed in a top-down manner.
+The splitting test (denoted by $s$) requires
first choosing a feature $j$ and depending on the type of $j$, either
a threshold $\sigma$, in case $j$ is continuous-valued, or a subset of
values $S \subseteq \text{Dom}(j)$ where $\text{Dom}(j)$ denotes
domain of $j$, in case it is categorical. For continuous-valued
features the test is thus of form $x_j < \sigma$ and for categorical
-features it is of form $x_j \in S$, where $x_j$ denotes the $j^{th}$
+features it is of form $x_j \in S$, where $x_j$ denotes the $j$th
feature value of feature vector $x$. One way to determine which test
-to include, is to compare impurities of the subtrees induced by the
-test and this implementation uses {\it Gini impurity}.
+to include, is to compare impurities of the tree nodes induced by the test.
+The {\it node impurity} measures the homogeneity of the labels at the node. This implementation supports two commonly used impurity measures (denoted by $\mathcal{I}$): {\it Entropy} $\mathcal{E}=\sum_{i=1}^{C}-f_i \log f_i$, as well as {\it Gini impurity} $\mathcal{G}=\sum_{i=1}^{C}f_i (1-f_i)$, where $C$ denotes the number of unique labels and $f_i$ is the frequency of label $i$.
+Once the impurity at the tree nodes has been obtained, the {\it best split} is chosen from a set of possible splits that maximizes the {\it information gain} at the node, i.e., $\arg\max_{s}\mathcal{IG}(X,s)$, where $\mathcal{IG}(X,s)$ denotes the information gain when the splitting test $s$ partitions the feature matrix $X$.
+Assuming that $s$ partitions $X$ that contains $N$ feature vectors into $X_\text{left}$ and $X_\text{right}$ each including $N_\text{left}$ and $N_\text{right}$ feature vectors, respectively, $\mathcal{IG}(X,s)$ is given by
+\begin{equation*}
+\mathcal{IG}(X,s)=\mathcal{I}(X)-\frac{N_\text{left}}{N}\mathcal{I}(X_\text{left})-\frac{N_\text{right}}{N}\mathcal{I}(X_\text{right}),
+\end{equation*}
+where $\mathcal{I}\in\{\mathcal{E},\mathcal{G}\}$.
+In the following we discuss the implementation details specific to {\tt decision-tree.dml}.
-\par
-The current implementation allows the user to specify the maximum
-depth of the learnt tree using the argument {\it depth}. It also
-allows for some automated pruning via the argument {\it num\_leaf}. If
-a node receives $\leq$ {\it num\_leaf} training examples, then a leaf
-is built in its place. Furthermore, for a continuous-valued feature
-$j$ the number of candidate thresholds $\sigma$ to choose from is of
-the order of the number of examples present in the training set. Since
-for large-scale data this can result in a large number of candidate
-thresholds, the user can limit this number via the arguments {\it
- bins} which controls the number of candidate thresholds considered
-for each continuous-valued feature. For each continuous-valued
-feature, the implementation computes an equi-height histogram to
-generate one candidate threshold per equi-height bin. To determine the
-best value subset to split on in the case of categorical features,
-this implementation greedily includes values from the feature's domain
-until the sum of the impurities of the subtrees induced stops
-improving.
+\textbf{Input format.}
+In general implementations of the decision tree algorithm do not require categorical features to be dummy coded. For improved efficiency and reducing the training time, our implementation however assumes dummy coded categorical features and dummy coded class labels.
-\par
-Learning a decision tree on large-scale data has received some
+\textbf{Tree construction.}
+Learning a decision tree on large-scale data has received some
attention in the literature. The current implementation includes logic
for choosing tests for multiple nodes that belong to the same level in
the decision tree in parallel (breadth-first expansion) and for
@@ -122,44 +145,67 @@ building entire subtrees under multiple nodes in parallel (depth-first
subtree building). Empirically it has been demonstrated that it is
advantageous to perform breadth-first expansion for the nodes
belonging to the top levels of the tree and to perform depth-first
-subtree building for nodes belonging to the lower levels of the tree
-(Panda et al, 2009). The parameter {\it num\_samples} controls when we
+subtree building for nodes belonging to the lower levels of the tree~\cite{PandaHBB09:dtree}. The parameter {\tt num\_samples} controls when we
switch to depth-first subtree building. Any node in the decision tree
-that receives $\leq$ {\it num\_samples} training examples, the subtree
+that receives $\leq$ {\tt num\_samples} training examples, the subtree
under it is built in its entirety in one shot.
-\par
-{\it Description of the model}: The learnt decision tree is represented using a matrix that
-contains at least 3 rows. Each column in the matrix contains the parameters relevant
-to a single node in the tree. The $i^{th}$ column denotes the parameters for the $i^{th}$ node
-whose left child is stored in the $2i^{th}$ column and right child is stored in the $2i+1^{th}$
-column. Here is a brief description of what each row in the matrix contains:
+\textbf{Stopping rule and pruning.}
+The splitting of data at the internal nodes stops when at least one the following criteria is satisfied:
+\begin{itemize}
+ \item the depth of the internal node reaches the input parameter {\tt depth} controlling the maximum depth of the learned tree, or
+ \item no candidate split achieves information gain.
+\end{itemize}
+This implementation also allows for some automated pruning via the argument {\tt num\_leaf}. If
+a node receives $\leq$ {\tt num\_leaf} training examples, then a leaf
+is built in its place.
+
+
+\textbf{Continuous-valued features.}
+For a continuous-valued feature
+$j$ the number of candidate thresholds $\sigma$ to choose from is of
+the order of the number of examples present in the training set. Since
+for large-scale data this can result in a large number of candidate
+thresholds, the user can limit this number via the arguments {\tt bins} which controls the number of candidate thresholds considered
+for each continuous-valued feature. For each continuous-valued
+feature, the implementation computes an equi-height histogram to
+generate one candidate threshold per equi-height bin.
+
+
+\textbf{Categorical features.}
+In order to determine the best value subset to split on in the case of categorical features, this implementation greedily includes values from the feature's domain until the information gain stops improving.
+In particular, for a categorical feature $j$ the $|Dom(j)|$ feature values are sorted by impurity and the resulting split candidates $|Dom(j)|-1$ are examined; the sequence of feature values which results in the maximum information gain is then selected.
+
+
+\textbf{Description of the model.}
+The learned decision tree is represented in a matrix $M$ that
+contains at least 6 rows. Each column in the matrix contains the parameters relevant to a single node in the tree.
+Note that for building the tree model, our implementation splits the feature matrix $X$ into $X_\text{cont}$ containing continuous-valued features and $X_\text{cat}$ containing categorical features. In the following, the continuous-valued (resp. categorical) feature-ids correspond to the indices of the features in $X_\text{cont}$ (resp. $X_\text{cat}$).
+Moreover, we refer to an internal node as a continuous-valued (categorical) node if the feature that this nodes looks at is continuous-valued (categorical).
+Below is a description of what each row in the matrix contains.
\begin{itemize}
-\item $1^{st}$ row: Contains the feature index of the feature that this node looks at if
-the node is an internal node, otherwise -1.
-\item $2^{nd}$ row: Contains the type of the feature that this node looks at if the node is
-an internal node, otherwise the label this leaf node is supposed to predict.
-1 denotes continuous-valued feature and 2 denotes categorical.
-\item $3^{rd}$: Only applicable for internal nodes. Contains the threshold the example's
-feature value is compared to if the feature chosen for this node is a continuous-valued feature.
-If on the other hand, the feature chosen for this node is categorical then the size of the
-subset of values is stored here.
-\item $4^{th}$ row onwards: Only applicable in the case of internal nodes where the feature
-chosen is a categorical feature. Rows $4, 5 \ldots$ depict the value subset
-chosen for this node.
+\item Row 1: stores the node-ids. These ids correspond to the node-ids in a complete binary tree.
+\item Row 2: for internal nodes stores the offsets (the number of columns) in $M$ to the left child, and otherwise 0.
+\item Row 3: stores the feature index of the feature (id of a continuous-valued feature in $X_\text{cont}$ if the feature is continuous-valued or id of a categorical feature in $X_\text{cat}$ if the feature is categorical) that this node looks at if the node is an internal node, otherwise 0.
+\item Row 4: store the type of the feature that this node looks at if the node is an internal node: 1 for continuous-valued and 2 for categorical features,
+otherwise the label this leaf node is supposed to predict.
+\item Row 5: for the internal nodes contains 1 if the feature chosen for the node is continuous-valued, or the size of the subset of values used for splitting at the node stored in rows 6,7,$\ldots$ if the feature chosen for the node is categorical. For the leaf nodes, Row 5 contains the number of misclassified training examples reaching at this node.
+\item Row 6,7,$\ldots$: for the internal nodes, row 6 stores the threshold to which the example's feature value is compared if the feature chosen for this node is continuous-valued, otherwise if the feature chosen for this node is categorical rows 6,7,$\ldots$ store the value subset chosen for the node.
+For the leaf nodes, row 6 contains 1 if the node is impure and the number of training examples at the node is greater than {\tt num\_leaf}, otherwise 0.
\end{itemize}
-As an example, Figure \ref{dtree} shows a decision tree with $5$ nodes and its matrix
+As an example, Figure~\ref{dtree} shows a decision tree with $5$ nodes and its matrix
representation.
\begin{figure}
\begin{minipage}{0.3\linewidth}
\begin{center}
\begin{tikzpicture}
-\node (labelleft) [draw,shape=circle,minimum size=16pt] at (0,0) {$2$};
-\node (labelright) [draw,shape=circle,minimum size=16pt] at (1,0) {$1$};
-\node (rootleft) [draw,shape=rectangle,minimum size=16pt] at (0.5,1) {$x_5 \in \{2,3\}$};
-\node (rootlabel) [draw,shape=circle,minimum size=16pt] at (2.5,1) {$1$};
+\node (labelleft) [draw,shape=circle,minimum size=16pt] at (2,0) {$2$};
+\node (labelright) [draw,shape=circle,minimum size=16pt] at (3.25,0) {$1$};
+
+\node (rootleft) [draw,shape=rectangle,minimum size=16pt] at (2.5,1) {$x_5 \in \{2,3\}$};
+\node (rootlabel) [draw,shape=circle,minimum size=16pt] at (0.9,1) {$1$};
\node (root) [draw,shape=rectangle,minimum size=16pt] at (1.75,2) {$x_3 < 0.45$};
\draw[-latex] (root) -- (rootleft);
@@ -179,15 +225,19 @@ representation.
\begin{tabular}{c|c|c|c|c|c|}
& Col 1 & Col 2 & Col 3 & Col 4 & Col 5\\
\hline
-Row 1 & 3 & 5 & -1 & -1 & -1\\
+Row 1 & 1 & 2 & 3 & 6 & 7 \\
+\hline
+Row 2 & 1 & 0 & 1 & 0 & 0 \\
\hline
-Row 2 & 1 & 2 & 1 & 2 & 1\\
+Row 3 & 3 & 5 & 0 & 0 & 0 \\
\hline
-Row 3 & 0.45 & 2 & & & \\
+Row 4 & 1 & 1 & 2 & 2 & 1 \\
\hline
-Row 4 & & 2 & & & \\
+Row 5 & 1 & 0 & 2 & 0 & 0 \\
\hline
-Row 5 & & 3 & & & \\
+Row 6 & 0.45 & 0 & 2 & 0 & 0 \\
+\hline
+Row 7 & & & 3 & & \\
\hline
\end{tabular}
\end{center}
@@ -195,47 +245,47 @@ Row 5 & & 3 & & & \\
(b)
\end{center}
\end{minipage}
-\caption{(a) An example tree and its (b) matrix representation. $x$ denotes an example and $x_j$ the $j^{th}$ feature's value in it.}
+\caption{(a) An example tree and its (b) matrix representation. $x$ denotes an example and $x_j$ is the value of the $j$th continuous-valued (resp. categorical) feature in $X_\text{cont}$ (resp. $X_\text{cat}$). In this example all leaf nodes are pure and no training example is misclassified.}
\label{dtree}
\end{figure}
-\vspace{16pt}
+\smallskip
\noindent{\bf Returns}
+\smallskip
+
-The matrix corresponding to the learnt model is written to a file in the format requested. See
-details where the structure of the model matrix is
-described. Depending on what arguments are provided during
-invocation, decision-tree-predict.dml may compute one or more of
-predictions, accuracy and confusion matrix in the requested output format.
-\\
+The matrix corresponding to the learned model as well as the training accuracy (if requested) is written to a file in the format specified. See
+details where the structure of the model matrix is described.
+Recall that in our implementation $X$ is split into $X_\text{cont}$ and $X_\text{cat}$. If requested, the mappings of the continuous-valued feature-ids in $X_\text{cont}$ (stored at {\tt S\_map}) and the categorical feature-ids in $X_\text{cat}$ (stored at {\tt C\_map}) to the global feature-ids in $X$ will be provided.
+Depending on what arguments are provided during
+invocation, the {\tt decision-tree-predict.dml} script may compute one or more of predictions, accuracy and confusion matrix in the requested output format.
+\smallskip
\noindent{\bf Examples}
-\begin{verbatim}
-hadoop jar SystemML.jar -f decision-tree.dml -nvargs
- X=/user/biadmin/X.mtx
- Y=/user/biadmin/y.mtx
- types=/user/biadmin/types.mtx
- model=/user/biadmin/model.mtx
- bins=50 depth=10 num_leaf=1
- num_samples=250 fmt=csv
- Log=/user/biadmin/accuracy.csv
-\end{verbatim}
-
-\begin{verbatim}
-hadoop jar SystemML.jar -f decision-tree-predict.dml -nvargs
- X=/user/biadmin/X.mtx
- Y=/user/biadmin/y.mtx
- model=/user/biadmin/model.mtx
- fmt=csv
- predictions=/user/biadmin/probabilities.csv
- accuracy=/user/biadmin/accuracy.csv
- confusion=/user/biadmin/confusion.csv
-\end{verbatim}
-
-\noindent{\bf References}
+\smallskip
-\begin{itemize}
-\item B. Panda, J. Herbach, S. Basu, and R. Bayardo. \newblock{PLANET: massively parallel learning of tree ensembles with MapReduce}. In Proceedings of the VLDB Endowment, 2009.
-\item L. Breiman, J. Friedman, R. Olshen, and C. Stone. \newblock{Classification and Regression Trees}. Wadsworth and Brooks, 1984.
-\end{itemize}
+{\hangindent=\parindent\noindent\tt
+ \hml -f decision-tree.dml -nvargs X=/user/biadmin/X.mtx Y=/user/biadmin/Y.mtx
+ R=/user/biadmin/R.csv M=/user/biadmin/model.csv
+ bins=20 depth=25 num\_leaf=10 num\_samples=3000 impurity=Gini fmt=csv
+
+}\smallskip
+
+
+\noindent To compute predictions:
+
+{\hangindent=\parindent\noindent\tt
+ \hml -f decision-tree-predict.dml -nvargs X=/user/biadmin/X.mtx Y=/user/biadmin/Y.mtx R=/user/biadmin/R.csv
+ M=/user/biadmin/model.csv P=/user/biadmin/predictions.csv
+ A=/user/biadmin/accuracy.csv CM=/user/biadmin/confusion.csv fmt=csv
+
+}\smallskip
+
+
+%\noindent{\bf References}
+%
+%\begin{itemize}
+%\item B. Panda, J. Herbach, S. Basu, and R. Bayardo. \newblock{PLANET: massively parallel learning of tree ensembles with MapReduce}. In Proceedings of the VLDB Endowment, 2009.
+%\item L. Breiman, J. Friedman, R. Olshen, and C. Stone. \newblock{Classification and Regression Trees}. Wadsworth and Brooks, 1984.
+%\end{itemize}
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/53e814f8/Algorithms Reference/GLM.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/GLM.tex b/Algorithms Reference/GLM.tex
index e4d13bf..92cef0a 100644
--- a/Algorithms Reference/GLM.tex
+++ b/Algorithms Reference/GLM.tex
@@ -1,4 +1,5 @@
\subsection{Generalized Linear Models (GLM)}
+\label{sec:GLM}
\noindent{\bf Description}
\smallskip
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/53e814f8/Algorithms Reference/KaplanMeier.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/KaplanMeier.tex b/Algorithms Reference/KaplanMeier.tex
new file mode 100644
index 0000000..754f6c2
--- /dev/null
+++ b/Algorithms Reference/KaplanMeier.tex
@@ -0,0 +1,268 @@
+\subsection{Kaplan-Meier Survival Analysis}
+\label{sec:kaplan-meier}
+
+\noindent{\bf Description}
+\smallskip
+
+
+Survival analysis examines the time needed for a particular event of interest to occur.
+In medical research, for example, the prototypical such event is the death of a patient but the methodology can be applied to other application areas, e.g., completing a task by an individual in a psychological experiment or the failure of electrical components in engineering.
+Kaplan-Meier or (product limit) method is a simple non-parametric approach for estimating survival probabilities from both censored and uncensored survival times.\\
+
+
+
+\smallskip
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+{\tt{}-f }path/\/{\tt{}KM.dml}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} TE=}path/file
+{\tt{} GI=}path/file
+{\tt{} SI=}path/file
+{\tt{} O=}path/file
+{\tt{} M=}path/file
+{\tt{} T=}path/file
+{\tt{} alpha=}double
+{\tt{} etype=}greenwood$\mid$peto
+{\tt{} ctype=}plain$\mid$log$\mid$log-log
+{\tt{} ttype=}none$\mid$log-rank$\mid$wilcoxon
+{\tt{} fmt=}format
+
+}
+
+\smallskip
+\noindent{\bf Arguments}
+\begin{Description}
+\item[{\tt X}:]
+Location (on HDFS) to read the input matrix of the survival data containing:
+\begin{Itemize}
+ \item timestamps,
+ \item whether event occurred (1) or data is censored (0),
+ \item a number of factors (i.e., categorical features) for grouping and/or stratifying
+\end{Itemize}
+\item[{\tt TE}:]
+Location (on HDFS) to read the 1-column matrix $TE$ that contains the column indices of the input matrix $X$ corresponding to timestamps (first entry) and event information (second entry)
+\item[{\tt GI}:]
+Location (on HDFS) to read the 1-column matrix $GI$ that contains the column indices of the input matrix $X$ corresponding to the factors (i.e., categorical features) to be used for grouping
+\item[{\tt SI}:]
+Location (on HDFS) to read the 1-column matrix $SI$ that contains the column indices of the input matrix $X$ corresponding to the factors (i.e., categorical features) to be used for grouping
+\item[{\tt O}:]
+Location (on HDFS) to write the matrix containing the results of the Kaplan-Meier analysis $KM$
+\item[{\tt M}:]
+Location (on HDFS) to write Matrix $M$ containing the following statistics: total number of events, median and its confidence intervals; if survival data for multiple groups and strata are provided each row of $M$ contains the above statistics per group and stratum.
+\item[{\tt T}:]
+If survival data from multiple groups is available and {\tt ttype=log-rank} or {\tt ttype=wilcoxon}, location (on HDFS) to write the two matrices that contains the result of the (stratified) test for comparing these groups; see below for details.
+\item[{\tt alpha}:](default:\mbox{ }{\tt 0.05})
+Parameter to compute $100(1-\alpha)\%$ confidence intervals for the survivor function and its median
+\item[{\tt etype}:](default:\mbox{ }{\tt "greenwood"})
+Parameter to specify the error type according to "greenwood" or "peto"
+\item[{\tt ctype}:](default:\mbox{ }{\tt "log"})
+Parameter to modify the confidence interval; "plain" keeps the lower and upper bound of the confidence interval unmodified, "log" corresponds to logistic transformation and "log-log" corresponds to the complementary log-log transformation
+\item[{\tt ttype}:](default:\mbox{ }{\tt "none"})
+If survival data for multiple groups is available specifies which test to perform for comparing
+survival data across multiple groups: "none", "log-rank" or "wilcoxon" test
+\item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+see read/write functions in SystemML Language Reference for details.
+\end{Description}
+
+
+\noindent{\bf Details}
+\smallskip
+
+The Kaplan-Meier estimate is a non-parametric maximum likelihood estimate (MLE) of the survival function $S(t)$, i.e., the probability of survival from the time origin to a given future time.
+As an illustration suppose that there are $n$ individuals with observed survival times $t_1,t_2,\ldots t_n$ out of which there are $r\leq n$ distinct death times $t_{(1)}\leq t_{(2)}\leq t_{(r)}$---since some of the observations may be censored, in the sense that the end-point of interest has not been observed for those individuals, and there may be more than one individual with the same survival time.
+Let $S(t_j)$ denote the probability of survival until time $t_j$, $d_j$ be the number of events at time $t_j$, and $n_j$ denote the number of individual at risk (i.e., those who die at time $t_j$ or later).
+Assuming that the events occur independently, in Kaplan-Meier method the probability of surviving from $t_j$ to $t_{j+1}$ is estimated from $S(t_j)$ and given by
+\begin{equation*}
+\hat{S}(t) = \prod_{j=1}^{k} \left( \frac{n_j-d_j}{n_j} \right),
+\end{equation*}
+for $t_k\leq t<t_{k+1}$, $k=1,2,\ldots r$, $\hat{S}(t)=1$ for $t<t_{(1)}$, and $t_{(r+1)}=\infty$.
+Note that the value of $\hat{S}(t)$ is constant between times of event and therefore
+the estimate is a step function with jumps at observed event times.
+If there are no censored data this estimator would simply reduce to the empirical survivor function defined as $\frac{n_j}{n}$. Thus, the Kaplan-Meier estimate can be seen as the generalization of the empirical survivor function that handles censored observations.
+
+The methodology used in our {\tt KM.dml} script closely follows~\cite[Sec.~2]{collett2003:kaplanmeier}.
+For completeness we briefly discuss the equations used in our implementation.
+
+% standard error of the survivor function
+\textbf{Standard error of the survivor function.}
+The standard error of the estimated survivor function (controlled by parameter {\tt etype}) can be calculated as
+\begin{equation*}
+\text{se} \{\hat{S}(t)\} \approx \hat{S}(t) {\bigg\{ \sum_{j=1}^{k} \frac{d_j}{n_j(n_j - d_j)}\biggr\}}^2,
+\end{equation*}
+for $t_{(k)}\leq t<t_{(k+1)}$.
+This equation is known as the {\it Greenwood's} formula.
+An alternative approach is to apply the {\it Petos's} expression %~\cite{PetoPABCHMMPS1979:kaplanmeier}
+\begin{equation*}
+\text{se}\{\hat{S}(t)\}=\frac{\hat{S}(t)\sqrt{1-\hat{S}(t)}}{\sqrt{n_k}},
+\end{equation*}
+for $t_{(k)}\leq t<t_{(k+1)}$.
+%Note that this estimate is known to be conservative producing larger standard errors than they ought to be. The Greenwood estimate is therefore recommended for general use.
+Once the standard error of $\hat{S}$ has been found we compute the following types of confidence intervals (controlled by parameter {\tt cctype}):
+The ``plain'' $100(1-\alpha)\%$ confidence interval for $S(t)$ is computed using
+\begin{equation*}
+\hat{S}(t)\pm z_{\alpha/2} \text{se}\{\hat{S}(t)\},
+\end{equation*}
+where $z_{\alpha/2}$ is the upper $\alpha/2$-point of the standard normal distribution.
+Alternatively, we can apply the ``log'' transformation using
+\begin{equation*}
+\hat{S}(t)^{\exp[\pm z_{\alpha/2} \text{se}\{\hat{S}(t)\}/\hat{S}(t)]}
+\end{equation*}
+or the ``log-log'' transformation using
+\begin{equation*}
+\hat{S}(t)^{\exp [\pm z_{\alpha/2} \text{se} \{\log [-\log \hat{S}(t)]\}]}.
+\end{equation*}
+
+% standard error of the median of survival times
+\textbf{Median, its standard error and confidence interval.}
+Denote by $\hat{t}(50)$ the estimated median of $\hat{S}$, i.e.,
+$\hat{t}(50)=\min \{ t_i \mid \hat{S}(t_i) < 0.5\}$,
+where $t_i$ is the observed survival time for individual $i$.
+The standard error of $\hat{t}(50)$ is given by
+\begin{equation*}
+\text{se}\{ \hat{t}(50) \} = \frac{1}{\hat{f}\{\hat{t}(50)\}} \text{se}[\hat{S}\{ \hat{t}(50) \}],
+\end{equation*}
+where $\hat{f}\{ \hat{t}(50) \}$ can be found from
+\begin{equation*}
+\hat{f}\{ \hat{t}(50) \} = \frac{\hat{S}\{ \hat{u}(50) \} -\hat{S}\{ \hat{l}(50) \} }{\hat{l}(50) - \hat{u}(50)}.
+\end{equation*}
+Above, $\hat{u}(50)$ is the largest survival time for which $\hat{S}$ exceeds $0.5+\epsilon$, i.e., $\hat{u}(50)=\max \bigl\{ t_{(j)} \mid \hat{S}(t_{(j)}) \geq 0.5+\epsilon \bigr\}$,
+and $\hat{l}(50)$ is the smallest survivor time for which $\hat{S}$ is less than $0.5-\epsilon$,
+i.e., $\hat{l}(50)=\min \bigl\{ t_{(j)} \mid \hat{S}(t_{(j)}) \leq 0.5+\epsilon \bigr\}$,
+for small $\epsilon$.
+
+
+% comparing two or more groups of data
+\textbf{Log-rank test and Wilcoxon test.}
+Our implementation supports comparison of survival data from several groups using two non-parametric procedures (controlled by parameter {\tt ttype}): the {\it log-rank test} and the {\it Wilcoxon test} (also known as the {\it Breslow test}).
+Assume that the survival times in $g\geq 2$ groups of survival data are to be compared.
+Consider the {\it null hypothesis} that there is no difference in the survival times of the individuals in different groups. One way to examine the null hypothesis is to consider the difference between the observed number of deaths with the numbers expected under the null hypothesis.
+In both tests we define the $U$-statistics ($U_{L}$ for the log-rank test and $U_{W}$ for the Wilcoxon test) to compare the observed and the expected number of deaths in $1,2,\ldots,g-1$ groups as follows:
+\begin{align*}
+U_{Lk} &= \sum_{j=1}^{r}\left( d_{kj} - \frac{n_{kj}d_j}{n_j} \right), \\
+U_{Wk} &= \sum_{j=1}^{r}n_j\left( d_{kj} - \frac{n_{kj}d_j}{n_j} \right),
+\end{align*}
+where $d_{kj}$ is the of number deaths at time $t_{(j)}$ in group $k$,
+$n_{kj}$ is the number of individuals at risk at time $t_{(j)}$ in group $k$, and
+$k=1,2,\ldots,g-1$ to form the vectors $U_L$ and $U_W$ with $(g-1)$ components.
+The covariance (variance) between $U_{Lk}$ and $U_{Lk'}$ (when $k=k'$) is computed as
+\begin{equation*}
+V_{Lkk'}=\sum_{j=1}^{r} \frac{n_{kj}d_j(n_j-d_j)}{n_j(n_j-1)} \left( \delta_{kk'}-\frac{n_{k'j}}{n_j} \right),
+\end{equation*}
+for $k,k'=1,2,\ldots,g-1$, with
+\begin{equation*}
+\delta_{kk'} =
+\begin{cases}
+1 & \text{if } k=k'\\
+0 & \text{otherwise.}
+\end{cases}
+\end{equation*}
+These terms are combined in a {\it variance-covariance} matrix $V_L$ (referred to as the $V$-statistic).
+Similarly, the variance-covariance matrix for the Wilcoxon test $V_W$ is a matrix where the entry at position $(k,k')$ is given by
+\begin{equation*}
+V_{Wkk'}=\sum_{j=1}^{r} n_j^2 \frac{n_{kj}d_j(n_j-d_j)}{n_j(n_j-1)} \left( \delta_{kk'}-\frac{n_{k'j}}{n_j} \right).
+\end{equation*}
+
+Under the null hypothesis of no group differences, the test statistics $U_L^\top V_L^{-1} U_L$ for the log-rank test and $U_W^\top V_W^{-1} U_W$ for the Wilcoxon test have a Chi-squared distribution on $(g-1)$ degrees of freedom.
+Our {\tt KM.dml} script also provides a stratified version of the log-rank or Wilcoxon test if requested.
+In this case, the values of the $U$- and $V$- statistics are computed for each stratum and then combined over all strata.
+
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+
+Blow we list the results of the survival analysis computed by {\tt KM.dml}.
+The calculated statistics are stored in matrix $KM$ with the following schema:
+\begin{itemize}
+ \item Column 1: timestamps
+ \item Column 2: number of individuals at risk
+ \item Column 3: number of events
+ \item Column 4: Kaplan-Meier estimate of the survivor function $\hat{S}$
+ \item Column 5: standard error of $\hat{S}$
+ \item Column 6: lower bound of $100(1-\alpha)\%$ confidence interval for $\hat{S}$
+ \item Column 7: upper bound of $100(1-\alpha)\%$ confidence interval for $\hat{S}$
+\end{itemize}
+Note that if survival data for multiple groups and/or strata is available, each collection of 7 columns in $KM$ stores the results per group and/or per stratum.
+In this case $KM$ has $7g+7s$ columns, where $g\geq 1$ and $s\geq 1$ denote the number of groups and strata, respectively.
+
+
+Additionally, {\tt KM.dml} stores the following statistics in the 1-row matrix $M$ whose number of columns depends on the number of groups ($g$) and strata ($s$) in the data. Below $k$ denotes the number of factors used for grouping and $l$ denotes the number of factors used for stratifying.
+\begin{itemize}
+ \item Columns 1 to $k$: unique combination of values in the $k$ factors used for grouping
+ \item Columns $k+1$ to $k+l$: unique combination of values in the $l$ factors used for stratifying
+ \item Column $k+l+1$: total number of records
+ \item Column $k+l+2$: total number of events
+ \item Column $k+l+3$: median of $\hat{S}$
+ \item Column $k+l+4$: lower bound of $100(1-\alpha)\%$ confidence interval for the median of $\hat{S}$
+ \item Column $k+l+5$: upper bound of $100(1-\alpha)\%$ confidence interval for the median of $\hat{S}$.
+\end{itemize}
+If there is only 1 group and 1 stratum available $M$ will be a 1-row matrix with 5 columns where
+\begin{itemize}
+ \item Column 1: total number of records
+ \item Column 2: total number of events
+ \item Column 3: median of $\hat{S}$
+ \item Column 4: lower bound of $100(1-\alpha)\%$ confidence interval for the median of $\hat{S}$
+ \item Column 5: upper bound of $100(1-\alpha)\%$ confidence interval for the median of $\hat{S}$.
+\end{itemize}
+
+If a comparison of the survival data across multiple groups needs to be performed, {\tt KM.dml} computes two matrices $T$ and $T\_GROUPS\_OE$ that contain a summary of the test. The 1-row matrix $T$ stores the following statistics:
+\begin{itemize}
+ \item Column 1: number of groups in the survival data
+ \item Column 2: degree of freedom for Chi-squared distributed test statistic
+ \item Column 3: value of test statistic
+ \item Column 4: $P$-value.
+\end{itemize}
+Matrix $T\_GROUPS\_OE$ contains the following statistics for each of $g$ groups:
+\begin{itemize}
+ \item Column 1: number of events
+ \item Column 2: number of observed death times ($O$)
+ \item Column 3: number of expected death times ($E$)
+ \item Column 4: $(O-E)^2/E$
+ \item Column 5: $(O-E)^2/V$.
+\end{itemize}
+
+
+\smallskip
+\noindent{\bf Examples}
+\smallskip
+
+{\hangindent=\parindent\noindent\tt
+ \hml -f KM.dml -nvargs X=/user/biadmin/X.mtx TE=/user/biadmin/TE
+ GI=/user/biadmin/GI SI=/user/biadmin/SI O=/user/biadmin/kaplan-meier.csv
+ M=/user/biadmin/model.csv alpha=0.01 etype=greenwood ctype=plain fmt=csv
+
+}\smallskip
+
+{\hangindent=\parindent\noindent\tt
+ \hml -f KM.dml -nvargs X=/user/biadmin/X.mtx TE=/user/biadmin/TE
+ GI=/user/biadmin/GI SI=/user/biadmin/SI O=/user/biadmin/kaplan-meier.csv
+ M=/user/biadmin/model.csv T=/user/biadmin/test.csv alpha=0.01 etype=peto
+ ctype=log ttype=log-rank fmt=csv
+
+}
+
+%
+%\smallskip
+%\noindent{\bf References}
+%\begin{itemize}
+% \item
+% R.~Peto, M.C.~Pike, P.~Armitage, N.E.~Breslow, D.R.~Cox, S.V.~Howard, N.~Mantel, K.~McPherson, J.~Peto, and P.G.~Smith.
+% \newblock Design and analysis of randomized clinical trials requiring prolonged observation of each patient.
+% \newblock {\em British Journal of Cancer}, 35:1--39, 1979.
+%\end{itemize}
+
+%@book{collett2003:kaplanmeier,
+% title={Modelling Survival Data in Medical Research, Second Edition},
+% author={Collett, D.},
+% isbn={9781584883258},
+% lccn={2003040945},
+% series={Chapman \& Hall/CRC Texts in Statistical Science},
+% year={2003},
+% publisher={Taylor \& Francis}
+%}
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/53e814f8/Algorithms Reference/LinReg.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/LinReg.tex b/Algorithms Reference/LinReg.tex
index ad0dab8..383971b 100644
--- a/Algorithms Reference/LinReg.tex
+++ b/Algorithms Reference/LinReg.tex
@@ -1,4 +1,5 @@
\subsection{Linear Regression}
+\label{sec:LinReg}
\noindent{\bf Description}
\smallskip
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/53e814f8/Algorithms Reference/RandomForest.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/RandomForest.tex b/Algorithms Reference/RandomForest.tex
new file mode 100644
index 0000000..5539391
--- /dev/null
+++ b/Algorithms Reference/RandomForest.tex
@@ -0,0 +1,194 @@
+\subsection{Random Forests}
+\label{random_forests}
+
+\noindent{\bf Description}
+\smallskip
+
+
+Random forest is one of the most successful machine learning methods for classification and regression.
+It is an ensemble learning method that creates a model composed of a set of tree models.
+This implementation is well-suited to handle large-scale data and builds a random forest model for classification in parallel.\\
+
+
+\smallskip
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+ {\tt{}-f }path/\/{\tt{}random-forest.dml}
+ {\tt{} -nvargs}
+ {\tt{} X=}path/file
+ {\tt{} Y=}path/file
+ {\tt{} R=}path/file
+ {\tt{} bins=}integer
+ {\tt{} depth=}integer
+ {\tt{} num\_leaf=}integer
+ {\tt{} num\_samples=}integer
+ {\tt{} num\_trees=}integer
+ {\tt{} subsamp\_rate=}double
+ {\tt{} feature\_subset=}double
+ {\tt{} impurity=}Gini$\mid$entropy
+ {\tt{} M=}path/file
+ {\tt{} C=}path/file
+ {\tt{} S\_map=}path/file
+ {\tt{} C\_map=}path/file
+ {\tt{} fmt=}format
+
+}
+
+ \smallskip
+ \noindent{\bf Usage: Prediction}
+ \smallskip
+
+ {\hangindent=\parindent\noindent\it%
+ {\tt{}-f }path/\/{\tt{}random-forest-predict.dml}
+ {\tt{} -nvargs}
+ {\tt{} X=}path/file
+ {\tt{} Y=}path/file
+ {\tt{} R=}path/file
+ {\tt{} M=}path/file
+ {\tt{} C=}path/file
+ {\tt{} P=}path/file
+ {\tt{} A=}path/file
+ {\tt{} OOB=}path/file
+ {\tt{} CM=}path/file
+ {\tt{} fmt=}format
+
+ }\smallskip
+
+
+\noindent{\bf Arguments}
+\begin{Description}
+ \item[{\tt X}:]
+ Location (on HDFS) to read the matrix of feature vectors;
+ each row constitutes one feature vector. Note that categorical features in $X$ need to be both recoded and dummy coded.
+ \item[{\tt Y}:]
+ Location (on HDFS) to read the matrix of (categorical)
+ labels that correspond to feature vectors in $X$. Note that classes are assumed to be both recoded and dummy coded.
+ This argument is optional for prediction.
+ \item[{\tt R}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to read matrix $R$ which for each feature in $X$ contains column-ids (first column), start indices (second column), and end indices (third column).
+ If $R$ is not provided by default all features are assumed to be continuous-valued.
+ \item[{\tt bins}:] (default:\mbox{ }{\tt 20})
+ Number of thresholds to choose for each continuous-valued feature (determined by equi-height binning).
+ \item[{\tt depth}:] (default:\mbox{ }{\tt 25})
+ Maximum depth of the learned trees in the random forest model
+ \item[{\tt num\_leaf}:] (default:\mbox{ }{\tt 10})
+ Parameter that controls pruning. The tree
+ is not expanded if a node receives less than {\tt num\_leaf} training examples.
+ \item[{\tt num\_samples}:] (default:\mbox{ }{\tt 3000})
+ Parameter that decides when to switch to in-memory building of the subtrees in each tree of the random forest model.
+ If a node $v$ receives less than {\tt num\_samples}
+ training examples then this implementation switches to an in-memory subtree
+ building procedure to build the subtree under $v$ in its entirety.
+ \item[{\tt num\_trees}:] (default:\mbox{ }{\tt 10})
+ Number of trees to be learned in the random forest model
+ \item[{\tt subsamp\_rate}:] (default:\mbox{ }{\tt 1.0})
+ Parameter controlling the size of each tree in the random forest model; samples are selected from a Poisson distribution with parameter {\tt subsamp\_rate}.
+ \item[{\tt feature\_subset}:] (default:\mbox{ }{\tt 0.5})
+ Parameter that controls the number of feature used as candidates for splitting at each tree node as a power of the number of features in the data, i.e., assuming the training set has $D$ features $D^{\tt feature\_subset}$ are used at each tree node.
+ \item[{\tt impurity}:] (default:\mbox{ }{\tt "Gini"})
+ Impurity measure used at internal nodes of the trees in the random forest model for selecting which features to split on. Possible value are entropy or Gini.
+ \item[{\tt M}:]
+ Location (on HDFS) to write matrix $M$ containing the learned random forest (see Section~\ref{sec:decision_trees} and below for the schema)
+ \item[{\tt C}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to store the number of counts (generated according to a Poisson distribution with parameter {\tt subsamp\_rate}) for each feature vector. Note that this argument is optional. If Out-Of-Bag (OOB) error estimate needs to be computed this parameter is passed as input to {\tt random-forest-predict.dml}.
+ \item[{\tt A}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to store the testing accuracy (\%) from a
+ held-out test set during prediction. Note that this argument is optional.
+ \item[{\tt OOB}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to store the Out-Of-Bag (OOB) error estimate of the training set. Note that the matrix of sample counts (stored at {\tt C}) needs to be provided for computing OOB error estimate. Note that this argument is optional.
+ \item[{\tt P}:]
+ Location (on HDFS) to store predictions for a held-out test set
+ \item[{\tt CM}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to store the confusion matrix computed using a held-out test set. Note that this argument is optional.
+ \item[{\tt S\_map}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to write the mappings from the continuous-valued feature-ids to the global feature-ids in $X$ (see below for details). Note that this argument is optional.
+ \item[{\tt C\_map}:] (default:\mbox{ }{\tt " "})
+ Location (on HDFS) to write the mappings from the categorical feature-ids to the global feature-ids in $X$ (see below for details). Note that this argument is optional.
+ \item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+ Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+ see read/write functions in SystemML Language Reference for details.
+\end{Description}
+
+
+ \noindent{\bf Details}
+ \smallskip
+
+Random forests~\cite{Breiman01:rforest} are learning algorithms for ensembles of decision trees.
+The main idea is to build a number of decision trees on bootstrapped training samples, i.e., by taking repeatedly samples from a (single) training set.
+Moreover, instead of considering all the features when building the trees only a random subset of the features---typically $\approx \sqrt{D}$, where $D$ is the number of features---is chosen each time a split test at a tree node is performed.
+This procedure {\it decorrelates} the trees and makes it less prone to overfitting.
+To build decision trees we utilize the techniques discussed in Section~\ref{sec:decision_trees} proposed in~\cite{PandaHBB09:dtree};
+the implementation details are similar to those of the decision trees script.
+Below we review some features of our implementation which differ from {\tt decision-tree.dml}.
+
+
+\textbf{Bootstrapped sampling.}
+Each decision tree is fitted to a bootstrapped training set sampled with replacement (WR).
+To improve efficiency, we generate $N$ sample counts according to a Poisson distribution with parameter {\tt subsamp\_rate},
+where $N$ denotes the total number of training points.
+These sample counts approximate WR sampling when $N$ is large enough and are generated upfront for each decision tree.
+
+
+\textbf{Bagging.}
+Decision trees suffer from {\it high variance} resulting in different models whenever trained on a random subsets of the data points.
+{\it Bagging} is a general-purpose method to reduce the variance of a statistical learning method like decision trees.
+In the context of decision trees (for classification), for a given test feature vector
+the prediction is computed by taking a {\it majority vote}: the overall prediction is the most commonly occurring class among all the tree predictions.
+
+
+\textbf{Out-Of-Bag error estimation.}
+Note that each bagged tree in a random forest model is trained on a subset (around $\frac{2}{3}$) of the observations (i.e., feature vectors).
+The remaining ($\frac{1}{3}$ of the) observations not used for training is called the {\it Out-Of-Bag} (OOB) observations.
+This gives us a straightforward way to estimate the test error: to predict the class label of each test observation $i$ we use the trees in which $i$ was OOB.
+Our {\tt random-forest-predict.dml} script provides the OOB error estimate for a given training set if requested.
+
+
+\textbf{Description of the model.}
+Similar to decision trees, the learned random forest model is presented in a matrix $M$ with at least 7 rows.
+The information stored in the model is similar to that of decision trees with the difference that the tree-ids are stored
+in the second row and rows $2,3,\ldots$ from the decision tree model are shifted by one. See Section~\ref{sec:decision_trees} for a description of the model.
+
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+
+The matrix corresponding to the learned model is written to a file in the format specified. See Section~\ref{sec:decision_trees} where the details about the structure of the model matrix is described.
+Similar to {\tt decision-tree.dml}, $X$ is split into $X_\text{cont}$ and $X_\text{cat}$.
+If requested, the mappings of the continuous feature-ids in $X_\text{cont}$ (stored at {\tt S\_map}) as well as the categorical feature-ids in $X_\text{cat}$ (stored at {\tt C\_map}) to the global feature-ids in $X$ will be provided.
+The {\tt random-forest-predict.dml} script may compute one or more of
+predictions, accuracy, confusion matrix, and OOB error estimate in the requested output format depending on the input arguments used.
+
+
+
+\smallskip
+\noindent{\bf Examples}
+\smallskip
+
+{\hangindent=\parindent\noindent\tt
+ \hml -f random-forest.dml -nvargs X=/user/biadmin/X.mtx Y=/user/biadmin/Y.mtx
+ R=/user/biadmin/R.csv M=/user/biadmin/model.csv
+ bins=20 depth=25 num\_leaf=10 num\_samples=3000 num\_trees=10 impurity=Gini fmt=csv
+
+}\smallskip
+
+
+\noindent To compute predictions:
+
+{\hangindent=\parindent\noindent\tt
+ \hml -f random-forest-predict.dml -nvargs X=/user/biadmin/X.mtx Y=/user/biadmin/Y.mtx R=/user/biadmin/R.csv
+ M=/user/biadmin/model.csv P=/user/biadmin/predictions.csv
+ A=/user/biadmin/accuracy.csv CM=/user/biadmin/confusion.csv fmt=csv
+
+}\smallskip
+
+
+%\noindent{\bf References}
+%
+%\begin{itemize}
+%\item B. Panda, J. Herbach, S. Basu, and R. Bayardo. \newblock{PLANET: massively parallel learning of tree ensembles with MapReduce}. In Proceedings of the VLDB Endowment, 2009.
+%\item L. Breiman. \newblock{Random Forests}. Machine Learning, 45(1), 5--32, 2001.
+%\end{itemize}
[21/47] incubator-systemml git commit: [SYSML-53] - Add SystemML
Quick Start Guide
Posted by du...@apache.org.
[SYSML-53] - Add SystemML Quick Start Guide
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/f4b41e0d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/f4b41e0d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/f4b41e0d
Branch: refs/heads/gh-pages
Commit: f4b41e0db8e35debcddf5913c5cfca5e4f4e5ed9
Parents: 66eb331
Author: Christian Kadner <ck...@us.ibm.com>
Authored: Tue Sep 8 17:47:28 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Tue Sep 8 17:47:28 2015 -0700
----------------------------------------------------------------------
index.md | 3 +-
quick-start-guide.md | 398 ++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 399 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/f4b41e0d/index.md
----------------------------------------------------------------------
diff --git a/index.md b/index.md
index 4dbaf86..49f69d8 100644
--- a/index.md
+++ b/index.md
@@ -16,8 +16,7 @@ and (3) automatic optimization.
For more information about SystemML, please consult the following references:
* [SystemML GitHub README](http://www.github.com/SparkTC/systemml)
-* Quick Start - **Coming Soon**
+* [Quick Start Guide](quick-start-guide.html)
* [Algorithms Reference](algorithms-reference.html)
* [DML (Declarative Machine Learning) Language Reference](dml-language-reference.html)
* PYDML (Python-Like Declarative Machine Learning) Language Reference - **Coming Soon**
-
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/f4b41e0d/quick-start-guide.md
----------------------------------------------------------------------
diff --git a/quick-start-guide.md b/quick-start-guide.md
new file mode 100644
index 0000000..c8d5f15
--- /dev/null
+++ b/quick-start-guide.md
@@ -0,0 +1,398 @@
+---
+layout: global
+title: SystemML Quick Start Guide
+author: Christian Kadner
+description: SystemML Quick Start Guide
+displayTitle: SystemML Quick Start Guide
+---
+
+* This will become a table of contents (this text will be scraped).
+{:toc}
+
+<br/>
+
+This tutorial provides a quick introduction to using SystemML by
+running existing SystemML algorithms in standalone mode. More information
+about running SystemML in distributed execution mode (Hadoop, Spark) will
+be added soon.
+For more in depth information, please refer to the
+[Algorithms Reference](algorithms-reference.html) and the
+[SystemML Language Reference](dml-language-reference.html).
+
+
+<br/>
+
+# What is SystemML
+
+SystemML enables large-scale machine learning (ML) via a high-level declarative
+language with R-like syntax called [DML](dml-language-reference.html).
+This language allows Data Scientists to
+express their ML algorithms with full flexibility but without the need to fine-tune
+distributed runtime execution plans and system configurations.
+These ML programs are dynamically compiled and optimized based on data
+and cluster characteristics using rule and cost-based optimization techniques.
+The compiler automatically generates hybrid runtime execution plans ranging
+from in-memory, single node execution to distributed computation for Hadoop M/R
+or Spark Batch execution.
+SystemML features a suite of algorithms for Descriptive Statistics, Classification,
+Clustering, Regression, and Matrix Factorization. Detailed descriptions of these
+algorithms can be found in the [Algorithms Reference](algorithms-reference.html).
+
+<br/>
+
+# Distributed vs Standalone Execution Mode
+
+SystemML can operate in standalone mode, allowing data
+scientists to quikly develop algorithms on a single machine.
+For large-scale production environments, SystemML algorithm execution can be
+distributed across a multi-node cluster using [Apache Hadoop](https://hadoop.apache.org/)
+or [Apache Spark](http://spark.apache.org/).
+We will make use of standalone mode throughout this tutorial.
+
+The examples described in `docs/SystemML_Algorithms_Reference.pdf` are written
+primarily for the hadoop distributed environment. To run those examples in
+standalone mode, modify the commands by replacing "`hadoop jar SystemML.jar -f ...`"
+with "`./runStandaloneSystemML.sh ...`" on Mac/UNIX or
+with "`./runStandaloneSystemML.bat ...`" on Windows.
+
+<br/>
+
+# Contents of the SystemML Standalone Package
+
+To follow along with this guide, first build a standalone package of SystemML
+{% comment %} TODO: Where to download a packaged release of SystemML, ...standalone jar
+
+ $ wget http://spark.tc/system-ml/system-ml-LATEST-standalone.tar.gz -O - | tar -xz -C ~/systemml-tutorial
+
+
+or build it from source {% endcomment %} using [Apache Maven](http://maven.apache.org)
+and unpack it to your working directory, i.e. ```~/systemml-tutorial```.
+
+ $ git clone https://github.com/SparkTC/systemml.git
+ $ mvn clean package
+ $ tar -xzf `find . -name 'system-ml*standalone.tar.gz'` -C ~/systemml-tutorial
+
+The extracted package should have these contents:
+
+ $ ls -lF ~/systemml-tutorial
+ total 56
+ drwxr-xr-x 23 algorithms/
+ drwxr-xr-x 5 docs/
+ drwxr-xr-x 35 lib/
+ -rw-r--r-- 1 log4j.properties
+ -rw-r--r-- 1 readme.txt
+ -rwxr-xr-x 1 runStandaloneSystemML.bat*
+ -rwxr-xr-x 1 runStandaloneSystemML.sh*
+ -rw-r--r-- 1 SystemML-config.xml
+
+Refer to `docs/SystemML_Algorithms_Reference.pdf` for more information
+about each algorithm included in the `algorithms` folder.
+
+For the rest of the tutorial we switch our working directory to ```~/systemml-tutorial```.
+
+ $ cd ~/systemml-tutorial
+
+<br/>
+
+# Choosing Test Data
+
+In this tutorial we will use the [Haberman's Survival Data Set](http://archive.ics.uci.edu/ml/datasets/Haberman%27s+Survival)
+which can be downloaded in CSV format from the [Center for Machine Learning and Intelligent Systems](http://cml.ics.uci.edu/)
+
+ $ wget -P data/ http://archive.ics.uci.edu/ml/machine-learning-databases/haberman/haberman.data
+
+The [Haberman Data Set](http://archive.ics.uci.edu/ml/machine-learning-databases/haberman/haberman.names)
+has 306 instances and 4 attributes (including the class attribute):
+
+ 1. Age of patient at time of operation (numerical)
+ 2. Patient's year of operation (year - 1900, numerical)
+ 3. Number of positive axillary nodes detected (numerical)
+ 4. Survival status (class attribute)
+ * `1` = the patient survived 5 years or longer
+ * `2` = the patient died within 5 year
+
+
+We will need to create a metadata file (MTD) which stores metadata information
+about the content of the data file. The name of the MTD file associated with the
+data file `<filename>` must be `<filename>.mtd`.
+
+ $ echo '{"rows": 306, "cols": 4, "format": "csv"}' > data/haberman.data.mtd
+
+<br/>
+
+# Example 1 - Univariate Statistics
+
+Let's start with a simple example, computing certain [univariate statistics](systemml-algorithms-descriptive-statistics.html#univariate-statistics)
+for each feature column using the algorithm `Univar-Stats.dml` which requires 3
+[arguments](systemml-algorithms-descriptive-statistics.html#arguments):
+
+* `X`: location of the input data file to analyze
+* `TYPES`: location of the file that contains the feature column types encoded by integer numbers: `1` = scale, `2` = nominal, `3` = ordinal
+* `STATS`: location of the output matrix of computed statistics will be stored
+
+We need to create a file `types.csv` that describes the type of each column in
+the data along with it's metadata file `types.csv.mtd`.
+
+ $ echo '1,1,1,2' > data/types.csv
+ $ echo '{"rows": 1, "cols": 4, "format": "csv"}' > data/types.csv.mtd
+
+
+To run the `Univar-Stats.dml` algorithm, issue the following command:
+
+ $ ./runStandaloneSystemML.sh algorithms/Univar-Stats.dml -nvargs X=data/haberman.data TYPES=data/types.csv STATS=data/univarOut.mtx
+
+The resulting matrix has one row per each univariate statistic and one column
+per input feature. The output file `univarOut.mtx` describes that
+matrix. The elements of the first column denote the number of the statistic,
+the elements of the second column refer to the number of the feature column in
+the input data, and the elements of the third column show the value of the
+univariate statistic.
+
+ 1 1 30.0
+ 1 2 58.0
+ 2 1 83.0
+ 2 2 69.0
+ 2 3 52.0
+ 3 1 53.0
+ 3 2 11.0
+ 3 3 52.0
+ 4 1 52.45751633986928
+ 4 2 62.85294117647059
+ 4 3 4.026143790849673
+ 5 1 116.71458266366658
+ 5 2 10.558630665380907
+ 5 3 51.691117539912135
+ 6 1 10.803452349303281
+ 6 2 3.2494046632238507
+ 6 3 7.189653506248555
+ 7 1 0.6175922641866753
+ 7 2 0.18575610076612029
+ 7 3 0.41100513466216837
+ 8 1 0.20594669940735139
+ 8 2 0.051698529971741194
+ 8 3 1.7857418611299172
+ 9 1 0.1450718616532357
+ 9 2 0.07798443581479181
+ 9 3 2.954633471088322
+ 10 1 -0.6150152487211726
+ 10 2 -1.1324380182967442
+ 10 3 11.425776549251449
+ 11 1 0.13934809593495995
+ 11 2 0.13934809593495995
+ 11 3 0.13934809593495995
+ 12 1 0.277810485320835
+ 12 2 0.277810485320835
+ 12 3 0.277810485320835
+ 13 1 52.0
+ 13 2 63.0
+ 13 3 1.0
+ 14 1 52.16013071895425
+ 14 2 62.80392156862745
+ 14 3 1.2483660130718954
+ 15 4 2.0
+ 16 4 1.0
+ 17 4 1.0
+
+The following table lists the number and name of each univariate statistic. The row
+numbers below correspond to the elements of the first column in the output
+matrix above. The signs “+” show applicability to scale or/and to categorical
+features.
+
+ | Row | Name of Statistic | Scale | Categ. |
+ | :-: |:-------------------------- |:-----:| :-----:|
+ | 1 | Minimum | + | |
+ | 2 | Maximum | + | |
+ | 3 | Range | + | |
+ | 4 | Mean | + | |
+ | 5 | Variance | + | |
+ | 6 | Standard deviation | + | |
+ | 7 | Standard error of mean | + | |
+ | 8 | Coefficient of variation | + | |
+ | 9 | Skewness | + | |
+ | 10 | Kurtosis | + | |
+ | 11 | Standard error of skewness | + | |
+ | 12 | Standard error of kurtosis | + | |
+ | 13 | Median | + | |
+ | 14 | Inter quartile mean | + | |
+ | 15 | Number of categories | | + |
+ | 16 | Mode | | + |
+ | 17 | Number of modes | | + |
+
+
+<br/>
+<br/>
+
+# Example 2 - Binary-class Support Vector Machines
+
+Let's take the same `haberman.data` to explore the
+[binary-class support vector machines](systemml-algorithms-classification.html#binary-class-support-vector-machines) algorithm `l2-svm.dml`.
+This example also illustrates how to use of the sampling algorithm `sample.dml`
+and the data split algorithm `spliXY.dml`.
+
+## Sampling the Test Data
+
+First we need to use the `sample.dml` algorithm to separate the input into one
+training data set and one data set for model prediction.
+
+Parameters:
+
+ * `X` : (input) input data set: filename of input data set
+ * `sv` : (input) sampling vector: filename of 1-column vector w/ percentages. sum(sv) must be 1.
+ * `O` : (output) folder name w/ samples generated
+ * `ofmt` : (output) format of O: "csv", "binary" (default)
+
+
+We will create the file `perc.csv` and `perc.csv.mtd` to define the sampling vector with a sampling rate of
+50% to generate 2 data sets:
+
+ $ printf "0.5\n0.5" > data/perc.csv
+ $ echo '{"rows": 2, "cols": 1, "format": "csv"}' > data/perc.csv.mtd
+
+Let's run the sampling algortihm to create the two data samples:
+
+ $ ./runStandaloneSystemML.sh algorithms/utils/sample.dml -nvargs X=data/haberman.data sv=data/perc.csv O=data/haberman.part ofmt="csv"
+
+
+
+## Splitting Labels from Features
+
+Next we use the `splitXY.dml` algortithm to separate the feature columns from
+the label column(s).
+
+Parameters:
+
+ * `X` : (input) filename of data matrix
+ * `y` : (input) colIndex: starting index is 1
+ * `OX` : (output) filename of output matrix with all columns except y
+ * `OY` : (output) filename of output matrix with y column
+ * `ofmt` : (output) format of OX and OY output matrix: "csv", "binary" (default)
+
+We specify `y=4` as the 4th column contains the labels to be predicted and run
+the `splitXY.dml` algorithm on our training and test data sets.
+
+ $ ./runStandaloneSystemML.sh algorithms/utils/splitXY.dml -nvargs X=data/haberman.part/1 y=4 OX=data/haberman.train.data.csv OY=data/haberman.train.labels.csv ofmt="csv"
+
+ $ ./runStandaloneSystemML.sh algorithms/utils/splitXY.dml -nvargs X=data/haberman.part/2 y=4 OX=data/haberman.test.data.csv OY=data/haberman.test.labels.csv ofmt="csv"
+
+## Training and Testing the Model
+
+Now we need to train our model using the `l2-svm.dml` algorithm.
+
+[Parameters](systemml-algorithms-classification.html#arguments-1):
+
+ * `X` : (input) filename of training data features
+ * `Y` : (input) filename of training data labels
+ * `model` : (output) filename of model that contains the learnt weights
+ * `fmt` : (output) format of model: "csv", "text" (sparse-matrix)
+ * `Log` : (output) log file for metrics and progress while training
+ * `confusion` : (output) filename of confusion matrix computed using a held-out test set (optional)
+
+The `l2-svm.dml` algorithm is used on our training data sample to train the model.
+
+ $ ./runStandaloneSystemML.sh algorithms/l2-svm.dml -nvargs X=data/haberman.train.data.csv Y=data/haberman.train.labels.csv model=data/l2-svm-model.csv fmt="csv" Log=data/l2-svm-log.csv
+
+The `l2-svm-predict.dml` algorithm is used on our test data sample to predict the labels based on the trained model.
+
+ $ ./runStandaloneSystemML.sh algorithms/l2-svm-predict.dml -nvargs X=data/haberman.test.data.csv Y=data/haberman.test.labels.csv model=data/l2-svm-model.csv fmt="csv" confusion=data/l2-svm-confusion.csv
+
+The console output should show the accuracy of the trained model in percent, i.e.:
+
+ 15/09/01 01:32:51 INFO api.DMLScript: BEGIN DML run 09/01/2015 01:32:51
+ 15/09/01 01:32:51 INFO conf.DMLConfig: Updating localtmpdir with value /tmp/systemml
+ 15/09/01 01:32:51 INFO conf.DMLConfig: Updating scratch with value scratch_space
+ 15/09/01 01:32:51 INFO conf.DMLConfig: Updating optlevel with value 2
+ 15/09/01 01:32:51 INFO conf.DMLConfig: Updating numreducers with value 10
+ 15/09/01 01:32:51 INFO conf.DMLConfig: Updating jvmreuse with value false
+ 15/09/01 01:32:51 INFO conf.DMLConfig: Updating defaultblocksize with value 1000
+ 15/09/01 01:32:51 INFO conf.DMLConfig: Updating dml.yarn.appmaster with value false
+ 15/09/01 01:32:51 INFO conf.DMLConfig: Updating dml.yarn.appmaster.mem with value 2048
+ 15/09/01 01:32:51 INFO conf.DMLConfig: Updating dml.yarn.mapreduce.mem with value 2048
+ 15/09/01 01:32:51 INFO conf.DMLConfig: Updating dml.yarn.app.queue with value default
+ 15/09/01 01:32:51 INFO conf.DMLConfig: Updating cp.parallel.matrixmult with value true
+ 15/09/01 01:32:51 INFO conf.DMLConfig: Updating cp.parallel.textio with value true
+ Accuracy (%): 74.14965986394557
+ 15/09/01 01:32:52 INFO api.DMLScript: SystemML Statistics:
+ Total execution time: 0.130 sec.
+ Number of executed MR Jobs: 0.
+
+
+The generated file `l2-svm-confusion.csv` should contain the following confusion matrix of this form:
+
+ |t1 t2|
+ |t3 t4|
+
+ * The model correctly predicted label 1 `t1` times
+ * The model incorrectly predicted label 1 as opposed to label 2 `t2` times
+ * The model incorrectly predicted label 2 as opposed to label 1 `t3` times
+ * The model correctly predicted label 2 `t4` times.
+
+If the confusion matrix looks like this ..
+
+ 107.0,38.0
+ 0.0,2.0
+
+... then the accuracy of the model is (t1+t4)/(t1+t2+t3+t4) = (107+2)/107+38+0+2) = 0.741496599
+
+<br/>
+
+Refer to the *SystemML Algorithms Reference* (`docs/SystemML_Algorithms_Reference.pdf`) for more details.
+
+<br/>
+
+# Troubleshooting
+
+If you encounter a `"java.lang.OutOfMemoryError"` you can edit the invocation
+script (`runStandaloneSystemML.sh` or `runStandaloneSystemML.bat`) to increase
+the memory available to the JVM, i.e:
+
+ java -Xmx16g -Xms4g -Xmn1g -cp ${CLASSPATH} com.ibm.bi.dml.api.DMLScript \
+ -f ${SCRIPT_FILE} -exec singlenode -config=SystemML-config.xml \
+ $@
+
+<br/>
+
+# Next Steps
+
+Check out the [SystemML Algorithm Reference](systemml-algorithms.html) and run
+some of the other pre-packaged algorithms.
+
+<style type="text/css">
+<!--
+ ol { list-style-type: none; counter-reset: item; margin: 1em; }
+ ol > li { display: table; counter-increment: item; margin-bottom: 0.1em; }
+ ol > li:before { content: counters(item, ".") ". "; display: table-cell; padding-right: 0.6em; }
+ li ol > li { margin: 0; }
+ li ol > li:before { content: counters(item, ".") " "; }
+-->
+</style>
+
+1. Descriptive Statistics
+ - [Univariate Statistics](systemml-algorithms-descriptive-statistics.html#univariate-statistics) (`Univar-Stats.dml`)
+ - [Bivariate Statistics](systemml-algorithms-descriptive-statistics.html#bivariate-statistics) (`bivar-stats.dml`)
+ - [Stratified Bivariate Statistics](systemml-algorithms-descriptive-statistics.html#stratified-bivariate-statistics) (`stratstats.dml`)
+2. Classification
+ - [Multinomial Logistic Regression](systemml-algorithms-classification.html#multinomial-logistic-regression) (`MultiLogReg.dml`)
+ - [Binary-Class Support Vector Machines](systemml-algorithms-classification.html#binary-class-support-vector-machines) (`l2-svm.dml`, `l2-svm-predict.dml`)
+ - [Multi-Class Support Vector Machines](systemml-algorithms-classification.html#multi-class-support-vector-machines) (`m-svm.dml`, `m-svm-predict.dml`)
+ - [Naive Bayes](systemml-algorithms-classification.html#naive-bayes) (`naive-bayes.dml`, `naive-bayes-predict.dml`)
+3. Clustering
+ - [K-Means Clustering](systemml-algorithms-clustering.html#k-means-clustering) (`Kmeans.dml`, `Kmeans-predict.dml`)
+4. Regression
+ - [Linear Regression](systemml-algorithms-regression.html#linear-regression) (`LinearRegDS.dml`, `LinearRegCG.dml`)
+ - [Generalized Linear Models](systemml-algorithms-regression.html#generalized-linear-models) (`GLM.dml`)
+ - [Regression Scoring and Prediction](systemml-algorithms-regression.html#regression-scoring-and-prediction) (`GLM-predict.dml`)
+5. Matrix Factorization
+ - [Principle Component Analysis](systemml-algorithms-matrix-factorization.html#principle-component-analysis) (`PCA.dml`)
+6. Interpolation
+ - [Cubic Spline Interpolation](systemml-algorithms-interpolations.html#cubic-spline)
+
+
+{% comment %} TODO: update the algorithms in the standalone package {% endcomment %}
+
+{% comment %} TODO: update the algorithms reference doc to include all algorithms, currently missing decision-tree.dml {% endcomment %}
+
+<br/>
+
+Follow the [Programming Guide](systemml-programming-guide.html) to implement
+your own SystemML algorithms.
+
[19/47] incubator-systemml git commit: Merge branch 'master' of
https://github.com/SparkTC/systemml
Posted by du...@apache.org.
Merge branch 'master' of https://github.com/SparkTC/systemml
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/42373ba0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/42373ba0
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/42373ba0
Branch: refs/heads/gh-pages
Commit: 42373ba0ad65f37151fdd98c8b65458bd0dbc6bf
Parents: a6e038a f0a60f0
Author: frreiss <fr...@us.ibm.com>
Authored: Sun Aug 23 16:55:55 2015 -0700
Committer: frreiss <fr...@us.ibm.com>
Committed: Sun Aug 23 16:55:55 2015 -0700
----------------------------------------------------------------------
README.md | 21 +
_config.yml | 16 +
_layouts/global.html | 95 +
algorithms-bibliography.md | 93 +
algorithms-classification.md | 739 +++
algorithms-clustering.md | 479 ++
algorithms-descriptive-statistics.md | 1321 ++++
algorithms-matrix-factorization.md | 143 +
algorithms-reference.md | 30 +
algorithms-regression.md | 1404 +++++
css/bootstrap-responsive.css | 1040 ++++
css/bootstrap-responsive.min.css | 9 +
css/bootstrap.css | 5624 ++++++++++++++++++
css/bootstrap.min.css | 9 +
css/main.css | 172 +
css/pygments-default.css | 76 +
dml-language-reference.md | 1256 ++++
...-quartiles-median-and-interquartile-mean.png | Bin 0 -> 26477 bytes
index.md | 23 +
js/main.js | 97 +
js/vendor/anchor.min.js | 6 +
js/vendor/bootstrap.js | 2027 +++++++
js/vendor/bootstrap.min.js | 6 +
js/vendor/jquery-1.8.0.min.js | 2 +
js/vendor/modernizr-2.6.1-respond-1.1.0.min.js | 11 +
25 files changed, 14699 insertions(+)
----------------------------------------------------------------------
[03/47] incubator-systemml git commit: Add Algorithms and Language
References
Posted by du...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/DescriptiveUnivarStats.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/DescriptiveUnivarStats.tex b/Algorithms Reference/DescriptiveUnivarStats.tex
new file mode 100644
index 0000000..fcea4ec
--- /dev/null
+++ b/Algorithms Reference/DescriptiveUnivarStats.tex
@@ -0,0 +1,582 @@
+\subsection{Univariate Statistics}
+
+\noindent{\bf Description}
+\smallskip
+
+\emph{Univariate statistics} are the simplest form of descriptive statistics in data
+analysis. They are used to quantitatively describe the main characteristics of each
+feature in the data. For a given dataset matrix, script \UnivarScriptName{} computes
+certain univariate statistics for each feature column in the
+matrix. The feature type governs the exact set of statistics computed for that feature.
+For example, the statistic \emph{mean} can only be computed on a quantitative (scale)
+feature like `Height' and `Temperature'. It does not make sense to compute the mean
+of a categorical attribute like `Hair Color'.
+
+
+\smallskip
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%\tolerance=0
+{\tt{}-f } \UnivarScriptName{}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} TYPES=}path/file
+{\tt{} STATS=}path/file
+% {\tt{} fmt=}format
+
+}
+
+
+\medskip
+\pagebreak[2]
+\noindent{\bf Arguments}
+\begin{Description}
+\item[{\tt X}:]
+Location (on HDFS) to read the data matrix $X$ whose columns we want to
+analyze as the features.
+\item[{\tt TYPES}:] % (default:\mbox{ }{\tt " "})
+Location (on HDFS) to read the single-row matrix whose $i^{\textrm{th}}$
+column-cell contains the type of the $i^{\textrm{th}}$ feature column
+\texttt{X[,$\,i$]} in the data matrix. Feature types must be encoded by
+integer numbers: $1 = {}$scale, $2 = {}$nominal, $3 = {}$ordinal.
+% The default value means ``treat all $X$-columns as scale.''
+\item[{\tt STATS}:]
+Location (on HDFS) where the output matrix of computed statistics
+will be stored. The format of the output matrix is defined by
+Table~\ref{table:univars}.
+% \item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+% Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+% see read/write functions in SystemML Language Reference for details.
+\end{Description}
+
+\begin{table}[t]\hfil
+\begin{tabular}{|rl|c|c|}
+\hline
+\multirow{2}{*}{Row}& \multirow{2}{*}{Name of Statistic} & \multicolumn{2}{c|}{Applies to:} \\
+ & & Scale & Categ.\\
+\hline
+\OutputRowIDMinimum & Minimum & + & \\
+\OutputRowIDMaximum & Maximum & + & \\
+\OutputRowIDRange & Range & + & \\
+\OutputRowIDMean & Mean & + & \\
+\OutputRowIDVariance & Variance & + & \\
+\OutputRowIDStDeviation & Standard deviation & + & \\
+\OutputRowIDStErrorMean & Standard error of mean & + & \\
+\OutputRowIDCoeffVar & Coefficient of variation & + & \\
+\OutputRowIDSkewness & Skewness & + & \\
+\OutputRowIDKurtosis & Kurtosis & + & \\
+\OutputRowIDStErrorSkewness & Standard error of skewness & + & \\
+\OutputRowIDStErrorCurtosis & Standard error of kurtosis & + & \\
+\OutputRowIDMedian & Median & + & \\
+\OutputRowIDIQMean & Inter quartile mean & + & \\
+\OutputRowIDNumCategories & Number of categories & & + \\
+\OutputRowIDMode & Mode & & + \\
+\OutputRowIDNumModes & Number of modes & & + \\
+\hline
+\end{tabular}\hfil
+\caption{The output matrix of \UnivarScriptName{} has one row per each
+univariate statistic and one column per input feature. This table lists
+the meaning of each row. Signs ``+'' show applicability to scale or/and
+to categorical features.}
+\label{table:univars}
+\end{table}
+
+
+\pagebreak[1]
+
+\smallskip
+\noindent{\bf Details}
+\smallskip
+
+Given an input matrix \texttt{X}, this script computes the set of all
+relevant univariate statistics for each feature column \texttt{X[,$\,i$]}
+in~\texttt{X}. The list of statistics to be computed depends on the
+\emph{type}, or \emph{measurement level}, of each column.
+The \textrm{TYPES} command-line argument points to a vector containing
+the types of all columns. The types must be provided as per the following
+convention: $1 = {}$scale, $2 = {}$nominal, $3 = {}$ordinal.
+
+Below we list all univariate statistics computed by script \UnivarScriptName.
+The statistics are collected by relevance into several groups, namely: central
+tendency, dispersion, shape, and categorical measures. The first three groups
+contain statistics computed for a quantitative (also known as: numerical, scale,
+or continuous) feature; the last group contains the statistics for a categorical
+(either nominal or ordinal) feature.
+
+Let~$n$ be the number of data records (rows) with feature values.
+In what follows we fix a column index \texttt{idx} and consider
+sample statistics of feature column \texttt{X[}$\,$\texttt{,}$\,$\texttt{idx]}.
+Let $v = (v_1, v_2, \ldots, v_n)$ be the values of \texttt{X[}$\,$\texttt{,}$\,$\texttt{idx]}
+in their original unsorted order: $v_i = \texttt{X[}i\texttt{,}\,\texttt{idx]}$.
+Let $v^s = (v^s_1, v^s_2, \ldots, v^s_n)$ be the same values in the sorted order,
+preserving duplicates: $v^s_1 \leq v^s_2 \leq \ldots \leq v^s_n$.
+
+\paragraph{Central tendency measures.}
+Sample statistics that describe the location of the quantitative (scale) feature distribution,
+represent it with a single value.
+\begin{Description}
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Mean]
+\OutputRowText{\OutputRowIDMean}
+The arithmetic average over a sample of a quantitative feature.
+Computed as the ratio between the sum of values and the number of values:
+$\left(\sum_{i=1}^n v_i\right)\!/n$.
+Example: the mean of sample $\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$
+equals~5.2.
+
+Note that the mean is significantly affected by extreme values in the sample
+and may be misleading as a central tendency measure if the feature varies on
+exponential scale. For example, the mean of $\{$0.01, 0.1, 1.0, 10.0, 100.0$\}$
+is 22.222, greater than all the sample values except the~largest.
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\begin{figure}[t]
+\setlength{\unitlength}{10pt}
+\begin{picture}(33,12)
+\put( 6.2, 0.0){\small 2.2}
+\put(10.2, 0.0){\small 3.2}
+\put(12.2, 0.0){\small 3.7}
+\put(15.0, 0.0){\small 4.4}
+\put(18.6, 0.0){\small 5.3}
+\put(20.2, 0.0){\small 5.7}
+\put(21.75,0.0){\small 6.1}
+\put(23.05,0.0){\small 6.4}
+\put(26.2, 0.0){\small 7.2}
+\put(28.6, 0.0){\small 7.8}
+\put( 0.5, 0.7){\small 0.0}
+\put( 0.1, 3.2){\small 0.25}
+\put( 0.5, 5.7){\small 0.5}
+\put( 0.1, 8.2){\small 0.75}
+\put( 0.5,10.7){\small 1.0}
+\linethickness{1.5pt}
+\put( 2.0, 1.0){\line(1,0){4.8}}
+\put( 6.8, 1.0){\line(0,1){1.0}}
+\put( 6.8, 2.0){\line(1,0){4.0}}
+\put(10.8, 2.0){\line(0,1){1.0}}
+\put(10.8, 3.0){\line(1,0){2.0}}
+\put(12.8, 3.0){\line(0,1){1.0}}
+\put(12.8, 4.0){\line(1,0){2.8}}
+\put(15.6, 4.0){\line(0,1){1.0}}
+\put(15.6, 5.0){\line(1,0){3.6}}
+\put(19.2, 5.0){\line(0,1){1.0}}
+\put(19.2, 6.0){\line(1,0){1.6}}
+\put(20.8, 6.0){\line(0,1){1.0}}
+\put(20.8, 7.0){\line(1,0){1.6}}
+\put(22.4, 7.0){\line(0,1){1.0}}
+\put(22.4, 8.0){\line(1,0){1.2}}
+\put(23.6, 8.0){\line(0,1){1.0}}
+\put(23.6, 9.0){\line(1,0){3.2}}
+\put(26.8, 9.0){\line(0,1){1.0}}
+\put(26.8,10.0){\line(1,0){2.4}}
+\put(29.2,10.0){\line(0,1){1.0}}
+\put(29.2,11.0){\line(1,0){4.8}}
+\linethickness{0.3pt}
+\put( 6.8, 1.0){\circle*{0.3}}
+\put(10.8, 1.0){\circle*{0.3}}
+\put(12.8, 1.0){\circle*{0.3}}
+\put(15.6, 1.0){\circle*{0.3}}
+\put(19.2, 1.0){\circle*{0.3}}
+\put(20.8, 1.0){\circle*{0.3}}
+\put(22.4, 1.0){\circle*{0.3}}
+\put(23.6, 1.0){\circle*{0.3}}
+\put(26.8, 1.0){\circle*{0.3}}
+\put(29.2, 1.0){\circle*{0.3}}
+\put( 6.8, 1.0){\vector(1,0){27.2}}
+\put( 2.0, 1.0){\vector(0,1){10.8}}
+\put( 2.0, 3.5){\line(1,0){10.8}}
+\put( 2.0, 6.0){\line(1,0){17.2}}
+\put( 2.0, 8.5){\line(1,0){21.6}}
+\put( 2.0,11.0){\line(1,0){27.2}}
+\put(12.8, 1.0){\line(0,1){2.0}}
+\put(19.2, 1.0){\line(0,1){5.0}}
+\put(20.0, 1.0){\line(0,1){5.0}}
+\put(23.6, 1.0){\line(0,1){7.0}}
+\put( 9.0, 4.0){\line(1,0){3.8}}
+\put( 9.2, 2.7){\vector(0,1){0.8}}
+\put( 9.2, 4.8){\vector(0,-1){0.8}}
+\put(19.4, 8.0){\line(1,0){3.0}}
+\put(19.6, 7.2){\vector(0,1){0.8}}
+\put(19.6, 9.3){\vector(0,-1){0.8}}
+\put(13.0, 2.2){\small $q_{25\%}$}
+\put(17.3, 2.2){\small $q_{50\%}$}
+\put(23.8, 2.2){\small $q_{75\%}$}
+\put(20.15,3.5){\small $\mu$}
+\put( 8.0, 3.75){\small $\phi_1$}
+\put(18.35,7.8){\small $\phi_2$}
+\end{picture}
+\label{fig:example_quartiles}
+\caption{The computation of quartiles, median, and interquartile mean from the
+empirical distribution function of the 10-point
+sample $\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$. Each vertical step in
+the graph has height~$1{/}n = 0.1$. Values $q_{25\%}$, $q_{50\%}$, and $q_{75\%}$ denote
+the $1^{\textrm{st}}$, $2^{\textrm{nd}}$, and $3^{\textrm{rd}}$ quartiles correspondingly;
+value~$\mu$ denotes the median. Values $\phi_1$ and $\phi_2$ show the partial contribution
+of border points (quartiles) $v_3=3.7$ and $v_8=6.4$ into the interquartile mean.}
+\end{figure}
+
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Median]
+\OutputRowText{\OutputRowIDMedian}
+The ``middle'' value that separates the higher half of the sample values
+(in a sorted order) from the lower half.
+To compute the median, we sort the sample in the increasing order, preserving
+duplicates: $v^s_1 \leq v^s_2 \leq \ldots \leq v^s_n$.
+If $n$ is odd, the median equals $v^s_i$ where $i = (n\,{+}\,1)\,{/}\,2$,
+same as the $50^{\textrm{th}}$~percentile of the sample.
+If $n$ is even, there are two ``middle'' values $v^s_{n/2}$ and $v^s_{n/2\,+\,1}$,
+so we compute the median as the mean of these two values.
+(For even~$n$ we compute the $50^{\textrm{th}}$~percentile as~$v^s_{n/2}$,
+not as the median.) Example: the median of sample
+$\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$
+equals $(5.3\,{+}\,5.7)\,{/}\,2$~${=}$~5.5, see Figure~\ref{fig:example_quartiles}.
+
+Unlike the mean, the median is not sensitive to extreme values in the sample,
+i.e.\ it is robust to outliers. It works better as a measure of central tendency
+for heavy-tailed distributions and features that vary on exponential scale.
+However, the median is sensitive to small sample size.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Interquartile mean]
+\OutputRowText{\OutputRowIDIQMean}
+For a sample of a quantitative feature, this is
+the mean of the values greater than or equal to the $1^{\textrm{st}}$ quartile
+and less than or equal the $3^{\textrm{rd}}$ quartile.
+In other words, it is a ``truncated mean'' where the lowest 25$\%$ and
+the highest 25$\%$ of the sorted values are omitted in its computation.
+The two ``border values'', i.e.\ the $1^{\textrm{st}}$ and the $3^{\textrm{rd}}$
+quartiles themselves, contribute to this mean only partially.
+This measure is occasionally used as the ``robust'' version of the mean
+that is less sensitive to the extreme values.
+
+To compute the measure, we sort the sample in the increasing order,
+preserving duplicates: $v^s_1 \leq v^s_2 \leq \ldots \leq v^s_n$.
+We set $j = \lceil n{/}4 \rceil$ for the $1^{\textrm{st}}$ quartile index
+and $k = \lceil 3n{/}4 \rceil$ for the $3^{\textrm{rd}}$ quartile index,
+then compute the following weighted mean:
+\begin{equation*}
+\frac{1}{3{/}4 - 1{/}4} \left[
+\left(\frac{j}{n} - \frac{1}{4}\right) v^s_j \,\,+
+\sum_{j<i<k} \left(\frac{i}{n} - \frac{i\,{-}\,1}{n}\right) v^s_i
+\,\,+\,\, \left(\frac{3}{4} - \frac{k\,{-}\,1}{n}\right) v^s_k\right]
+\end{equation*}
+In other words, all sample values between the $1^{\textrm{st}}$ and the $3^{\textrm{rd}}$
+quartile enter the sum with weights $2{/}n$, times their number of duplicates, while the
+two quartiles themselves enter the sum with reduced weights. The weights are proportional
+to the vertical steps in the empirical distribution function of the sample, see
+Figure~\ref{fig:example_quartiles} for an illustration.
+Example: the interquartile mean of sample
+$\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$ equals the sum
+$0.1 (3.7\,{+}\,6.4) + 0.2 (4.4\,{+}\,5.3\,{+}\,5.7\,{+}\,6.1)$,
+which equals~5.31.
+\end{Description}
+
+
+\paragraph{Dispersion measures.}
+Statistics that describe the amount of variation or spread in a quantitative
+(scale) data feature.
+\begin{Description}
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Variance]
+\OutputRowText{\OutputRowIDVariance}
+A measure of dispersion, or spread-out, of sample values around their mean,
+expressed in units that are the square of those of the feature itself.
+Computed as the sum of squared differences between the values
+in the sample and their mean, divided by one less than the number of
+values: $\sum_{i=1}^n (v_i - \bar{v})^2\,/\,(n\,{-}\,1)$ where
+$\bar{v}=\left(\sum_{i=1}^n v_i\right)\!/n$.
+Example: the variance of sample
+$\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$ equals~3.24.
+Note that at least two values ($n\geq 2$) are required to avoid division
+by zero. Sample variance is sensitive to outliers, even more than the mean.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Standard deviation]
+\OutputRowText{\OutputRowIDStDeviation}
+A measure of dispersion around the mean, the square root of variance.
+Computed by taking the square root of the sample variance;
+see \emph{Variance} above on computing the variance.
+Example: the standard deviation of sample
+$\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$ equals~1.8.
+At least two values are required to avoid division by zero.
+Note that standard deviation is sensitive to outliers.
+
+Standard deviation is used in conjunction with the mean to determine
+an interval containing a given percentage of the feature values,
+assuming the normal distribution. In a large sample from a normal
+distribution, around 68\% of the cases fall within one standard
+deviation and around 95\% of cases fall within two standard deviations
+of the mean. For example, if the mean age is 45 with a standard deviation
+of 10, around 95\% of the cases would be between 25 and 65 in a normal
+distribution.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Coefficient of variation]
+\OutputRowText{\OutputRowIDCoeffVar}
+The ratio of the standard deviation to the mean, i.e.\ the
+\emph{relative} standard deviation, of a quantitative feature sample.
+Computed by dividing the sample \emph{standard deviation} by the
+sample \emph{mean}, see above for their computation details.
+Example: the coefficient of variation for sample
+$\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$
+equals 1.8$\,{/}\,$5.2~${\approx}$~0.346.
+
+This metric is used primarily with non-negative features such as
+financial or population data. It is sensitive to outliers.
+Note: zero mean causes division by zero, returning infinity or \texttt{NaN}.
+At least two values (records) are required to compute the standard deviation.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Minimum]
+\OutputRowText{\OutputRowIDMinimum}
+The smallest value of a quantitative sample, computed as $\min v = v^s_1$.
+Example: the minimum of sample
+$\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$
+equals~2.2.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Maximum]
+\OutputRowText{\OutputRowIDMaximum}
+The largest value of a quantitative sample, computed as $\max v = v^s_n$.
+Example: the maximum of sample
+$\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$
+equals~7.8.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Range]
+\OutputRowText{\OutputRowIDRange}
+The difference between the largest and the smallest value of a quantitative
+sample, computed as $\max v - \min v = v^s_n - v^s_1$.
+It provides information about the overall spread of the sample values.
+Example: the range of sample
+$\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$
+equals 7.8$\,{-}\,$2.2~${=}$~5.6.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Standard error of the mean]
+\OutputRowText{\OutputRowIDStErrorMean}
+A measure of how much the value of the sample mean may vary from sample
+to sample taken from the same (hypothesized) distribution of the feature.
+It helps to roughly bound the distribution mean, i.e.\
+the limit of the sample mean as the sample size tends to infinity.
+Under certain assumptions (e.g.\ normality and large sample), the difference
+between the distribution mean and the sample mean is unlikely to exceed
+2~standard errors.
+
+The measure is computed by dividing the sample standard deviation
+by the square root of the number of values~$n$; see \emph{standard deviation}
+for its computation details. Ensure $n\,{\geq}\,2$ to avoid division by~0.
+Example: for sample
+$\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$
+with the mean of~5.2 the standard error of the mean
+equals 1.8$\,{/}\sqrt{10}$~${\approx}$~0.569.
+
+Note that the standard error itself is subject to sample randomness.
+Its accuracy as an error estimator may be low if the sample size is small
+or \mbox{non-i.i.d.}, if there are outliers, or if the distribution has
+heavy tails.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+% \item[\it Quartiles]
+% \OutputRowText{\OutputRowIDQuartiles}
+% %%% dsDefn %%%%
+% The values of a quantitative feature
+% that divide an ordered/sorted set of data records into four equal-size groups.
+% The $1^{\textrm{st}}$ quartile, or the $25^{\textrm{th}}$ percentile, splits
+% the sorted data into the lowest $25\%$ and the highest~$75\%$. In other words,
+% it is the middle value between the minimum and the median. The $2^{\textrm{nd}}$
+% quartile is the median itself, the value that separates the higher half of
+% the data (in the sorted order) from the lower half. Finally, the $3^{\textrm{rd}}$
+% quartile, or the $75^{\textrm{th}}$ percentile, divides the sorted data into
+% lowest $75\%$ and highest~$25\%$.\par
+% %%% dsComp %%%%
+% To compute the quartiles for a data column \texttt{X[,i]} with $n$ numerical values
+% we sort it in the increasing order, preserving duplicates, then return
+% \texttt{X}${}^{\textrm{sort}}$\texttt{[}$k$\texttt{,i]}
+% where $k = \lceil pn \rceil$ for $p = 0.25$, $0.5$, and~$0.75$.
+% When $n$ is even, the $2^{\textrm{nd}}$ quartile (the median) is further adjusted
+% to equal the mean of two middle values
+% $\texttt{X}^{\textrm{sort}}\texttt{[}n{/}2\texttt{,i]}$ and
+% $\texttt{X}^{\textrm{sort}}\texttt{[}n{/}2\,{+}\,1\texttt{,i]}$.
+% %%% dsWarn %%%%
+% We assume that the feature column does not contain \texttt{NaN}s or coded non-numeric values.
+% %%% dsExmpl %%%
+% \textbf{Example(s).}
+\end{Description}
+
+
+\paragraph{Shape measures.}
+Statistics that describe the shape and symmetry of the quantitative (scale)
+feature distribution estimated from a sample of its values.
+\begin{Description}
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Skewness]
+\OutputRowText{\OutputRowIDSkewness}
+It measures how symmetrically the values of a feature are spread out
+around the mean. A significant positive skewness implies a longer (or fatter)
+right tail, i.e. feature values tend to lie farther away from the mean on the
+right side. A significant negative skewness implies a longer (or fatter) left
+tail. The normal distribution is symmetric and has a skewness value of~0;
+however, its sample skewness is likely to be nonzero, just close to zero.
+As a guideline, a skewness value more than twice its standard error is taken
+to indicate a departure from symmetry.
+
+Skewness is computed as the $3^{\textrm{rd}}$~central moment divided by the cube
+of the standard deviation. We estimate the $3^{\textrm{rd}}$~central moment as
+the sum of cubed differences between the values in the feature column and their
+sample mean, divided by the number of values:
+$\sum_{i=1}^n (v_i - \bar{v})^3 / n$
+where $\bar{v}=\left(\sum_{i=1}^n v_i\right)\!/n$.
+The standard deviation is computed
+as described above in \emph{standard deviation}. To avoid division by~0,
+at least two different sample values are required. Example: for sample
+$\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$
+with the mean of~5.2 and the standard deviation of~1.8
+skewness is estimated as $-1.0728\,{/}\,1.8^3 \approx -0.184$.
+Note: skewness is sensitive to outliers.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Standard error in skewness]
+\OutputRowText{\OutputRowIDStErrorSkewness}
+A measure of how much the sample skewness may vary from sample to sample,
+assuming that the feature is normally distributed, which makes its
+distribution skewness equal~0.
+Given the number~$n$ of sample values, the standard error is computed as
+\begin{equation*}
+\sqrt{\frac{6n\,(n-1)}{(n-2)(n+1)(n+3)}}
+\end{equation*}
+This measure can tell us, for example:
+\begin{Itemize}
+\item If the sample skewness lands within two standard errors from~0, its
+positive or negative sign is non-significant, may just be accidental.
+\item If the sample skewness lands outside this interval, the feature
+is unlikely to be normally distributed.
+\end{Itemize}
+At least 3~values ($n\geq 3$) are required to avoid arithmetic failure.
+Note that the standard error is inaccurate if the feature distribution is
+far from normal or if the number of samples is small.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Kurtosis]
+\OutputRowText{\OutputRowIDKurtosis}
+As a distribution parameter, kurtosis is a measure of the extent to which
+feature values cluster around a central point. In other words, it quantifies
+``peakedness'' of the distribution: how tall and sharp the central peak is
+relative to a standard bell curve.
+
+Positive kurtosis (\emph{leptokurtic} distribution) indicates that, relative
+to a normal distribution:
+\begin{Itemize}
+\item observations cluster more about the center (peak-shaped),
+\item the tails are thinner at non-extreme values,
+\item the tails are thicker at extreme values.
+\end{Itemize}
+Negative kurtosis (\emph{platykurtic} distribution) indicates that, relative
+to a normal distribution:
+\begin{Itemize}
+\item observations cluster less about the center (box-shaped),
+\item the tails are thicker at non-extreme values,
+\item the tails are thinner at extreme values.
+\end{Itemize}
+Kurtosis of a normal distribution is zero; however, the sample kurtosis
+(computed here) is likely to deviate from zero.
+
+Sample kurtosis is computed as the $4^{\textrm{th}}$~central moment divided
+by the $4^{\textrm{th}}$~power of the standard deviation, minus~3.
+We estimate the $4^{\textrm{th}}$~central moment as the sum of the
+$4^{\textrm{th}}$~powers of differences between the values in the feature column
+and their sample mean, divided by the number of values:
+$\sum_{i=1}^n (v_i - \bar{v})^4 / n$
+where $\bar{v}=\left(\sum_{i=1}^n v_i\right)\!/n$.
+The standard deviation is computed as described above, see \emph{standard deviation}.
+
+Note that kurtosis is sensitive to outliers, and requires at least two different
+sample values. Example: for sample
+$\{$2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8$\}$
+with the mean of~5.2 and the standard deviation of~1.8,
+sample kurtosis equals $16.6962\,{/}\,1.8^4 - 3 \approx -1.41$.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Standard error in kurtosis]
+\OutputRowText{\OutputRowIDStErrorCurtosis}
+A measure of how much the sample kurtosis may vary from sample to sample,
+assuming that the feature is normally distributed, which makes its
+distribution kurtosis equal~0.
+Given the number~$n$ of sample values, the standard error is computed as
+\begin{equation*}
+\sqrt{\frac{24n\,(n-1)^2}{(n-3)(n-2)(n+3)(n+5)}}
+\end{equation*}
+This measure can tell us, for example:
+\begin{Itemize}
+\item If the sample kurtosis lands within two standard errors from~0, its
+positive or negative sign is non-significant, may just be accidental.
+\item If the sample kurtosis lands outside this interval, the feature
+is unlikely to be normally distributed.
+\end{Itemize}
+At least 4~values ($n\geq 4$) are required to avoid arithmetic failure.
+Note that the standard error is inaccurate if the feature distribution is
+far from normal or if the number of samples is small.
+\end{Description}
+
+
+\paragraph{Categorical measures.} Statistics that describe the sample of
+a categorical feature, either nominal or ordinal. We represent all
+categories by integers from~1 to the number of categories; we call
+these integers \emph{category~IDs}.
+\begin{Description}
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Number of categories]
+\OutputRowText{\OutputRowIDNumCategories}
+The maximum category~ID that occurs in the sample. Note that some
+categories with~IDs \emph{smaller} than this maximum~ID may have
+no~occurrences in the sample, without reducing the number of categories.
+However, any categories with~IDs \emph{larger} than the maximum~ID with
+no occurrences in the sample will not be counted.
+Example: in sample $\{$1, 3, 3, 3, 3, 4, 4, 5, 7, 7, 7, 7, 8, 8, 8$\}$
+the number of categories is reported as~8. Category~IDs 2 and~6, which have
+zero occurrences, are still counted; but if there is a category with
+ID${}=9$ and zero occurrences, it is not counted.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Mode]
+\OutputRowText{\OutputRowIDMode}
+The most frequently occurring category value.
+If several values share the greatest frequency of occurrence, then each
+of them is a mode; but here we report only the smallest of these modes.
+Example: in sample $\{$1, 3, 3, 3, 3, 4, 4, 5, 7, 7, 7, 7, 8, 8, 8$\}$
+the modes are 3 and~7, with 3 reported.
+
+Computed by counting the number of occurrences for each category,
+then taking the smallest category~ID that has the maximum count.
+Note that the sample modes may be different from the distribution modes,
+i.e.\ the categories whose (hypothesized) underlying probability is the
+maximum over all categories.
+%%%%%%%%%%%%%%%%%%%% DESCRIPTIVE STATISTIC %%%%%%%%%%%%%%%%%%%%
+\item[\it Number of modes]
+\OutputRowText{\OutputRowIDNumModes}
+The number of category values that each have the largest frequency
+count in the sample.
+Example: in sample $\{$1, 3, 3, 3, 3, 4, 4, 5, 7, 7, 7, 7, 8, 8, 8$\}$
+there are two category IDs (3 and~7) that occur the maximum count of 4~times;
+hence, we return~2.
+
+Computed by counting the number of occurrences for each category,
+then counting how many categories have the maximum count.
+Note that the sample modes may be different from the distribution modes,
+i.e.\ the categories whose (hypothesized) underlying probability is the
+maximum over all categories.
+\end{Description}
+
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+The output matrix containing all computed statistics is of size $17$~rows and
+as many columns as in the input matrix~\texttt{X}. Each row corresponds to
+a particular statistic, according to the convention specified in
+Table~\ref{table:univars}. The first $14$~statistics are applicable for
+\emph{scale} columns, and the last $3$~statistics are applicable for categorical,
+i.e.\ nominal and ordinal, columns.
+
+
+\pagebreak[2]
+
+\smallskip
+\noindent{\bf Examples}
+\smallskip
+
+{\hangindent=\parindent\noindent\tt
+\hml -f \UnivarScriptName{} -nvargs X=/user/biadmin/X.mtx
+ TYPES=/user/biadmin/types.mtx
+ STATS=/user/biadmin/stats.mtx
+
+}
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/GLM.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/GLM.tex b/Algorithms Reference/GLM.tex
new file mode 100644
index 0000000..e4d13bf
--- /dev/null
+++ b/Algorithms Reference/GLM.tex
@@ -0,0 +1,409 @@
+\subsection{Generalized Linear Models (GLM)}
+
+\noindent{\bf Description}
+\smallskip
+
+Generalized Linear Models~\cite{Gill2000:GLM,McCullagh1989:GLM,Nelder1972:GLM}
+extend the methodology of linear and logistic regression to a variety of
+distributions commonly assumed as noise effects in the response variable.
+As before, we are given a collection
+of records $(x_1, y_1)$, \ldots, $(x_n, y_n)$ where $x_i$ is a numerical vector of
+explanatory (feature) variables of size~\mbox{$\dim x_i = m$}, and $y_i$ is the
+response (dependent) variable observed for this vector. GLMs assume that some
+linear combination of the features in~$x_i$ determines the \emph{mean}~$\mu_i$
+of~$y_i$, while the observed $y_i$ is a random outcome of a noise distribution
+$\Prob[y\mid \mu_i]\,$\footnote{$\Prob[y\mid \mu_i]$ is given by a density function
+if $y$ is continuous.}
+with that mean~$\mu_i$:
+\begin{equation*}
+x_i \,\,\,\,\mapsto\,\,\,\, \eta_i = \beta_0 + \sum\nolimits_{j=1}^m \beta_j x_{i,j}
+\,\,\,\,\mapsto\,\,\,\, \mu_i \,\,\,\,\mapsto \,\,\,\, y_i \sim \Prob[y\mid \mu_i]
+\end{equation*}
+
+In linear regression the response mean $\mu_i$ \emph{equals} some linear combination
+over~$x_i$, denoted above by~$\eta_i$.
+In logistic regression with $y\in\{0, 1\}$ (Bernoulli) the mean of~$y$ is the same
+as $\Prob[y=1]$ and equals $1/(1+e^{-\eta_i})$, the logistic function of~$\eta_i$.
+In GLM, $\mu_i$ and $\eta_i$ can be related via any given smooth monotone function
+called the \emph{link function}: $\eta_i = g(\mu_i)$. The unknown linear combination
+parameters $\beta_j$ are assumed to be the same for all records.
+
+The goal of the regression is to estimate the parameters~$\beta_j$ from the observed
+data. Once the~$\beta_j$'s are accurately estimated, we can make predictions
+about~$y$ for a new feature vector~$x$. To do so, compute $\eta$ from~$x$ and use
+the inverted link function $\mu = g^{-1}(\eta)$ to compute the mean $\mu$ of~$y$;
+then use the distribution $\Prob[y\mid \mu]$ to make predictions about~$y$.
+Both $g(\mu)$ and $\Prob[y\mid \mu]$ are user-provided. Our GLM script supports
+a standard set of distributions and link functions, see below for details.
+
+\smallskip
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+{\tt{}-f }path/\/{\tt{}GLM.dml}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} Y=}path/file
+{\tt{} B=}path/file
+{\tt{} fmt=}format
+{\tt{} O=}path/file
+{\tt{} Log=}path/file
+{\tt{} dfam=}int
+{\tt{} vpow=}double
+{\tt{} link=}int
+{\tt{} lpow=}double
+{\tt{} yneg=}double
+{\tt{} icpt=}int
+{\tt{} reg=}double
+{\tt{} tol=}double
+{\tt{} disp=}double
+{\tt{} moi=}int
+{\tt{} mii=}int
+
+}
+
+\smallskip
+\noindent{\bf Arguments}
+\begin{Description}
+\item[{\tt X}:]
+Location (on HDFS) to read the matrix of feature vectors; each row constitutes
+an example.
+\item[{\tt Y}:]
+Location to read the response matrix, which may have 1 or 2 columns
+\item[{\tt B}:]
+Location to store the estimated regression parameters (the $\beta_j$'s), with the
+intercept parameter~$\beta_0$ at position {\tt B[}$m\,{+}\,1$, {\tt 1]} if available
+\item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+see read/write functions in SystemML Language Reference for details.
+\item[{\tt O}:] (default:\mbox{ }{\tt " "})
+Location to write certain summary statistics described in Table~\ref{table:GLM:stats},
+by default it is standard output.
+\item[{\tt Log}:] (default:\mbox{ }{\tt " "})
+Location to store iteration-specific variables for monitoring and debugging purposes,
+see Table~\ref{table:GLM:log} for details.
+\item[{\tt dfam}:] (default:\mbox{ }{\tt 1})
+Distribution family code to specify $\Prob[y\mid \mu]$, see Table~\ref{table:commonGLMs}:\\
+{\tt 1} = power distributions with $\Var(y) = \mu^{\alpha}$;
+{\tt 2} = binomial or Bernoulli
+\item[{\tt vpow}:] (default:\mbox{ }{\tt 0.0})
+When {\tt dfam=1}, this provides the~$q$ in $\Var(y) = a\mu^q$, the power
+dependence of the variance of~$y$ on its mean. In particular, use:\\
+{\tt 0.0} = Gaussian,
+{\tt 1.0} = Poisson,
+{\tt 2.0} = Gamma,
+{\tt 3.0} = inverse Gaussian
+\item[{\tt link}:] (default:\mbox{ }{\tt 0})
+Link function code to determine the link function~$\eta = g(\mu)$:\\
+{\tt 0} = canonical link (depends on the distribution family), see Table~\ref{table:commonGLMs};\\
+{\tt 1} = power functions,
+{\tt 2} = logit,
+{\tt 3} = probit,
+{\tt 4} = cloglog,
+{\tt 5} = cauchit
+\item[{\tt lpow}:] (default:\mbox{ }{\tt 1.0})
+When {\tt link=1}, this provides the~$s$ in $\eta = \mu^s$, the power link
+function; {\tt lpow=0.0} gives the log link $\eta = \log\mu$. Common power links:\\
+{\tt -2.0} = $1/\mu^2$,
+{\tt -1.0} = reciprocal,
+{\tt 0.0} = log,
+{\tt 0.5} = sqrt,
+{\tt 1.0} = identity
+\item[{\tt yneg}:] (default:\mbox{ }{\tt 0.0})
+When {\tt dfam=2} and the response matrix $Y$ has 1~column,
+this specifies the $y$-value used for Bernoulli ``No'' label.
+All other $y$-values are treated as the ``Yes'' label.
+For example, {\tt yneg=-1.0} may be used when $y\in\{-1, 1\}$;
+either {\tt yneg=1.0} or {\tt yneg=2.0} may be used when $y\in\{1, 2\}$.
+\item[{\tt icpt}:] (default:\mbox{ }{\tt 0})
+Intercept and shifting/rescaling of the features in~$X$:\\
+{\tt 0} = no intercept (hence no~$\beta_0$), no shifting/rescaling of the features;\\
+{\tt 1} = add intercept, but do not shift/rescale the features in~$X$;\\
+{\tt 2} = add intercept, shift/rescale the features in~$X$ to mean~0, variance~1
+\item[{\tt reg}:] (default:\mbox{ }{\tt 0.0})
+L2-regularization parameter (lambda)
+\item[{\tt tol}:] (default:\mbox{ }{\tt 0.000001})
+Tolerance (epsilon) used in the convergence criterion: we terminate the outer iterations
+when the deviance changes by less than this factor; see below for details
+\item[{\tt disp}:] (default:\mbox{ }{\tt 0.0})
+Dispersion parameter, or {\tt 0.0} to estimate it from data
+\item[{\tt moi}:] (default:\mbox{ }{\tt 200})
+Maximum number of outer (Fisher scoring) iterations
+\item[{\tt mii}:] (default:\mbox{ }{\tt 0})
+Maximum number of inner (conjugate gradient) iterations, or~0 if no maximum
+limit provided
+\end{Description}
+
+
+\begin{table}[t]\small\centerline{%
+\begin{tabular}{|ll|}
+\hline
+Name & Meaning \\
+\hline
+{\tt TERMINATION\_CODE} & A positive integer indicating success/failure as follows: \\
+ & $1 = {}$Converged successfully;
+ $2 = {}$Maximum \# of iterations reached; \\
+ & $3 = {}$Input ({\tt X}, {\tt Y}) out of range;
+ $4 = {}$Distribution/link not supported \\
+{\tt BETA\_MIN} & Smallest beta value (regression coefficient), excluding the intercept \\
+{\tt BETA\_MIN\_INDEX} & Column index for the smallest beta value \\
+{\tt BETA\_MAX} & Largest beta value (regression coefficient), excluding the intercept \\
+{\tt BETA\_MAX\_INDEX} & Column index for the largest beta value \\
+{\tt INTERCEPT} & Intercept value, or NaN if there is no intercept (if {\tt icpt=0}) \\
+{\tt DISPERSION} & Dispersion used to scale deviance, provided in {\tt disp} input argument \\
+ & or estimated (same as {\tt DISPERSION\_EST}) if {\tt disp} argument is${} \leq 0$ \\
+{\tt DISPERSION\_EST} & Dispersion estimated from the dataset \\
+{\tt DEVIANCE\_UNSCALED} & Deviance from the saturated model, assuming dispersion${} = 1.0$ \\
+{\tt DEVIANCE\_SCALED} & Deviance from the saturated model, scaled by {\tt DISPERSION} value \\
+\hline
+\end{tabular}}
+\caption{Besides~$\beta$, GLM regression script computes a few summary statistics listed above.
+They are provided in CSV format, one comma-separated name-value pair per each line.}
+\label{table:GLM:stats}
+\end{table}
+
+
+
+
+
+
+\begin{table}[t]\small\centerline{%
+\begin{tabular}{|ll|}
+\hline
+Name & Meaning \\
+\hline
+{\tt NUM\_CG\_ITERS} & Number of inner (Conj.\ Gradient) iterations in this outer iteration \\
+{\tt IS\_TRUST\_REACHED} & $1 = {}$trust region boundary was reached, $0 = {}$otherwise \\
+{\tt POINT\_STEP\_NORM} & L2-norm of iteration step from old point ($\beta$-vector) to new point \\
+{\tt OBJECTIVE} & The loss function we minimize (negative partial log-likelihood) \\
+{\tt OBJ\_DROP\_REAL} & Reduction in the objective during this iteration, actual value \\
+{\tt OBJ\_DROP\_PRED} & Reduction in the objective predicted by a quadratic approximation \\
+{\tt OBJ\_DROP\_RATIO} & Actual-to-predicted reduction ratio, used to update the trust region \\
+{\tt GRADIENT\_NORM} & L2-norm of the loss function gradient (omitted if point is rejected) \\
+{\tt LINEAR\_TERM\_MIN} & The minimum value of $X \pxp \beta$, used to check for overflows \\
+{\tt LINEAR\_TERM\_MAX} & The maximum value of $X \pxp \beta$, used to check for overflows \\
+{\tt IS\_POINT\_UPDATED} & $1 = {}$new point accepted; $0 = {}$new point rejected, old point restored \\
+{\tt TRUST\_DELTA} & Updated trust region size, the ``delta'' \\
+\hline
+\end{tabular}}
+\caption{
+The {\tt Log} file for GLM regression contains the above \mbox{per-}iteration
+variables in CSV format, each line containing triple (Name, Iteration\#, Value) with Iteration\#
+being~0 for initial values.}
+\label{table:GLM:log}
+\end{table}
+
+\begin{table}[t]\hfil
+\begin{tabular}{|ccccccc|}
+\hline
+\multicolumn{4}{|c}{INPUT PARAMETERS} & Distribution & Link & Cano- \\
+{\tt dfam} & {\tt vpow} & {\tt link} & {\tt\ lpow} & family & function & nical?\\
+\hline
+{\tt 1} & {\tt 0.0} & {\tt 1} & {\tt -1.0} & Gaussian & inverse & \\
+{\tt 1} & {\tt 0.0} & {\tt 1} & {\tt\ 0.0} & Gaussian & log & \\
+{\tt 1} & {\tt 0.0} & {\tt 1} & {\tt\ 1.0} & Gaussian & identity & Yes \\
+{\tt 1} & {\tt 1.0} & {\tt 1} & {\tt\ 0.0} & Poisson & log & Yes \\
+{\tt 1} & {\tt 1.0} & {\tt 1} & {\tt\ 0.5} & Poisson & sq.root & \\
+{\tt 1} & {\tt 1.0} & {\tt 1} & {\tt\ 1.0} & Poisson & identity & \\
+{\tt 1} & {\tt 2.0} & {\tt 1} & {\tt -1.0} & Gamma & inverse & Yes \\
+{\tt 1} & {\tt 2.0} & {\tt 1} & {\tt\ 0.0} & Gamma & log & \\
+{\tt 1} & {\tt 2.0} & {\tt 1} & {\tt\ 1.0} & Gamma & identity & \\
+{\tt 1} & {\tt 3.0} & {\tt 1} & {\tt -2.0} & Inverse Gauss & $1/\mu^2$ & Yes \\
+{\tt 1} & {\tt 3.0} & {\tt 1} & {\tt -1.0} & Inverse Gauss & inverse & \\
+{\tt 1} & {\tt 3.0} & {\tt 1} & {\tt\ 0.0} & Inverse Gauss & log & \\
+{\tt 1} & {\tt 3.0} & {\tt 1} & {\tt\ 1.0} & Inverse Gauss & identity & \\
+\hline
+{\tt 2} & {\tt *} & {\tt 1} & {\tt\ 0.0} & Binomial & log & \\
+{\tt 2} & {\tt *} & {\tt 1} & {\tt\ 0.5} & Binomial & sq.root & \\
+{\tt 2} & {\tt *} & {\tt 2} & {\tt\ *} & Binomial & logit & Yes \\
+{\tt 2} & {\tt *} & {\tt 3} & {\tt\ *} & Binomial & probit & \\
+{\tt 2} & {\tt *} & {\tt 4} & {\tt\ *} & Binomial & cloglog & \\
+{\tt 2} & {\tt *} & {\tt 5} & {\tt\ *} & Binomial & cauchit & \\
+\hline
+\end{tabular}\hfil
+\caption{Common GLM distribution families and link functions.
+(Here ``{\tt *}'' stands for ``any value.'')}
+\label{table:commonGLMs}
+\end{table}
+
+\noindent{\bf Details}
+\smallskip
+
+In GLM, the noise distribution $\Prob[y\mid \mu]$ of the response variable~$y$
+given its mean~$\mu$ is restricted to have the \emph{exponential family} form
+\begin{equation}
+Y \sim\, \Prob[y\mid \mu] \,=\, \exp\left(\frac{y\theta - b(\theta)}{a}
++ c(y, a)\right),\,\,\textrm{where}\,\,\,\mu = \E(Y) = b'(\theta).
+\label{eqn:GLM}
+\end{equation}
+Changing the mean in such a distribution simply multiplies all \mbox{$\Prob[y\mid \mu]$}
+by~$e^{\,y\hspace{0.2pt}\theta/a}$ and rescales them so that they again integrate to~1.
+Parameter $\theta$ is called \emph{canonical}, and the function $\theta = b'^{\,-1}(\mu)$
+that relates it to the mean is called the~\emph{canonical link}; constant~$a$ is called
+\emph{dispersion} and rescales the variance of~$y$. Many common distributions can be put
+into this form, see Table~\ref{table:commonGLMs}. The canonical parameter~$\theta$
+is often chosen to coincide with~$\eta$, the linear combination of the regression features;
+other choices for~$\eta$ are possible too.
+
+Rather than specifying the canonical link, GLM distributions are commonly defined
+by their variance $\Var(y)$ as the function of the mean~$\mu$. It can be shown
+from Eq.~(\ref{eqn:GLM}) that $\Var(y) = a\,b''(\theta) = a\,b''(b'^{\,-1}(\mu))$.
+For example, for the Bernoulli distribution $\Var(y) = \mu(1-\mu)$, for the Poisson
+distribution \mbox{$\Var(y) = \mu$}, and for the Gaussian distribution
+$\Var(y) = a\cdot 1 = \sigma^2$.
+It turns out that for many common distributions $\Var(y) = a\mu^q$, a power function.
+We support all distributions where $\Var(y) = a\mu^q$, as well as the Bernoulli and
+the binomial distributions.
+
+For distributions with $\Var(y) = a\mu^q$ the canonical link is also a power function,
+namely $\theta = \mu^{1-q}/(1-q)$, except for the Poisson ($q = 1$) whose canonical link is
+$\theta = \log\mu$. We support all power link functions in the form $\eta = \mu^s$,
+dropping any constant factor, with $\eta = \log\mu$ for $s=0$. The binomial distribution
+has its own family of link functions, which includes logit (the canonical link),
+probit, cloglog, and cauchit (see Table~\ref{table:binomial_links}); we support these
+only for the binomial and Bernoulli distributions. Links and distributions are specified
+via four input parameters: {\tt dfam}, {\tt vpow}, {\tt link}, and {\tt lpow} (see
+Table~\ref{table:commonGLMs}).
+
+\begin{table}[t]\hfil
+\begin{tabular}{|cc|cc|}
+\hline
+Name & Link function & Name & Link function \\
+\hline
+Logit & $\displaystyle \eta = 1 / \big(1 + e^{-\mu}\big)^{\mathstrut}$ &
+Cloglog & $\displaystyle \eta = \log \big(\!- \log(1 - \mu)\big)^{\mathstrut}$ \\
+Probit & $\displaystyle \mu = \frac{1}{\sqrt{2\pi}}\int\nolimits_{-\infty_{\mathstrut}}^{\,\eta\mathstrut}
+ \!\!\!\!\! e^{-\frac{t^2}{2}} dt$ &
+Cauchit & $\displaystyle \eta = \tan\pi(\mu - 1/2)$ \\
+\hline
+\end{tabular}\hfil
+\caption{The supported non-power link functions for the Bernoulli and the binomial
+distributions. (Here $\mu$~is the Bernoulli mean.)}
+\label{table:binomial_links}
+\end{table}
+
+The observed response values are provided to the regression script as matrix~$Y$
+having 1 or 2 columns. If a power distribution family is selected ({\tt dfam=1}),
+matrix $Y$ must have 1~column that provides $y_i$ for each~$x_i$ in the corresponding
+row of matrix~$X$. When {\tt dfam=2} and $Y$ has 1~column, we assume the Bernoulli
+distribution for $y_i\in\{y_{\mathrm{neg}}, y_{\mathrm{pos}}\}$ with $y_{\mathrm{neg}}$
+from the input parameter {\tt yneg} and with $y_{\mathrm{pos}} \neq y_{\mathrm{neg}}$.
+When {\tt dfam=2} and $Y$ has 2~columns, we assume the
+binomial distribution; for each row~$i$ in~$X$, cells $Y[i, 1]$ and $Y[i, 2]$ provide
+the positive and the negative binomial counts respectively. Internally we convert
+the 1-column Bernoulli into the 2-column binomial with 0-versus-1 counts.
+
+We estimate the regression parameters via L2-regularized negative log-likelihood
+minimization:
+\begin{equation*}
+f(\beta; X, Y) \,\,=\,\, -\sum\nolimits_{i=1}^n \big(y_i\theta_i - b(\theta_i)\big)
+\,+\,(\lambda/2) \sum\nolimits_{j=1}^m \beta_j^2\,\,\to\,\,\min
+\end{equation*}
+where $\theta_i$ and $b(\theta_i)$ are from~(\ref{eqn:GLM}); note that $a$
+and $c(y, a)$ are constant w.r.t.~$\beta$ and can be ignored here.
+The canonical parameter $\theta_i$ depends on both $\beta$ and~$x_i$:
+\begin{equation*}
+\theta_i \,\,=\,\, b'^{\,-1}(\mu_i) \,\,=\,\, b'^{\,-1}\big(g^{-1}(\eta_i)\big) \,\,=\,\,
+\big(b'^{\,-1}\circ g^{-1}\big)\left(\beta_0 + \sum\nolimits_{j=1}^m \beta_j x_{i,j}\right)
+\end{equation*}
+The user-provided (via {\tt reg}) regularization coefficient $\lambda\geq 0$ can be used
+to mitigate overfitting and degeneracy in the data. Note that the intercept is never
+regularized.
+
+Our iterative minimizer for $f(\beta; X, Y)$ uses the Fisher scoring approximation
+to the difference $\varDelta f(z; \beta) = f(\beta + z; X, Y) \,-\, f(\beta; X, Y)$,
+recomputed at each iteration:
+\begin{gather*}
+\varDelta f(z; \beta) \,\,\,\approx\,\,\, 1/2 \cdot z^T A z \,+\, G^T z,
+\,\,\,\,\textrm{where}\,\,\,\, A \,=\, X^T\!\diag(w) X \,+\, \lambda I\\
+\textrm{and}\,\,\,\,G \,=\, - X^T u \,+\, \lambda\beta,
+\,\,\,\textrm{with $n\,{\times}\,1$ vectors $w$ and $u$ given by}\\
+\forall\,i = 1\ldots n: \,\,\,\,
+w_i = \big[v(\mu_i)\,g'(\mu_i)^2\big]^{-1}
+\!\!\!\!\!\!,\,\,\,\,\,\,\,\,\,
+u_i = (y_i - \mu_i)\big[v(\mu_i)\,g'(\mu_i)\big]^{-1}
+\!\!\!\!\!\!.\,\,\,\,
+\end{gather*}
+Here $v(\mu_i)=\Var(y_i)/a$, the variance of $y_i$ as the function of the mean, and
+$g'(\mu_i) = d \eta_i/d \mu_i$ is the link function derivative. The Fisher scoring
+approximation is minimized by trust-region conjugate gradient iterations (called the
+\emph{inner} iterations, with the Fisher scoring iterations as the \emph{outer}
+iterations), which approximately solve the following problem:
+\begin{equation*}
+1/2 \cdot z^T A z \,+\, G^T z \,\,\to\,\,\min\,\,\,\,\textrm{subject to}\,\,\,\,
+\|z\|_2 \leq \delta
+\end{equation*}
+The conjugate gradient algorithm closely follows Algorithm~7.2 on page~171
+of~\cite{Nocedal2006:Optimization}.
+The trust region size $\delta$ is initialized as $0.5\sqrt{m}\,/ \max\nolimits_i \|x_i\|_2$
+and updated as described in~\cite{Nocedal2006:Optimization}.
+The user can specify the maximum number of the outer and the inner iterations with
+input parameters {\tt moi} and {\tt mii}, respectively. The Fisher scoring algorithm
+terminates successfully if $2|\varDelta f(z; \beta)| < (D_1(\beta) + 0.1)\hspace{0.5pt}\eps$
+where $\eps > 0$ is a tolerance supplied by the user via {\tt tol}, and $D_1(\beta)$ is
+the unit-dispersion deviance estimated as
+\begin{equation*}
+D_1(\beta) \,\,=\,\, 2 \cdot \big(\Prob[Y \mid \!
+\begin{smallmatrix}\textrm{saturated}\\\textrm{model}\end{smallmatrix}, a\,{=}\,1]
+\,\,-\,\,\Prob[Y \mid X, \beta, a\,{=}\,1]\,\big)
+\end{equation*}
+The deviance estimate is also produced as part of the output. Once the Fisher scoring
+algorithm terminates, if requested by the user, we estimate the dispersion~$a$ from
+Eq.~\ref{eqn:GLM} using Pearson residuals
+\begin{equation}
+\hat{a} \,\,=\,\, \frac{1}{n-m}\cdot \sum_{i=1}^n \frac{(y_i - \mu_i)^2}{v(\mu_i)}
+\label{eqn:dispersion}
+\end{equation}
+and use it to adjust our deviance estimate: $D_{\hat{a}}(\beta) = D_1(\beta)/\hat{a}$.
+If input argument {\tt disp} is {\tt 0.0} we estimate $\hat{a}$, otherwise we use its
+value as~$a$. Note that in~(\ref{eqn:dispersion}) $m$~counts the intercept
+($m \leftarrow m+1$) if it is present.
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+The estimated regression parameters (the $\hat{\beta}_j$'s) are populated into
+a matrix and written to an HDFS file whose path/name was provided as the ``{\tt B}''
+input argument. What this matrix contains, and its size, depends on the input
+argument {\tt icpt}, which specifies the user's intercept and rescaling choice:
+\begin{Description}
+\item[{\tt icpt=0}:] No intercept, matrix~$B$ has size $m\,{\times}\,1$, with
+$B[j, 1] = \hat{\beta}_j$ for each $j$ from 1 to~$m = {}$ncol$(X)$.
+\item[{\tt icpt=1}:] There is intercept, but no shifting/rescaling of~$X$; matrix~$B$
+has size $(m\,{+}\,1) \times 1$, with $B[j, 1] = \hat{\beta}_j$ for $j$ from 1 to~$m$,
+and $B[m\,{+}\,1, 1] = \hat{\beta}_0$, the estimated intercept coefficient.
+\item[{\tt icpt=2}:] There is intercept, and the features in~$X$ are shifted to
+mean${} = 0$ and rescaled to variance${} = 1$; then there are two versions of
+the~$\hat{\beta}_j$'s, one for the original features and another for the
+shifted/rescaled features. Now matrix~$B$ has size $(m\,{+}\,1) \times 2$, with
+$B[\cdot, 1]$ for the original features and $B[\cdot, 2]$ for the shifted/rescaled
+features, in the above format. Note that $B[\cdot, 2]$ are iteratively estimated
+and $B[\cdot, 1]$ are obtained from $B[\cdot, 2]$ by complementary shifting and
+rescaling.
+\end{Description}
+Our script also estimates the dispersion $\hat{a}$ (or takes it from the user's input)
+and the deviances $D_1(\hat{\beta})$ and $D_{\hat{a}}(\hat{\beta})$, see
+Table~\ref{table:GLM:stats} for details. A log file with variables monitoring
+progress through the iterations can also be made available, see Table~\ref{table:GLM:log}.
+
+\smallskip
+\noindent{\bf Examples}
+\smallskip
+
+{\hangindent=\parindent\noindent\tt
+\hml -f GLM.dml -nvargs X=/user/biadmin/X.mtx Y=/user/biadmin/Y.mtx
+ B=/user/biadmin/B.mtx fmt=csv dfam=2 link=2 yneg=-1.0 icpt=2 reg=0.01 tol=0.00000001
+ disp=1.0 moi=100 mii=10 O=/user/biadmin/stats.csv Log=/user/biadmin/log.csv
+
+}
+
+\smallskip
+\noindent{\bf See Also}
+\smallskip
+
+In case of binary classification problems, consider using L2-SVM or binary logistic
+regression; for multiclass classification, use multiclass~SVM or multinomial logistic
+regression. For the special cases of linear regression and logistic regression, it
+may be more efficient to use the corresponding specialized scripts instead of~GLM.
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a891c0a1/Algorithms Reference/GLMpredict.tex
----------------------------------------------------------------------
diff --git a/Algorithms Reference/GLMpredict.tex b/Algorithms Reference/GLMpredict.tex
new file mode 100644
index 0000000..7fb91c8
--- /dev/null
+++ b/Algorithms Reference/GLMpredict.tex
@@ -0,0 +1,453 @@
+\subsection{Regression Scoring and Prediction}
+
+\noindent{\bf Description}
+\smallskip
+
+Script {\tt GLM-predict.dml} is intended to cover all linear model based regressions,
+including linear regression, binomial and multinomial logistic regression, and GLM
+regressions (Poisson, gamma, binomial with probit link~etc.). Having just one scoring
+script for all these regressions simplifies maintenance and enhancement while ensuring
+compatible interpretations for output statistics.
+
+The script performs two functions, prediction and scoring. To perform prediction,
+the script takes two matrix inputs: a collection of records $X$ (without the response
+attribute) and the estimated regression parameters~$B$, also known as~$\beta$.
+To perform scoring, in addition to $X$ and~$B$, the script takes the matrix of actual
+response values~$Y$ that are compared to the predictions made with $X$ and~$B$. Of course
+there are other, non-matrix, input arguments that specify the model and the output
+format, see below for the full list.
+
+We assume that our test/scoring dataset is given by $n\,{\times}\,m$-matrix $X$ of
+numerical feature vectors, where each row~$x_i$ represents one feature vector of one
+record; we have \mbox{$\dim x_i = m$}. Each record also includes the response
+variable~$y_i$ that may be numerical, single-label categorical, or multi-label categorical.
+A single-label categorical $y_i$ is an integer category label, one label per record;
+a multi-label $y_i$ is a vector of integer counts, one count for each possible label,
+which represents multiple single-label events (observations) for the same~$x_i$. Internally
+we convert single-label categoricals into multi-label categoricals by replacing each
+label~$l$ with an indicator vector~$(0,\ldots,0,1_l,0,\ldots,0)$. In prediction-only
+tasks the actual $y_i$'s are not needed to the script, but they are needed for scoring.
+
+To perform prediction, the script matrix-multiplies $X$ and $B$, adding the intercept
+if available, then applies the inverse of the model's link function.
+All GLMs assume that the linear combination of the features in~$x_i$ and the betas
+in~$B$ determines the means~$\mu_i$ of the~$y_i$'s (in numerical or multi-label
+categorical form) with $\dim \mu_i = \dim y_i$. The observed $y_i$ is assumed to follow
+a specified GLM family distribution $\Prob[y\mid \mu_i]$ with mean(s)~$\mu_i$:
+\begin{equation*}
+x_i \,\,\,\,\mapsto\,\,\,\, \eta_i = \beta_0 + \sum\nolimits_{j=1}^m \beta_j x_{i,j}
+\,\,\,\,\mapsto\,\,\,\, \mu_i \,\,\,\,\mapsto \,\,\,\, y_i \sim \Prob[y\mid \mu_i]
+\end{equation*}
+If $y_i$ is numerical, the predicted mean $\mu_i$ is a real number. Then our script's
+output matrix $M$ is the $n\,{\times}\,1$-vector of these means~$\mu_i$.
+Note that $\mu_i$ predicts the mean of $y_i$, not the actual~$y_i$. For example,
+in Poisson distribution, the mean is usually fractional, but the actual~$y_i$ is
+always integer.
+
+If $y_i$ is categorical, i.e.\ a vector of label counts for record~$i$, then $\mu_i$
+is a vector of non-negative real numbers, one number $\mu_{i,l}$ per each label~$l$.
+In this case we divide the $\mu_{i,l}$ by their sum $\sum_l \mu_{i,l}$ to obtain
+predicted label probabilities~\mbox{$p_{i,l}\in [0, 1]$}. The output matrix $M$ is
+the $n \times (k\,{+}\,1)$-matrix of these probabilities, where $n$ is the number of
+records and $k\,{+}\,1$ is the number of categories\footnote{We use $k+1$ because
+there are $k$ non-baseline categories and one baseline category, with regression
+parameters $B$ having $k$~columns.}. Note again that we do not predict the labels
+themselves, nor their actual counts per record, but we predict the labels' probabilities.
+
+Going from predicted probabilities to predicted labels, in the single-label categorical
+case, requires extra information such as the cost of false positive versus
+false negative errors. For example, if there are 5 categories and we \emph{accurately}
+predicted their probabilities as $(0.1, 0.3, 0.15, 0.2, 0.25)$, just picking the
+highest-probability label would be wrong 70\% of the time, whereas picking the
+lowest-probability label might be right if, say, it represents a diagnosis of cancer
+or another rare and serious outcome. Hence, we keep this step outside the scope of
+{\tt GLM-predict.dml} for now.
+
+\smallskip
+\noindent{\bf Usage}
+\smallskip
+
+{\hangindent=\parindent\noindent\it%
+{\tt{}-f }path/\/{\tt{}GLM-predict.dml}
+{\tt{} -nvargs}
+{\tt{} X=}path/file
+{\tt{} Y=}path/file
+{\tt{} B=}path/file
+{\tt{} M=}path/file
+{\tt{} O=}path/file
+{\tt{} dfam=}int
+{\tt{} vpow=}double
+{\tt{} link=}int
+{\tt{} lpow=}double
+{\tt{} disp=}double
+{\tt{} fmt=}format
+
+}
+
+\smallskip
+\noindent{\bf Arguments}
+\begin{Description}
+\item[{\tt X}:]
+Location (on HDFS) to read the $n\,{\times}\,m$-matrix $X$ of feature vectors, each row
+constitutes one feature vector (one record)
+\item[{\tt Y}:] (default:\mbox{ }{\tt " "})
+Location to read the response matrix $Y$ needed for scoring (but optional for prediction),
+with the following dimensions: \\
+ $n \:{\times}\: 1$: acceptable for all distributions ({\tt dfam=1} or {\tt 2} or {\tt 3}) \\
+ $n \:{\times}\: 2$: for binomial ({\tt dfam=2}) if given by (\#pos, \#neg) counts \\
+ $n \:{\times}\: k\,{+}\,1$: for multinomial ({\tt dfam=3}) if given by category counts
+\item[{\tt M}:] (default:\mbox{ }{\tt " "})
+Location to write, if requested, the matrix of predicted response means (for {\tt dfam=1}) or
+probabilities (for {\tt dfam=2} or {\tt 3}):\\
+ $n \:{\times}\: 1$: for power-type distributions ({\tt dfam=1}) \\
+ $n \:{\times}\: 2$: for binomial distribution ({\tt dfam=2}), col\#~2 is the ``No'' probability \\
+ $n \:{\times}\: k\,{+}\,1$: for multinomial logit ({\tt dfam=3}), col\#~$k\,{+}\,1$ is for the baseline
+\item[{\tt B}:]
+Location to read matrix $B$ of the \mbox{betas}, i.e.\ estimated GLM regression parameters,
+with the intercept at row\#~$m\,{+}\,1$ if available:\\
+ $\dim(B) \,=\, m \:{\times}\: k$: do not add intercept \\
+ $\dim(B) \,=\, m\,{+}\,1 \:{\times}\: k$: add intercept as given by the last $B$-row \\
+ if $k > 1$, use only $B${\tt [, 1]} unless it is Multinomial Logit ({\tt dfam=3})
+\item[{\tt O}:] (default:\mbox{ }{\tt " "})
+Location to store the CSV-file with goodness-of-fit statistics defined in
+Table~\ref{table:GLMpred:stats}, the default is to print them to the standard output
+\item[{\tt dfam}:] (default:\mbox{ }{\tt 1})
+GLM distribution family code to specify the type of distribution $\Prob[y\,|\,\mu]$
+that we assume: \\
+{\tt 1} = power distributions with $\Var(y) = \mu^{\alpha}$, see Table~\ref{table:commonGLMs};\\
+{\tt 2} = binomial;
+{\tt 3} = multinomial logit
+\item[{\tt vpow}:] (default:\mbox{ }{\tt 0.0})
+Power for variance defined as (mean)${}^{\textrm{power}}$ (ignored if {\tt dfam}$\,{\neq}\,1$):
+when {\tt dfam=1}, this provides the~$q$ in $\Var(y) = a\mu^q$, the power
+dependence of the variance of~$y$ on its mean. In particular, use:\\
+{\tt 0.0} = Gaussian,
+{\tt 1.0} = Poisson,
+{\tt 2.0} = Gamma,
+{\tt 3.0} = inverse Gaussian
+\item[{\tt link}:] (default:\mbox{ }{\tt 0})
+Link function code to determine the link function~$\eta = g(\mu)$, ignored for
+multinomial logit ({\tt dfam=3}):\\
+{\tt 0} = canonical link (depends on the distribution family), see Table~\ref{table:commonGLMs};\\
+{\tt 1} = power functions,
+{\tt 2} = logit,
+{\tt 3} = probit,
+{\tt 4} = cloglog,
+{\tt 5} = cauchit
+\item[{\tt lpow}:] (default:\mbox{ }{\tt 1.0})
+Power for link function defined as (mean)${}^{\textrm{power}}$ (ignored if {\tt link}$\,{\neq}\,1$):
+when {\tt link=1}, this provides the~$s$ in $\eta = \mu^s$, the power link
+function; {\tt lpow=0.0} gives the log link $\eta = \log\mu$. Common power links:\\
+{\tt -2.0} = $1/\mu^2$,
+{\tt -1.0} = reciprocal,
+{\tt 0.0} = log,
+{\tt 0.5} = sqrt,
+{\tt 1.0} = identity
+\item[{\tt disp}:] (default:\mbox{ }{\tt 1.0})
+Dispersion value, when available; must be positive
+\item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
+Matrix {\tt M} file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
+see read/write functions in SystemML Language Reference for details.
+\end{Description}
+
+
+\begin{table}[t]\small\centerline{%
+\begin{tabular}{|lccl|}
+\hline
+Name & \hspace{-0.6em}CID\hspace{-0.5em} & \hspace{-0.3em}Disp?\hspace{-0.6em} & Meaning \\
+\hline
+{\tt LOGLHOOD\_Z} & & + & Log-likelihood $Z$-score (in st.\ dev.'s from the mean) \\
+{\tt LOGLHOOD\_Z\_PVAL} & & + & Log-likelihood $Z$-score p-value, two-sided \\
+{\tt PEARSON\_X2} & & + & Pearson residual $X^2$-statistic \\
+{\tt PEARSON\_X2\_BY\_DF} & & + & Pearson $X^2$ divided by degrees of freedom \\
+{\tt PEARSON\_X2\_PVAL} & & + & Pearson $X^2$ p-value \\
+{\tt DEVIANCE\_G2} & & + & Deviance from the saturated model $G^2$-statistic \\
+{\tt DEVIANCE\_G2\_BY\_DF} & & + & Deviance $G^2$ divided by degrees of freedom \\
+{\tt DEVIANCE\_G2\_PVAL} & & + & Deviance $G^2$ p-value \\
+{\tt AVG\_TOT\_Y} & + & & $Y$-column average for an individual response value \\
+{\tt STDEV\_TOT\_Y} & + & & $Y$-column st.\ dev.\ for an individual response value \\
+{\tt AVG\_RES\_Y} & + & & $Y$-column residual average of $Y - \mathop{\mathrm{pred.}\,\mathrm{mean}}(Y|X)$ \\
+{\tt STDEV\_RES\_Y} & + & & $Y$-column residual st.\ dev.\ of $Y - \mathop{\mathrm{pred.}\,\mathrm{mean}}(Y|X)$ \\
+{\tt PRED\_STDEV\_RES} & + & + & Model-predicted $Y$-column residual st.\ deviation\\
+{\tt PLAIN\_R2} & + & & Plain $R^2$ of $Y$-column residual with bias included \\
+{\tt ADJUSTED\_R2} & + & & Adjusted $R^2$ of $Y$-column residual w.\ bias included \\
+{\tt PLAIN\_R2\_NOBIAS} & + & & Plain $R^2$ of $Y$-column residual, bias subtracted \\
+{\tt ADJUSTED\_R2\_NOBIAS} & + & & Adjusted $R^2$ of $Y$-column residual, bias subtracted \\
+\hline
+\end{tabular}}
+\caption{The above goodness-of-fit statistics are provided in CSV format, one per each line, with four
+columns: (Name, [CID], [Disp?], Value). The columns are:
+``Name'' is the string identifier for the statistic, see the table;
+``CID'' is an optional integer value that specifies the $Y$-column index for \mbox{per-}column statistics
+(note that a bi-/multinomial one-column {\tt Y}-input is converted into multi-column);
+``Disp?'' is an optional Boolean value ({\tt TRUE} or {\tt FALSE}) that tells us
+whether or not scaling by the input dispersion parameter {\tt disp} has been applied to this
+statistic;
+``Value'' is the value of the statistic.}
+\label{table:GLMpred:stats}
+\end{table}
+
+\noindent{\bf Details}
+\smallskip
+
+The output matrix $M$ of predicted means (or probabilities) is computed by matrix-multiplying $X$
+with the first column of~$B$ or with the whole~$B$ in the multinomial case, adding the intercept
+if available (conceptually, appending an extra column of ones to~$X$); then applying the inverse
+of the model's link function. The difference between ``means'' and ``probabilities'' in the
+categorical case becomes significant when there are ${\geq}\,2$ observations per record
+(with the multi-label records) or when the labels such as $-1$ and~$1$ are viewed and averaged
+as numerical response values (with the single-label records). To avoid any \mbox{mix-up} or
+information loss, we separately return the predicted probability of each category label for each
+record.
+
+When the ``actual'' response values $Y$ are available, the summary statistics are computed
+and written out as described in Table~\ref{table:GLMpred:stats}. Below we discuss each of
+these statistics in detail. Note that in the categorical case (binomial and multinomial)
+$Y$ is internally represented as the matrix of observation counts for each label in each record,
+rather than just the label~ID for each record. The input~$Y$ may already be a matrix of counts,
+in which case it is used as-is. But if $Y$ is given as a vector of response labels, each
+response label is converted into an indicator vector $(0,\ldots,0,1_l,0,\ldots,0)$ where~$l$
+is the label~ID for this record. All negative (e.g.~$-1$) or zero label~IDs are converted to
+the $1 + {}$maximum label~ID. The largest label~ID is viewed as the ``baseline'' as explained
+in the section on Multinomial Logistic Regression. We assume that there are $k\geq 1$
+non-baseline categories and one (last) baseline category.
+
+We also estimate residual variances for each response value, although we do not output them,
+but use them only inside the summary statistics, scaled and unscaled by the input dispersion
+parameter {\tt disp}, as described below.
+
+\smallskip
+{\tt LOGLHOOD\_Z} and {\tt LOGLHOOD\_Z\_PVAL} statistics measure how far the log-likelihood
+of~$Y$ deviates from its expected value according to the model. The script implements them
+only for the binomial and the multinomial distributions, returning NaN for all other distributions.
+Pearson's~$X^2$ and deviance~$G^2$ often perform poorly with bi- and multinomial distributions
+due to low cell counts, hence we need this extra goodness-of-fit measure. To compute these
+statistics, we use:
+\begin{Itemize}
+\item the $n\times (k\,{+}\,1)$-matrix~$Y$ of multi-label response counts, in which $y_{i,j}$
+is the number of times label~$j$ was observed in record~$i$;
+\item the model-estimated probability matrix~$P$ of the same dimensions that satisfies
+$\sum_{j=1}^{k+1} p_{i,j} = 1$ for all~$i=1,\ldots,n$ and where $p_{i,j}$ is the model
+probability of observing label~$j$ in record~$i$;
+\item the $n\,{\times}\,1$-vector $N$ where $N_i$ is the aggregated count of observations
+in record~$i$ (all $N_i = 1$ if each record has only one response label).
+\end{Itemize}
+We start by computing the multinomial log-likelihood of $Y$ given~$P$ and~$N$, as well as
+the expected log-likelihood given a random~$Y$ and the variance of this log-likelihood if
+$Y$ indeed follows the proposed distribution:
+\begin{align*}
+\ell (Y) \,\,&=\,\, \log \Prob[Y \,|\, P, N] \,\,=\,\, \sum_{i=1}^{n} \,\sum_{j=1}^{k+1} \,y_{i,j}\log p_{i,j} \\
+\E_Y \ell (Y) \,\,&=\,\, \sum_{i=1}^{n}\, \sum_{j=1}^{k+1} \,\mu_{i,j} \log p_{i,j}
+ \,\,=\,\, \sum_{i=1}^{n}\, N_i \,\sum_{j=1}^{k+1} \,p_{i,j} \log p_{i,j} \\
+\Var_Y \ell (Y) \,&=\, \sum_{i=1}^{n} \,N_i \left(\sum_{j=1}^{k+1} \,p_{i,j} \big(\log p_{i,j}\big)^2
+ - \Bigg( \sum_{j=1}^{k+1} \,p_{i,j} \log p_{i,j}\Bigg) ^ {\!\!2\,} \right)
+\end{align*}
+Then we compute the $Z$-score as the difference between the actual and the expected
+log-likelihood $\ell(Y)$ divided by its expected standard deviation, and its two-sided
+p-value in the Normal distribution assumption ($\ell(Y)$ should approach normality due
+to the Central Limit Theorem):
+\begin{equation*}
+Z \,=\, \frac {\ell(Y) - \E_Y \ell(Y)}{\sqrt{\Var_Y \ell(Y)}};\quad
+\mathop{\textrm{p-value}}(Z) \,=\, \Prob \Big[\,\big|\mathop{\textrm{Normal}}(0,1)\big| \, > \, |Z|\,\Big]
+\end{equation*}
+A low p-value would indicate ``underfitting'' if $Z\ll 0$ or ``overfitting'' if $Z\gg 0$. Here
+``overfitting'' means that higher-probability labels occur more often than their probabilities
+suggest.
+
+We also apply the dispersion input ({\tt disp}) to compute the ``scaled'' version of the $Z$-score
+and its p-value. Since $\ell(Y)$ is a linear function of~$Y$, multiplying the GLM-predicted
+variance of~$Y$ by {\tt disp} results in multiplying $\Var_Y \ell(Y)$ by the same {\tt disp}. This, in turn,
+translates into dividing the $Z$-score by the square root of the dispersion:
+\begin{equation*}
+Z_{\texttt{disp}} \,=\, \big(\ell(Y) \,-\, \E_Y \ell(Y)\big) \,\big/\, \sqrt{\texttt{disp}\cdot\Var_Y \ell(Y)}
+\,=\, Z / \sqrt{\texttt{disp}}
+\end{equation*}
+Finally, we recalculate the p-value with this new $Z$-score.
+
+\smallskip
+{\tt PEARSON\_X2}, {\tt PEARSON\_X2\_BY\_DF}, and {\tt PEARSON\_X2\_PVAL}:
+Pearson's residual $X^2$-statistic is a commonly used goodness-of-fit measure for linear models~\cite{McCullagh1989:GLM}.
+The idea is to measure how well the model-predicted means and variances match the actual behavior
+of response values. For each record $i$, we estimate the mean $\mu_i$ and the variance $v_i$
+(or $\texttt{disp}\cdot v_i$) and use them to normalize the residual:
+$r_i = (y_i - \mu_i) / \sqrt{v_i}$. These normalized residuals are then squared, aggregated
+by summation, and tested against an appropriate $\chi^2$~distribution. The computation of~$X^2$
+is slightly different for categorical data (bi- and multinomial) than it is for numerical data,
+since $y_i$ has multiple correlated dimensions~\cite{McCullagh1989:GLM}:
+\begin{equation*}
+X^2\,\textrm{(numer.)} \,=\, \sum_{i=1}^{n}\, \frac{(y_i - \mu_i)^2}{v_i};\quad
+X^2\,\textrm{(categ.)} \,=\, \sum_{i=1}^{n}\, \sum_{j=1}^{k+1} \,\frac{(y_{i,j} - N_i
+\hspace{0.5pt} p_{i,j})^2}{N_i \hspace{0.5pt} p_{i,j}}
+\end{equation*}
+The number of degrees of freedom~\#d.f.\ for the $\chi^2$~distribution is $n - m$ for numerical data and
+$(n - m)k$ for categorical data, where $k = \mathop{\texttt{ncol}}(Y) - 1$. Given the dispersion
+parameter {\tt disp}, the $X^2$ statistic is scaled by division: \mbox{$X^2_{\texttt{disp}} = X^2 / \texttt{disp}$}.
+If the dispersion is accurate, $X^2 / \texttt{disp}$ should be close to~\#d.f. In fact, $X^2 / \textrm{\#d.f.}$
+over the \emph{training} data is the dispersion estimator used in our {\tt GLM.dml} script,
+see~(\ref{eqn:dispersion}). Here we provide $X^2 / \textrm{\#d.f.}$ and $X^2_{\texttt{disp}} / \textrm{\#d.f.}$
+as {\tt PEARSON\_X2\_BY\_DF} to enable dispersion comparison between the training data and
+the test data.
+
+NOTE: For categorical data, both Pearson's $X^2$ and the deviance $G^2$ are unreliable (i.e.\ do not
+approach the $\chi^2$~distribution) unless the predicted means of multi-label counts
+$\mu_{i,j} = N_i \hspace{0.5pt} p_{i,j}$ are fairly large: all ${\geq}\,1$ and 80\% are
+at least~$5$~\cite{Cochran1954:chisq}. They should not be used for ``one label per record'' categoricals.
+
+\smallskip
+{\tt DEVIANCE\_G2}, {\tt DEVIANCE\_G2\_BY\_DF}, and {\tt DEVIANCE\_G2\_PVAL}:
+Deviance $G^2$ is the log of the likelihood ratio between the ``saturated'' model and the
+linear model being tested for the given dataset, multiplied by two:
+\begin{equation}
+G^2 \,=\, 2 \,\log \frac{\Prob[Y \mid \textrm{saturated model}\hspace{0.5pt}]}%
+{\Prob[Y \mid \textrm{tested linear model}\hspace{0.5pt}]}
+\label{eqn:GLMpred:deviance}
+\end{equation}
+The ``saturated'' model sets the mean $\mu_i^{\mathrm{sat}}$ to equal~$y_i$ for every record
+(for categorical data, $p^{\mathrm{sat}}_{i,j} = y_{i,j} / N_i$), which represents the ``perfect fit.''
+For records with $y_{i,j} \in \{0, N_i\}$ or otherwise at a boundary, by continuity we set
+$0 \log 0 = 0$. The GLM~likelihood functions defined in~(\ref{eqn:GLM}) become simplified
+in ratio~(\ref{eqn:GLMpred:deviance}) due to canceling out the term $c(y, a)$ since it is the same
+in both models.
+
+The log of a likelihood ratio between two nested models, times two, is known to approach
+a $\chi^2$ distribution as $n\to\infty$ if both models have fixed parameter spaces.
+But this is not the case for the ``saturated'' model: it adds more parameters with each record.
+In practice, however, $\chi^2$ distributions are used to compute the p-value of~$G^2$~\cite{McCullagh1989:GLM}.
+The number of degrees of freedom~\#d.f.\ and the treatment of dispersion are the same as for
+Pearson's~$X^2$, see above.
+
+\Paragraph{Column-wise statistics.} The rest of the statistics are computed separately
+for each column of~$Y$. As explained above, $Y$~has two or more columns in bi- and multinomial case,
+either at input or after conversion. Moreover, each $y_{i,j}$ in record~$i$ with $N_i \geq 2$ is
+counted as $N_i$ separate observations $y_{i,j,l}$ of 0 or~1 (where $l=1,\ldots,N_i$) with
+$y_{i,j}$~ones and $N_i-y_{i,j}$ zeros.
+For power distributions, including linear regression, $Y$~has only one column and all
+$N_i = 1$, so the statistics are computed for all~$Y$ with each record counted once.
+Below we denote $N = \sum_{i=1}^n N_i \,\geq n$.
+Here is the total average and the residual average (residual bias) of~$y_{i,j,l}$ for each $Y$-column:
+\begin{equation*}
+\texttt{AVG\_TOT\_Y}_j \,=\, \frac{1}{N} \sum_{i=1}^n y_{i,j}; \quad
+\texttt{AVG\_RES\_Y}_j \,=\, \frac{1}{N} \sum_{i=1}^n \, (y_{i,j} - \mu_{i,j})
+\end{equation*}
+Dividing by~$N$ (rather than~$n$) gives the averages for~$y_{i,j,l}$ (rather than~$y_{i,j}$).
+The total variance, and the standard deviation, for individual observations~$y_{i,j,l}$ is
+estimated from the total variance for response values~$y_{i,j}$ using independence assumption:
+$\Var y_{i,j} = \Var \sum_{l=1}^{N_i} y_{i,j,l} = \sum_{l=1}^{N_i} \Var y_{i,j,l}$.
+This allows us to estimate the sum of squares for~$y_{i,j,l}$ via the sum of squares for~$y_{i,j}$:
+\begin{equation*}
+\texttt{STDEV\_TOT\_Y}_j \,=\,
+\Bigg[\frac{1}{N-1} \sum_{i=1}^n \Big( y_{i,j} - \frac{N_i}{N} \sum_{i'=1}^n y_{i'\!,j}\Big)^2\Bigg]^{1/2}
+\end{equation*}
+Analogously, we estimate the standard deviation of the residual $y_{i,j,l} - \mu_{i,j,l}$:
+\begin{equation*}
+\texttt{STDEV\_RES\_Y}_j \,=\,
+\Bigg[\frac{1}{N-m'} \,\sum_{i=1}^n \Big( y_{i,j} - \mu_{i,j} - \frac{N_i}{N} \sum_{i'=1}^n (y_{i'\!,j} - \mu_{i'\!,j})\Big)^2\Bigg]^{1/2}
+\end{equation*}
+Here $m'=m$ if $m$ includes the intercept as a feature and $m'=m+1$ if it does not.
+The estimated standard deviations can be compared to the model-predicted residual standard deviation
+computed from the predicted means by the GLM variance formula and scaled by the dispersion:
+\begin{equation*}
+\texttt{PRED\_STDEV\_RES}_j \,=\, \Big[\frac{\texttt{disp}}{N} \, \sum_{i=1}^n \, v(\mu_{i,j})\Big]^{1/2}
+\end{equation*}
+We also compute the $R^2$ statistics for each column of~$Y$, see Table~\ref{table:GLMpred:R2} for details.
+We compute two versions of~$R^2$: in one version the residual sum-of-squares (RSS) includes any bias in
+the residual that might be present (due to the lack of, or inaccuracy in, the intercept); in the other
+version of~RSS the bias is subtracted by ``centering'' the residual. In both cases we subtract the bias in the total
+sum-of-squares (in the denominator), and $m'$ equals $m$~with the intercept or $m+1$ without the intercept.
+
+\begin{table}[t]\small\centerline{%
+\begin{tabular}{|c|c|}
+\multicolumn{2}{c}{$R^2$ where the residual sum-of-squares includes the bias contribution:} \\
+\hline
+\multicolumn{1}{|l|}{\tt PLAIN\_R2${}_j \,\,= {}$} & \multicolumn{1}{l|}{\tt ADJUSTED\_R2${}_j \,\,= {}$} \\
+$ \displaystyle 1 -
+\frac{\sum\limits_{i=1}^n \,(y_{i,j} - \mu_{i,j})^2}%
+{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}}\!\! \sum\limits_{i'=1}^n \! y_{i'\!,j} \Big)^{\! 2}} $ &
+$ \displaystyle 1 - {\textstyle\frac{N_{\mathstrut} - 1}{N^{\mathstrut} - m}} \,
+\frac{\sum\limits_{i=1}^n \,(y_{i,j} - \mu_{i,j})^2}%
+{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}}\!\! \sum\limits_{i'=1}^n \! y_{i'\!,j} \Big)^{\! 2}} $ \\
+\hline
+\multicolumn{2}{c}{} \\
+\multicolumn{2}{c}{$R^2$ where the residual sum-of-squares is centered so that the bias is subtracted:} \\
+\hline
+\multicolumn{1}{|l|}{\tt PLAIN\_R2\_NOBIAS${}_j \,\,= {}$} & \multicolumn{1}{l|}{\tt ADJUSTED\_R2\_NOBIAS${}_j \,\,= {}$} \\
+$ \displaystyle 1 -
+\frac{\sum\limits_{i=1}^n \Big(y_{i,j} \,{-}\, \mu_{i,j} \,{-}\, \frac{N_{i\mathstrut}}{N^{\mathstrut}}\!\!
+ \sum\limits_{i'=1}^n (y_{i'\!,j} \,{-}\, \mu_{i'\!,j}) \Big)^{\! 2}}%
+{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}}\!\! \sum\limits_{i'=1}^n \! y_{i'\!,j} \Big)^{\! 2}} $ &
+$ \displaystyle 1 - {\textstyle\frac{N_{\mathstrut} - 1}{N^{\mathstrut} - m'}} \,
+\frac{\sum\limits_{i=1}^n \Big(y_{i,j} \,{-}\, \mu_{i,j} \,{-}\, \frac{N_{i\mathstrut}}{N^{\mathstrut}}\!\!
+ \sum\limits_{i'=1}^n (y_{i'\!,j} \,{-}\, \mu_{i'\!,j}) \Big)^{\! 2}}%
+{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}}\!\! \sum\limits_{i'=1}^n \! y_{i'\!,j} \Big)^{\! 2}} $ \\
+\hline
+\end{tabular}}
+\caption{The $R^2$ statistics we compute in {\tt GLM-predict.dml}}
+\label{table:GLMpred:R2}
+\end{table}
+
+\smallskip
+\noindent{\bf Returns}
+\smallskip
+
+The matrix of predicted means (if the response is numerical) or probabilities (if the response
+is categorical), see ``Description'' subsection above for more information. Given {\tt Y}, we
+return some statistics in CSV format as described in Table~\ref{table:GLMpred:stats} and in the
+above text.
+
+\smallskip
+\noindent{\bf Examples}
+\smallskip
+
+Note that in the examples below the value for ``{\tt disp}'' input argument
+is set arbitrarily. The correct dispersion value should be computed from the training
+data during model estimation, or omitted if unknown (which sets it to~{\tt 1.0}).
+
+\smallskip\noindent
+Linear regression example:
+\par\hangindent=\parindent\noindent{\tt
+\hml -f GLM-predict.dml -nvargs dfam=1 vpow=0.0 link=1 lpow=1.0 disp=5.67
+ X=/user/biadmin/X.mtx B=/user/biadmin/B.mtx M=/user/biadmin/Means.mtx fmt=csv
+ Y=/user/biadmin/Y.mtx O=/user/biadmin/stats.csv
+
+}\smallskip\noindent
+Linear regression example, prediction only (no {\tt Y} given):
+\par\hangindent=\parindent\noindent{\tt
+\hml -f GLM-predict.dml -nvargs dfam=1 vpow=0.0 link=1 lpow=1.0
+ X=/user/biadmin/X.mtx B=/user/biadmin/B.mtx M=/user/biadmin/Means.mtx fmt=csv
+
+}\smallskip\noindent
+Binomial logistic regression example:
+\par\hangindent=\parindent\noindent{\tt
+\hml -f GLM-predict.dml -nvargs dfam=2 link=2 disp=3.0004464
+ X=/user/biadmin/X.mtx B=/user/biadmin/B.mtx M=/user/biadmin/Probabilities.mtx fmt=csv
+ Y=/user/biadmin/Y.mtx O=/user/biadmin/stats.csv
+
+}\smallskip\noindent
+Binomial probit regression example:
+\par\hangindent=\parindent\noindent{\tt
+\hml -f GLM-predict.dml -nvargs dfam=2 link=3 disp=3.0004464
+ X=/user/biadmin/X.mtx B=/user/biadmin/B.mtx M=/user/biadmin/Probabilities.mtx fmt=csv
+ Y=/user/biadmin/Y.mtx O=/user/biadmin/stats.csv
+
+}\smallskip\noindent
+Multinomial logistic regression example:
+\par\hangindent=\parindent\noindent{\tt
+\hml -f GLM-predict.dml -nvargs dfam=3
+ X=/user/biadmin/X.mtx B=/user/biadmin/B.mtx M=/user/biadmin/Probabilities.mtx fmt=csv
+ Y=/user/biadmin/Y.mtx O=/user/biadmin/stats.csv
+
+}\smallskip\noindent
+Poisson regression with the log link example:
+\par\hangindent=\parindent\noindent{\tt
+\hml -f GLM-predict.dml -nvargs dfam=1 vpow=1.0 link=1 lpow=0.0 disp=3.45
+ X=/user/biadmin/X.mtx B=/user/biadmin/B.mtx M=/user/biadmin/Means.mtx fmt=csv
+ Y=/user/biadmin/Y.mtx O=/user/biadmin/stats.csv
+
+}\smallskip\noindent
+Gamma regression with the inverse (reciprocal) link example:
+\par\hangindent=\parindent\noindent{\tt
+\hml -f GLM-predict.dml -nvargs dfam=1 vpow=2.0 link=1 lpow=-1.0 disp=1.99118
+ X=/user/biadmin/X.mtx B=/user/biadmin/B.mtx M=/user/biadmin/Means.mtx fmt=csv
+ Y=/user/biadmin/Y.mtx O=/user/biadmin/stats.csv
+
+}
[32/47] incubator-systemml git commit: Performance wdivmm minus
patterns (nz-indicator awareness), for als_cg
Posted by du...@apache.org.
Performance wdivmm minus patterns (nz-indicator awareness), for als_cg
In the initial version of wdivmm -minus patterns we supported the
general case of arbitrary weights which are not necessarily non-zero
indicators. Especially, for the spark/mr backends this generality led to
a severe performance bottleneck due to repeated shuffling of X and W.
This change restricts the wdivmm minus patterns to weights of non-zero
indicators which allows data-local map-side computation.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/62b9b9cb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/62b9b9cb
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/62b9b9cb
Branch: refs/heads/gh-pages
Commit: 62b9b9cb1898addf565b69c9c0e0264ecb95d3c8
Parents: e9c8583
Author: Matthias Boehm <mb...@us.ibm.com>
Authored: Sun Sep 27 01:34:10 2015 -0700
Committer: Matthias Boehm <mb...@us.ibm.com>
Committed: Sun Sep 27 10:51:17 2015 -0700
----------------------------------------------------------------------
devdocs/MatrixMultiplicationOperators.txt | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/62b9b9cb/devdocs/MatrixMultiplicationOperators.txt
----------------------------------------------------------------------
diff --git a/devdocs/MatrixMultiplicationOperators.txt b/devdocs/MatrixMultiplicationOperators.txt
index a7c2d25..02668a3 100644
--- a/devdocs/MatrixMultiplicationOperators.txt
+++ b/devdocs/MatrixMultiplicationOperators.txt
@@ -106,8 +106,8 @@ C) CORE MATRIX MULT PRIMITIVES LibMatrixMult (incl related script patterns)
* 7) wdivmm ((a) t(t(U)%*%(W/(U%*%t(V)))), (b) (W/(U%*%t(V)))%*%V,
(c) t(t(U)%*%(W*(U%*%t(V)))), (d) (W*(U%*%t(V)))%*%V,
- (e) W*(U%*%t(V)), (f) t(t(U)%*%(W*(U%*%t(V)-X))),
- (g) (W*(U%*%t(V)-X)%*%V)
+ (e) W*(U%*%t(V)), (f) t(t(U)%*%((X!=0)*(U%*%t(V)-X))),
+ (g) ((X!=0)*(U%*%t(V)-X)%*%V)
- sequential / multi-threaded (same block ops, par over rows in X)
- all dense, sparse-dense factors, sparse/dense-* x 7 patterns
[43/47] incubator-systemml git commit: Added Download SystemML
section and explained standalone node in the Quick Start Guide
Posted by du...@apache.org.
Added Download SystemML section and explained standalone node in the Quick Start Guide
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/ad66f730
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/ad66f730
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/ad66f730
Branch: refs/heads/gh-pages
Commit: ad66f730efe1fd2d910052a8505982df0cdaba59
Parents: 56c997a
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Sat Oct 31 23:44:08 2015 -0300
Committer: Luciano Resende <lr...@apache.org>
Committed: Sat Oct 31 23:44:08 2015 -0300
----------------------------------------------------------------------
quick-start-guide.md | 34 +++++++++++++++++++++++-----------
1 file changed, 23 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/ad66f730/quick-start-guide.md
----------------------------------------------------------------------
diff --git a/quick-start-guide.md b/quick-start-guide.md
index 5080a49..b01aea1 100644
--- a/quick-start-guide.md
+++ b/quick-start-guide.md
@@ -15,35 +15,47 @@ This tutorial provides a quick introduction to using SystemML by
running existing SystemML algorithms in standalone mode. More information
about running SystemML in distributed execution mode (Hadoop, Spark) will
be added soon.
-For more in depth information, please refer to the
+For more in-depth information, please refer to the
[Algorithms Reference](algorithms-reference.html) and the
[DML Language Reference](dml-language-reference.html).
-<br/>
-
# What is SystemML
SystemML enables large-scale machine learning (ML) via a high-level declarative
language with R-like syntax called [DML](dml-language-reference.html).
-This language allows Data Scientists to
+This language allows data scientists to
express their ML algorithms with full flexibility but without the need to fine-tune
distributed runtime execution plans and system configurations.
These ML programs are dynamically compiled and optimized based on data
-and cluster characteristics using rule and cost-based optimization techniques.
+and cluster characteristics using rule- and cost-based optimization techniques.
The compiler automatically generates hybrid runtime execution plans ranging
from in-memory, single node execution to distributed computation for Hadoop M/R
or Spark Batch execution.
SystemML features a suite of algorithms for Descriptive Statistics, Classification,
-Clustering, Regression, and Matrix Factorization. Detailed descriptions of these
+Clustering, Regression, Matrix Factorization, and Survival Analysis. Detailed descriptions of these
algorithms can be found in the [Algorithms Reference](algorithms-reference.html).
<br/>
-# Distributed vs Standalone Execution Mode
+# Download SystemML
+
+Binary releases of SystemML are available for download at
+[https://github.com/SparkTC/systemml/releases](https://github.com/SparkTC/systemml/releases).
+
+The SystemML project is available on GitHub at [https://github.com/SparkTC/systemml](https://github.com/SparkTC/systemml).
+SystemML can be downloaded from GitHub and built with Maven. Instructions to build and
+test SystemML can be found in the GitHub README file.
+
+<br/>
+
+# Standalone vs Distributed Execution Mode
+
+SystemML's standalone mode is designed to allow data scientists to rapidly prototype algorithms
+on a single machine. The standalone release packages all required libraries into a single distribution file.
+In standalone mode, all operations occur on a single node in a non-Hadoop environment. Standalone mode
+is not appropriate for large datasets.
-SystemML can operate in standalone mode, allowing data
-scientists to quikly develop algorithms on a single machine.
For large-scale production environments, SystemML algorithm execution can be
distributed across a multi-node cluster using [Apache Hadoop](https://hadoop.apache.org/)
or [Apache Spark](http://spark.apache.org/).
@@ -248,7 +260,7 @@ We will create the file `perc.csv` and `perc.csv.mtd` to define the sampling vec
$ printf "0.5\n0.5" > data/perc.csv
$ echo '{"rows": 2, "cols": 1, "format": "csv"}' > data/perc.csv.mtd
-Let's run the sampling algortihm to create the two data samples:
+Let's run the sampling algorithm to create the two data samples:
$ ./runStandaloneSystemML.sh algorithms/utils/sample.dml -nvargs X=data/haberman.data sv=data/perc.csv O=data/haberman.part ofmt="csv"
@@ -256,7 +268,7 @@ Let's run the sampling algortihm to create the two data samples:
## Splitting Labels from Features
-Next we use the `splitXY.dml` algortithm to separate the feature columns from
+Next we use the `splitXY.dml` algorithm to separate the feature columns from
the label column(s).
Parameters:
[26/47] incubator-systemml git commit: [SYSML-301] Update Algorithm
Ref MathJax to render on GitHub
Posted by du...@apache.org.
[SYSML-301] Update Algorithm Ref MathJax to render on GitHub
Fix MD syntax to properly render as GitHub pages
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/b966a815
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/b966a815
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/b966a815
Branch: refs/heads/gh-pages
Commit: b966a815c45754f3f637edd8e1f32856ef1b270c
Parents: 8fd0f74
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Fri Sep 11 15:46:37 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Fri Sep 11 15:46:37 2015 -0700
----------------------------------------------------------------------
_layouts/global.html | 3 +-
algorithms-classification.md | 16 +++++-----
algorithms-descriptive-statistics.md | 52 +++++++++++++++----------------
algorithms-regression.md | 44 +++++++++++++-------------
index.md | 4 +--
5 files changed, 60 insertions(+), 59 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b966a815/_layouts/global.html
----------------------------------------------------------------------
diff --git a/_layouts/global.html b/_layouts/global.html
index 2a75531..7b2deea 100644
--- a/_layouts/global.html
+++ b/_layouts/global.html
@@ -25,7 +25,7 @@
<div class="navbar navbar-fixed-top" id="topbar">
<div class="navbar-inner">
<div class="container">
- <div class="brand" style="padding: 15px 0px; font-size: 20px; font-style: italic; font-weight: bold;"><a href="index.html">SystemML - {{site.SYSTEMML_VERSION}}</a>
+ <div class="brand" style="padding: 15px 0px; font-size: 20px; font-style: italic; font-weight: bold;"><a href="index.html">SystemML {{site.SYSTEMML_VERSION}}</a>
</div>
<ul class="nav">
<li><a href="index.html">Home</a></li>
@@ -36,6 +36,7 @@
<li><a href="http://www.github.com/SparkTC/systemml">SystemML GitHub README</a></li>
<li><a href="quick-start-guide.html">Quick Start Guide</a></li>
+ <!-- <li><a href="programming-guide.html">Programming Guide</a></li> -->
<li><a href="algorithms-reference.html">Algorithms Reference</a></li>
<li><a href="dml-language-reference.html">DML Language Reference</a></li>
<li class="divider"></li>
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b966a815/algorithms-classification.md
----------------------------------------------------------------------
diff --git a/algorithms-classification.md b/algorithms-classification.md
index da46ded..8a16cd0 100644
--- a/algorithms-classification.md
+++ b/algorithms-classification.md
@@ -33,7 +33,7 @@ Just as linear regression estimates the mean value $\mu_i$ of a
numerical response variable, logistic regression does the same for
category label probabilities. In linear regression, the mean of $y_i$ is
estimated as a linear combination of the features:
-$\mu_i = \beta_0 + \beta_1 x_{i,1} + \ldots + \beta_m x_{i,m} = \beta_0 + x_i\beta_{1:m}$.
+$$\mu_i = \beta_0 + \beta_1 x_{i,1} + \ldots + \beta_m x_{i,m} = \beta_0 + x_i\beta_{1:m}$$.
In logistic regression, the label probability has to lie between 0
and 1, so a link function is applied to connect it to
$\beta_0 + x_i\beta_{1:m}$. If there are just two possible category
@@ -46,10 +46,10 @@ Prob[y_i\,{=}\,0\mid x_i; \beta] \,=\,
\frac{1}{1 + e^{\,\beta_0 + x_i\beta_{1:m}}}$$
Here category label 0
-serves as the *baseline*, and function $\exp(\beta_0 + x_i\beta_{1:m})$
+serves as the *baseline*, and function $$\exp(\beta_0 + x_i\beta_{1:m})$$
shows how likely we expect to see “$y_i = 1$” in comparison to the
baseline. Like in a loaded coin, the predicted odds of seeing 1 versus 0
-are $\exp(\beta_0 + x_i\beta_{1:m})$ to 1, with each feature $x_{i,j}$
+are $$\exp(\beta_0 + x_i\beta_{1:m})$$ to 1, with each feature $$x_{i,j}$$
multiplying its own factor $\exp(\beta_j x_{i,j})$ to the odds. Given a
large collection of pairs $(x_i, y_i)$, $i=1\ldots n$, logistic
regression seeks to find the $\beta_j$’s that maximize the product of
@@ -63,11 +63,11 @@ $k \geq 3$ possible categories. Again we identify one category as the
baseline, for example the $k$-th category. Instead of a coin, here we
have a loaded multisided die, one side per category. Each non-baseline
category $l = 1\ldots k\,{-}\,1$ has its own vector
-$(\beta_{0,l}, \beta_{1,l}, \ldots, \beta_{m,l})$ of regression
+$$(\beta_{0,l}, \beta_{1,l}, \ldots, \beta_{m,l})$$ of regression
parameters with the intercept, making up a matrix $B$ of size
$(m\,{+}\,1)\times(k\,{-}\,1)$. The predicted odds of seeing
non-baseline category $l$ versus the baseline $k$ are
-$\exp\big(\beta_{0,l} + \sum\nolimits_{j=1}^m x_{i,j}\beta_{j,l}\big)$
+$$\exp\big(\beta_{0,l} + \sum\nolimits_{j=1}^m x_{i,j}\beta_{j,l}\big)$$
to 1, and the predicted probabilities are:
$$
@@ -101,7 +101,7 @@ $$
The optional regularization term is added to
mitigate overfitting and degeneracy in the data; to reduce bias, the
-intercepts $\beta_{0,l}$ are not regularized. Once the $\beta_{j,l}$’s
+intercepts $$\beta_{0,l}$$ are not regularized. Once the $\beta_{j,l}$’s
are accurately estimated, we can make predictions about the category
label $y$ for a new feature vector $x$ using
Eqs. (1) and (2).
@@ -137,7 +137,7 @@ represent the (same) baseline category and are converted to label
$\max(\texttt{Y})\,{+}\,1$.
**B**: Location to store the matrix of estimated regression parameters (the
-$\beta_{j, l}$’s), with the intercept parameters $\beta_{0, l}$ at
+$$\beta_{j, l}$$’s), with the intercept parameters $\beta_{0, l}$ at
position B\[$m\,{+}\,1$, $l$\] if available.
The size of B is $(m\,{+}\,1)\times (k\,{-}\,1)$ with the
intercepts or $m \times (k\,{-}\,1)$ without the intercepts, one column
@@ -221,7 +221,7 @@ Newton method for logistic regression described in [[Lin2008]](algorithms-biblio
For convenience, let us make some changes in notation:
* Convert the input vector of observed category labels into an indicator
-matrix $Y$ of size $n \times k$ such that $Y_{i, l} = 1$ if the $i$-th
+matrix $Y$ of size $n \times k$ such that $$Y_{i, l} = 1$$ if the $i$-th
category label is $l$ and $Y_{i, l} = 0$ otherwise.
* Append an extra column of all ones, i.e. $(1, 1, \ldots, 1)^T$, as the
$m\,{+}\,1$-st column to the feature matrix $X$ to represent the
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b966a815/algorithms-descriptive-statistics.md
----------------------------------------------------------------------
diff --git a/algorithms-descriptive-statistics.md b/algorithms-descriptive-statistics.md
index dd276af..6c56344 100644
--- a/algorithms-descriptive-statistics.md
+++ b/algorithms-descriptive-statistics.md
@@ -184,7 +184,7 @@ order, preserving duplicates: $v^s_1 \leq v^s_2 \leq \ldots \leq v^s_n$.
**Figure 1**: The computation of quartiles, median, and interquartile mean from the
empirical distribution function of the 10-point
sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8}. Each vertical step in
-the graph has height $1{/}n = 0.1$. Values $q_{25\%}$, $q_{50\%}$, and $q_{75\%}$ denote
+the graph has height $1{/}n = 0.1$. Values $$q_{25\%}$$, $$q_{50\%}$$, and $$q_{75\%}$$ denote
the $1^{\textrm{st}}$, $2^{\textrm{nd}}$, and $3^{\textrm{rd}}$ quartiles correspondingly;
value $\mu$ denotes the median. Values $\phi_1$ and $\phi_2$ show the partial contribution
of border points (quartiles) $v_3=3.7$ and $v_8=6.4$ into the interquartile mean.
@@ -214,7 +214,7 @@ median, we sort the sample in the increasing order, preserving
duplicates: $v^s_1 \leq v^s_2 \leq \ldots \leq v^s_n$. If $n$ is odd,
the median equals $v^s_i$ where $i = (n\,{+}\,1)\,{/}\,2$, same as the
$50^{\textrm{th}}$ percentile of the sample. If $n$ is even, there are
-two “middle” values $v^s_{n/2}$ and $v^s_{n/2\,+\,1}$, so we compute the
+two “middle” values $$v^s_{n/2}$$ and $$v^s_{n/2\,+\,1}$$, so we compute the
median as the mean of these two values. (For even $n$ we compute the
$50^{\textrm{th}}$ percentile as $v^s_{n/2}$, not as the median.)
Example: the median of sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1,
@@ -269,7 +269,7 @@ quantitative (scale) data feature.
around their mean, expressed in units that are the square of those of
the feature itself. Computed as the sum of squared differences between
the values in the sample and their mean, divided by one less than the
-number of values: $\sum_{i=1}^n (v_i - \bar{v})^2\,/\,(n\,{-}\,1)$ where
+number of values: $$\sum_{i=1}^n (v_i - \bar{v})^2\,/\,(n\,{-}\,1)$$ where
$\bar{v}=\left(\sum_{i=1}^n v_i\right)/n$. Example: the variance of
sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8}
equals 3.24. Note that at least two values ($n\geq 2$) are required to
@@ -357,8 +357,8 @@ Skewness is computed as the $3^{\textrm{rd}}$ central moment divided by
the cube of the standard deviation. We estimate the
$3^{\textrm{rd}}$ central moment as the sum of cubed differences between
the values in the feature column and their sample mean, divided by the
-number of values: $\sum_{i=1}^n (v_i - \bar{v})^3 / n$ where
-$\bar{v}=\left(\sum_{i=1}^n v_i\right)/n$. The standard deviation is
+number of values: $$\sum_{i=1}^n (v_i - \bar{v})^3 / n$$ where
+$$\bar{v}=\left(\sum_{i=1}^n v_i\right)/n$$. The standard deviation is
computed as described above in *standard deviation*. To avoid division
by 0, at least two different sample values are required. Example: for
sample {2.2, 3.2, 3.7, 4.4, 5.3, 5.7, 6.1, 6.4, 7.2, 7.8} with the
@@ -411,7 +411,7 @@ divided by the $4^{\textrm{th}}$ power of the standard deviation,
minus 3. We estimate the $4^{\textrm{th}}$ central moment as the sum of
the $4^{\textrm{th}}$ powers of differences between the values in the
feature column and their sample mean, divided by the number of values:
-$\sum_{i=1}^n (v_i - \bar{v})^4 / n$ where
+$$\sum_{i=1}^n (v_i - \bar{v})^4 / n$$ where
$\bar{v}=\left(\sum_{i=1}^n v_i\right)/n$. The standard deviation is
computed as described above, see *standard deviation*.
@@ -634,7 +634,7 @@ Below we list all bivariate statistics computed by script
`bivar-stats.dml`. The statistics are collected into
several groups by the type of their input features. We refer to the two
input features as $v_1$ and $v_2$ unless specified otherwise; the value
-pairs are $(v_{1,i}, v_{2,i})$ for $i=1,\ldots,n$, where $n$ is the
+pairs are $$(v_{1,i}, v_{2,i})$$ for $i=1,\ldots,n$, where $n$ is the
number of rows in `X`, i.e. the sample size.
@@ -653,7 +653,7 @@ $$r
$$
Commonly denoted by $r$, correlation ranges between $-1$ and $+1$,
-reaching ${\pm}1$ when all value pairs $(v_{1,i}, v_{2,i})$ lie on the
+reaching ${\pm}1$ when all value pairs $$(v_{1,i}, v_{2,i})$$ lie on the
same line. Correlation near 0 means that a line is not a good way to
represent the dependence between the two features; however, this does
not imply independence. The sign indicates direction of the linear
@@ -665,9 +665,9 @@ not change if we transform $v_1$ and $v_2$ to $a + b v_1$ and
$c + d v_2$ where $a, b, c, d$ are constants and $b, d > 0$.
Suppose that we use simple linear regression to represent one feature
-given the other, say represent $v_{2,i} \approx \alpha + \beta v_{1,i}$
+given the other, say represent $$v_{2,i} \approx \alpha + \beta v_{1,i}$$
by selecting $\alpha$ and $\beta$ to minimize the least-squares error
-$\sum_{i=1}^n (v_{2,i} - \alpha - \beta v_{1,i})^2$. Then the best error
+$$\sum_{i=1}^n (v_{2,i} - \alpha - \beta v_{1,i})^2$$. Then the best error
equals
$$\min_{\alpha, \beta} \,\,\sum_{i=1}^n \big(v_{2,i} - \alpha - \beta v_{1,i}\big)^2 \,\,=\,\,
@@ -694,7 +694,7 @@ But we do not know these (hypothesized) probabilities; we only know the
sample frequency counts. Let $n_{a,b}$ be the frequency count of pair
$(a, b)$, let $n_a$ and $n_b$ be the frequency counts of $a$ alone and
of $b$ alone. Under independence, difference
-$n_{a,b}{/}n - (n_a{/}n)(n_b{/}n)$ is unlikely to be exactly 0 due to
+$$n_{a,b}{/}n - (n_a{/}n)(n_b{/}n)$$ is unlikely to be exactly 0 due to
sample randomness, yet it is unlikely to be too far from 0. For some
pairs $(a,b)$ it may deviate from 0 farther than for other pairs.
Pearson’s $\chi^2$ is an aggregate measure that combines
@@ -703,7 +703,7 @@ squares of these differences across all value pairs:
$$\chi^2 \,\,=\,\, \sum_{a,\,b} \Big(\frac{n_a n_b}{n}\Big)^{-1} \Big(n_{a,b} - \frac{n_a n_b}{n}\Big)^2
\,=\,\, \sum_{a,\,b} \frac{(O_{a,b} - E_{a,b})^2}{E_{a,b}}$$
-where $O_{a,b} = n_{a,b}$ are the *observed* frequencies and
+where $$O_{a,b} = n_{a,b}$$ are the *observed* frequencies and
$E_{a,b} = (n_a n_b){/}n$ are the *expected* frequencies for all
pairs $(a,b)$. Under independence (plus other standard assumptions) the
sample $\chi^2$ closely follows a well-known distribution, making it a
@@ -802,10 +802,10 @@ $$\eta^2 \,=\, 1 - \frac{\sum_{i=1}^{n} \big(y_i - \hat{y}[x_i]\big)^2}{\sum_{i=
\hat{y}[x] = \frac{1}{\mathop{\mathrm{freq}}(x)}\sum_{i=1}^n
\,\left\{\!\!\begin{array}{rl} y_i & \textrm{if $x_i = x$}\\ 0 & \textrm{otherwise}\end{array}\right.\!\!\!$$
-and $\bar{y} = (1{/}n)\sum_{i=1}^n y_i$ is the mean. Value $\hat{y}[x]$
+and $$\bar{y} = (1{/}n)\sum_{i=1}^n y_i$$ is the mean. Value $\hat{y}[x]$
is the average of $y_i$ among all records where $x_i = x$; it can also
be viewed as the “predictor” of $y$ given $x$. Then
-$\sum_{i=1}^{n} (y_i - \hat{y}[x_i])^2$ is the residual error
+$$\sum_{i=1}^{n} (y_i - \hat{y}[x_i])^2$$ is the residual error
sum-of-squares and $\sum_{i=1}^{n} (y_i - \bar{y})^2$ is the total
sum-of-squares for $y$. Hence, $\eta^2$ measures the accuracy of
predicting $y$ with $x$, just like the “R-squared” statistic measures
@@ -887,10 +887,10 @@ coefficient is geared towards features having small value domains
and large counts for the values. Given the two input vectors, we form a
contingency table $T$ of pairwise frequency counts, as well as a vector
of frequency counts for each feature: $f_1$ and $f_2$. Here in
-$T_{i,j}$, $f_{1,i}$, $f_{2,j}$ indices $i$ and $j$ refer to the
+$$T_{i,j}$$, $$f_{1,i}$$, $$f_{2,j}$$ indices $i$ and $j$ refer to the
order-preserving integer encoding of the feature values. We use prefix
sums over $f_1$ and $f_2$ to compute the values’ average ranks:
-$r_{1,i} = \sum_{j=1}^{i-1} f_{1,j} + (f_{1,i}\,{+}\,1){/}2$, and
+$$r_{1,i} = \sum_{j=1}^{i-1} f_{1,j} + (f_{1,i}\,{+}\,1){/}2$$, and
analogously for $r_2$. Finally, we compute rank variances for $r_1, r_2$
weighted by counts $f_1, f_2$ and their covariance weighted by $T$,
before applying the standard formula for Pearson’s correlation
@@ -899,7 +899,7 @@ coefficient:
$$\rho \,\,=\,\, \frac{Cov_T(r_1, r_2)}{\sqrt{Var_{f_1}(r_1)Var_{f_2}(r_2)}}
\,\,=\,\, \frac{\sum_{i,j} T_{i,j} (r_{1,i} - \bar{r}_1) (r_{2,j} - \bar{r}_2)}{\sqrt{\sum_i f_{1,i} (r_{1,i} - \bar{r}_1)^{2\mathstrut} \cdot \sum_j f_{2,j} (r_{2,j} - \bar{r}_2)^{2\mathstrut}}}$$
-where $\bar{r_1} = \sum_i r_{1,i} f_{1,i}{/}n$, analogously
+where $$\bar{r_1} = \sum_i r_{1,i} f_{1,i}{/}n$$, analogously
for $\bar{r}_2$. The value of $\rho$ lies between $-1$ and $+1$, with
sign indicating the prevalent direction of the association: $\rho > 0$
($\rho < 0$) means that one feature tends to increase (decrease) when
@@ -1226,9 +1226,9 @@ $$y_{i,j} \,=\, \alpha_i + \beta x_{i,j} + {\varepsilon}_{i,j}\,, \quad\textrm{w
Here $i = 1\ldots k$ is a stratum number and
$j = 1\ldots n_i$ is a record number in stratum $i$; by $n_i$ we denote
the number of records available in stratum $i$. The noise
-term $\varepsilon_{i,j}$ is assumed to have the same variance in all
-strata. When $n_i\,{>}\,0$, we can estimate the means of $x_{i, j}$ and
-$y_{i, j}$ in stratum $i$ as
+term $$\varepsilon_{i,j}$$ is assumed to have the same variance in all
+strata. When $n_i\,{>}\,0$, we can estimate the means of $$x_{i, j}$$ and
+$$y_{i, j}$$ in stratum $i$ as
$$\bar{x}_i \,= \Big(\sum\nolimits_{j=1}^{n_i} \,x_{i, j}\Big) / n_i\,;\quad
\bar{y}_i \,= \Big(\sum\nolimits_{j=1}^{n_i} \,y_{i, j}\Big) / n_i$$
@@ -1259,8 +1259,8 @@ estimates for $Var(X)$ and
$Var(Y)$ tend to be smaller
than the non-stratified ones (with the global mean instead of
$\bar{x_i}$ and $\bar{y_i}$) since $\bar{x_i}$ and $\bar{y_i}$ fit
-closer to $x_{i,j}$ and $y_{i,j}$ than the global means. The stratified
-variance estimates the uncertainty in $x_{i,j}$ and $y_{i,j}$ given
+closer to $$x_{i,j}$$ and $$y_{i,j}$$ than the global means. The stratified
+variance estimates the uncertainty in $$x_{i,j}$$ and $$y_{i,j}$$ given
their stratum $i$.
Minimizing over $\beta$ the error sum-of-squares
@@ -1274,13 +1274,13 @@ $$\mathrm{RSS} \,\,=\, \,
\,\,=\,\, V_y \,\big(1 \,-\, V_{x,y}^2 / (V_x V_y)\big)$$
The quantity
-$\hat{R}^2 = V_{x,y}^2 / (V_x V_y)$, called *$R$-squared*, estimates the
-fraction of stratified variance in $y_{i,j}$ explained by covariate
-$x_{i, j}$ in the linear regression model. We
+$$\hat{R}^2 = V_{x,y}^2 / (V_x V_y)$$, called *$R$-squared*, estimates the
+fraction of stratified variance in $$y_{i,j}$$ explained by covariate
+$$x_{i, j}$$ in the linear regression model. We
define *stratified correlation* as the square root of $\hat{R}^2$ taken
with the sign of $V_{x,y}$. We also use RSS to estimate the residual
standard deviation $\sigma$ in the linear regression model that models the
-prediction error of $y_{i,j}$ given $x_{i,j}$ and the stratum:
+prediction error of $$y_{i,j}$$ given $$x_{i,j}$$ and the stratum:
$$\hat{\beta}\, =\, \frac{V_{x,y}}{V_x}; \,\,\,\, \hat{R} \,=\, \frac{V_{x,y}}{\sqrt{V_x V_y}};
\,\,\,\, \hat{R}^2 \,=\, \frac{V_{x,y}^2}{V_x V_y};
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b966a815/algorithms-regression.md
----------------------------------------------------------------------
diff --git a/algorithms-regression.md b/algorithms-regression.md
index be39a27..0302a18 100644
--- a/algorithms-regression.md
+++ b/algorithms-regression.md
@@ -618,7 +618,7 @@ binomial distributions. Here $\mu$ is the Bernoulli mean.
| Name | Link Function |
| --------------------- | ------------- |
| Logit | $\displaystyle \eta = 1 / \big(1 + e^{-\mu}\big)^{\mathstrut}$
-| Probit | $\displaystyle \mu = \frac{1}{\sqrt{2\pi}}\int\nolimits_{-\infty_{\mathstrut}}^{\,\eta\mathstrut} e^{-\frac{t^2}{2}} dt$
+| Probit | $$\displaystyle \mu = \frac{1}{\sqrt{2\pi}}\int\nolimits_{-\infty_{\mathstrut}}^{\,\eta\mathstrut} e^{-\frac{t^2}{2}} dt$$
| Cloglog | $\displaystyle \eta = \log \big(- \log(1 - \mu)\big)^{\mathstrut}$
| Cauchit | $\displaystyle \eta = \tan\pi(\mu - 1/2)$
@@ -688,7 +688,7 @@ matrix $Y$ having 1 or 2 columns. If a power distribution family is
selected (`dfam=1`), matrix $Y$ must have 1 column that
provides $y_i$ for each $x_i$ in the corresponding row of matrix $X$.
When dfam=2 and $Y$ has 1 column, we assume the Bernoulli
-distribution for $y_i\in\{y_{\mathrm{neg}}, 1\}$ with $y_{\mathrm{neg}}$
+distribution for $$y_i\in\{y_{\mathrm{neg}}, 1\}$$ with $y_{\mathrm{neg}}$
from the input parameter `yneg`. When `dfam=2` and
$Y$ has 2 columns, we assume the binomial distribution; for each row $i$
in $X$, cells $Y[i, 1]$ and $Y[i, 2]$ provide the positive and the
@@ -872,7 +872,7 @@ fractional, but the actual $y_i$ is always integer.
If $y_i$ is categorical, i.e. a vector of label counts for record $i$,
then $\mu_i$ is a vector of non-negative real numbers, one number
-$\mu_{i,l}$ per each label $l$. In this case we divide the $\mu_{i,l}$
+$$\mu_{i,l}$$ per each label $l$. In this case we divide the $$\mu_{i,l}$$
by their sum $\sum_l \mu_{i,l}$ to obtain predicted label
probabilities . The output matrix $M$ is the
$n \times (k\,{+}\,1)$-matrix of these probabilities, where $n$ is the
@@ -1185,7 +1185,7 @@ extra goodness-of-fit measure. To compute these statistics, we use:
which $y_{i,j}$ is the number of times label $j$ was observed in
record $i$
* the model-estimated probability matrix $P$ of the same dimensions that
-satisfies $\sum_{j=1}^{k+1} p_{i,j} = 1$ for all $i=1,\ldots,n$ and
+satisfies $$\sum_{j=1}^{k+1} p_{i,j} = 1$$ for all $i=1,\ldots,n$ and
where $p_{i,j}$ is the model probability of observing label $j$ in
record $i$
* the $n\,{\times}\,1$-vector $N$ where $N_i$ is the aggregated count of
@@ -1259,7 +1259,7 @@ The number of
degrees of freedom \#d.f. for the $\chi^2$ distribution is $n - m$ for
numerical data and $(n - m)k$ for categorical data, where
$k = \mathop{\texttt{ncol}}(Y) - 1$. Given the dispersion parameter
-`disp` the $X^2$ statistic is scaled by division: $X^2_{\texttt{disp}} = X^2 / \texttt{disp}$. If the
+`disp` the $X^2$ statistic is scaled by division: $$X^2_{\texttt{disp}} = X^2 / \texttt{disp}$$. If the
dispersion is accurate, $X^2 / \texttt{disp}$ should be close to \#d.f.
In fact, $X^2 / \textrm{\#d.f.}$ over the *training* data is the
dispersion estimator used in our `GLM.dml` script,
@@ -1271,7 +1271,7 @@ the training data and the test data.
NOTE: For categorical data, both Pearson’s $X^2$ and the deviance $G^2$
are unreliable (i.e. do not approach the $\chi^2$ distribution) unless
the predicted means of multi-label counts
-$\mu_{i,j} = N_i \hspace{0.5pt} p_{i,j}$ are fairly large: all
+$$\mu_{i,j} = N_i \hspace{0.5pt} p_{i,j}$$ are fairly large: all
${\geq}\,1$ and 80% are at least $5$ [[Cochran1954]](algorithms-bibliography.html). They should not
be used for “one label per record” categoricals.
@@ -1288,7 +1288,7 @@ $$
The “saturated” model sets the mean
$\mu_i^{\mathrm{sat}}$ to equal $y_i$ for every record (for categorical
-data, $p_{i,j}^{sat} = y_{i,j} / N_i$), which represents the
+data, $$p_{i,j}^{sat} = y_{i,j} / N_i$$), which represents the
“perfect fit.” For records with $y_{i,j} \in \{0, N_i\}$ or otherwise at
a boundary, by continuity we set $0 \log 0 = 0$. The GLM likelihood
functions defined in (5) become simplified in
@@ -1310,31 +1310,31 @@ Pearson’s $X^2$, see above.
The rest of the statistics are computed separately for each column
of $Y$. As explained above, $Y$ has two or more columns in bi- and
multinomial case, either at input or after conversion. Moreover, each
-$y_{i,j}$ in record $i$ with $N_i \geq 2$ is counted as $N_i$ separate
-observations $y_{i,j,l}$ of 0 or 1 (where $l=1,\ldots,N_i$) with
-$y_{i,j}$ ones and $N_i-y_{i,j}$ zeros. For power distributions,
+$$y_{i,j}$$ in record $i$ with $N_i \geq 2$ is counted as $N_i$ separate
+observations $$y_{i,j,l}$$ of 0 or 1 (where $l=1,\ldots,N_i$) with
+$$y_{i,j}$$ ones and $$N_i-y_{i,j}$$ zeros. For power distributions,
including linear regression, $Y$ has only one column and all $N_i = 1$,
so the statistics are computed for all $Y$ with each record counted
-once. Below we denote $N = \sum_{i=1}^n N_i \,\geq n$. Here is the total
+once. Below we denote $$N = \sum_{i=1}^n N_i \,\geq n$$. Here is the total
average and the residual average (residual bias) of $y_{i,j,l}$ for each
$Y$-column:
$$\texttt{AVG_TOT_Y}_j \,=\, \frac{1}{N} \sum_{i=1}^n y_{i,j}; \quad
\texttt{AVG_RES_Y}_j \,=\, \frac{1}{N} \sum_{i=1}^n \, (y_{i,j} - \mu_{i,j})$$
-Dividing by $N$ (rather than $n$) gives the averages for $y_{i,j,l}$
-(rather than $y_{i,j}$). The total variance, and the standard deviation,
-for individual observations $y_{i,j,l}$ is estimated from the total
-variance for response values $y_{i,j}$ using independence assumption:
-$Var \,y_{i,j} = Var \sum_{l=1}^{N_i} y_{i,j,l} = \sum_{l=1}^{N_i} Var y_{i,j,l}$.
+Dividing by $N$ (rather than $n$) gives the averages for $$y_{i,j,l}$$
+(rather than $$y_{i,j}$$). The total variance, and the standard deviation,
+for individual observations $$y_{i,j,l}$$ is estimated from the total
+variance for response values $$y_{i,j}$$ using independence assumption:
+$$Var \,y_{i,j} = Var \sum_{l=1}^{N_i} y_{i,j,l} = \sum_{l=1}^{N_i} Var y_{i,j,l}$$.
This allows us to estimate the sum of squares for $y_{i,j,l}$ via the
-sum of squares for $y_{i,j}$:
+sum of squares for $$y_{i,j}$$:
$$\texttt{STDEV_TOT_Y}_j \,=\,
\Bigg[\frac{1}{N-1} \sum_{i=1}^n \Big( y_{i,j} - \frac{N_i}{N} \sum_{i'=1}^n y_{i'\!,j}\Big)^2\Bigg]^{1/2}$$
Analogously, we estimate the standard deviation of the residual
-$y_{i,j,l} - \mu_{i,j,l}$:
+$$y_{i,j,l} - \mu_{i,j,l}$$:
$$\texttt{STDEV_RES_Y}_j \,=\,
\Bigg[\frac{1}{N-m'} \,\sum_{i=1}^n \Big( y_{i,j} - \mu_{i,j} - \frac{N_i}{N} \sum_{i'=1}^n (y_{i'\!,j} - \mu_{i'\!,j})\Big)^2\Bigg]^{1/2}$$
@@ -1363,8 +1363,8 @@ $m$ with the intercept or $m+1$ without the intercept.
| Statistic | Formula |
| --------------------- | ------------- |
-| $\texttt{PLAIN_R2}_j$ | $ \displaystyle 1 - \frac{\sum\limits_{i=1}^n \,(y_{i,j} - \mu_{i,j})^2}{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n y_{i',j} \Big)^{2}} $
-| $\texttt{ADJUSTED_R2}_j$ | $ \displaystyle 1 - {\textstyle\frac{N_{\mathstrut} - 1}{N^{\mathstrut} - m}} \, \frac{\sum\limits_{i=1}^n \,(y_{i,j} - \mu_{i,j})^2}{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n y_{i',j} \Big)^{2}} $
+| $\texttt{PLAIN_R2}_j$ | $$ \displaystyle 1 - \frac{\sum\limits_{i=1}^n \,(y_{i,j} - \mu_{i,j})^2}{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n y_{i',j} \Big)^{2}} $$
+| $\texttt{ADJUSTED_R2}_j$ | $$ \displaystyle 1 - {\textstyle\frac{N_{\mathstrut} - 1}{N^{\mathstrut} - m}} \, \frac{\sum\limits_{i=1}^n \,(y_{i,j} - \mu_{i,j})^2}{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n y_{i',j} \Big)^{2}} $$
* * *
@@ -1374,8 +1374,8 @@ $m$ with the intercept or $m+1$ without the intercept.
| Statistic | Formula |
| --------------------- | ------------- |
-| $\texttt{PLAIN_R2_NOBIAS}_j$ | $ \displaystyle 1 - \frac{\sum\limits_{i=1}^n \Big(y_{i,j} \,{-}\, \mu_{i,j} \,{-}\, \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n (y_{i',j} \,{-}\, \mu_{i',j}) \Big)^{2}}{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n y_{i',j} \Big)^{2}} $
-| $\texttt{ADJUSTED_R2_NOBIAS}_j$ | $ \displaystyle 1 - {\textstyle\frac{N_{\mathstrut} - 1}{N^{\mathstrut} - m'}} \, \frac{\sum\limits_{i=1}^n \Big(y_{i,j} \,{-}\, \mu_{i,j} \,{-}\, \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n (y_{i',j} \,{-}\, \mu_{i',j}) \Big)^{2}}{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n y_{i',j} \Big)^{2}} $
+| $\texttt{PLAIN_R2_NOBIAS}_j$ | $$ \displaystyle 1 - \frac{\sum\limits_{i=1}^n \Big(y_{i,j} \,{-}\, \mu_{i,j} \,{-}\, \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n (y_{i',j} \,{-}\, \mu_{i',j}) \Big)^{2}}{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n y_{i',j} \Big)^{2}} $$
+| $\texttt{ADJUSTED_R2_NOBIAS}_j$ | $$ \displaystyle 1 - {\textstyle\frac{N_{\mathstrut} - 1}{N^{\mathstrut} - m'}} \, \frac{\sum\limits_{i=1}^n \Big(y_{i,j} \,{-}\, \mu_{i,j} \,{-}\, \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n (y_{i',j} \,{-}\, \mu_{i',j}) \Big)^{2}}{\sum\limits_{i=1}^n \Big(y_{i,j} - \frac{N_{i\mathstrut}}{N^{\mathstrut}} \sum\limits_{i'=1}^n y_{i',j} \Big)^{2}} $$
* * *
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b966a815/index.md
----------------------------------------------------------------------
diff --git a/index.md b/index.md
index 49f69d8..70efd1d 100644
--- a/index.md
+++ b/index.md
@@ -5,18 +5,18 @@ title: SystemML Overview
description: SystemML documentation homepage
---
-SystemML is a flexible, scalable machine learning (ML) library written in Java.
+SystemML is a flexible, scalable machine learning (ML) language written in Java.
SystemML's distinguishing characteristics are: (1) algorithm customizability,
(2) multiple execution modes, including Standalone, Hadoop Batch, and Spark Batch,
and (3) automatic optimization.
-
## SystemML Documentation
For more information about SystemML, please consult the following references:
* [SystemML GitHub README](http://www.github.com/SparkTC/systemml)
* [Quick Start Guide](quick-start-guide.html)
+<!-- * [Programming Guide](programming-guide.html) -->
* [Algorithms Reference](algorithms-reference.html)
* [DML (Declarative Machine Learning) Language Reference](dml-language-reference.html)
* PYDML (Python-Like Declarative Machine Learning) Language Reference - **Coming Soon**
[47/47] incubator-systemml git commit: New wumm quaternary op
(rewrite, cp/mr/sp compiler/runtime, tests, docs)
Posted by du...@apache.org.
New wumm quaternary op (rewrite, cp/mr/sp compiler/runtime, tests, docs)
This change adds a new quaternary operation 'wumm' for the pattern
X*uop(L%*%t(R)), where uop is an arbitrary unary operator (with few
exceptions) or matrix-scalar/scalar-matrix operation that is internally
mapped to an unary operator (e.g., X^2, 2*X).
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/cbc96022
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/cbc96022
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/cbc96022
Branch: refs/heads/gh-pages
Commit: cbc960226ba0c4fedf760e2d1ed1fdc207fa8017
Parents: c8cc6eb
Author: Matthias Boehm <mb...@us.ibm.com>
Authored: Sat Nov 21 22:00:15 2015 -0800
Committer: Matthias Boehm <mb...@us.ibm.com>
Committed: Sun Nov 22 19:38:28 2015 -0800
----------------------------------------------------------------------
devdocs/MatrixMultiplicationOperators.txt | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/cbc96022/devdocs/MatrixMultiplicationOperators.txt
----------------------------------------------------------------------
diff --git a/devdocs/MatrixMultiplicationOperators.txt b/devdocs/MatrixMultiplicationOperators.txt
index 54cb5ec..7bc8a9c 100644
--- a/devdocs/MatrixMultiplicationOperators.txt
+++ b/devdocs/MatrixMultiplicationOperators.txt
@@ -1,6 +1,6 @@
#####################################################################
# TITLE: An Overview of Matrix Multiplication Operators in SystemML #
-# DATE MODIFIED: 09/26/2015 #
+# DATE MODIFIED: 11/21/2015 #
#####################################################################
In the following, we give an overview of backend-specific physical matrix multiplication operators in SystemML as well as their internally used matrix multiplication block operations.
@@ -48,6 +48,7 @@ A QuaternaryOp hop can be compiled into the following physical operators. Note t
- WSigmoid (weighted sigmoid) --> wsigmoid
- WDivMM (weighted divide matrix multiplication) --> wdivmm
- WCeMM (weighted cross entropy matrix multiplication) --> wcemm
+ - WuMM (weighted unary op matrix multiplication) --> wumm
* 2) Physical Operator in MR (distributed, mapreduce)
- MapWSLoss (map-side weighted squared loss) --> wsloss
@@ -58,6 +59,8 @@ A QuaternaryOp hop can be compiled into the following physical operators. Note t
- RedWDivMM (reduce-side weighted divide matrix mult) --> wdivmm
- MapWCeMM (map-side weighted cross entr. matrix mult) --> wcemm
- RedWCeMM (reduce-side w. cross entr. matrix mult) --> wcemm
+ - MapWuMM (map-side weighted unary op matrix mult) --> wumm
+ - RedWuMM (reduce-side weighted unary op matrix mult) --> wumm
* 3) Physical Operators in SPARK (distributed, spark)
- MapWSLoss (see MR, mappartitions + reduce) --> wsloss
@@ -70,8 +73,11 @@ A QuaternaryOp hop can be compiled into the following physical operators. Note t
- RedWDivMM (see MR, 1/2x flatmaptopair + 1/2x join + --> wdivmm
maptopair + reducebykey)
- MapWCeMM (see MR, mappartitions + reduce) --> wcemm
- - RedWDivMM (see MR, 1/2x flatmaptopair + 1/2x join + --> wcemm
+ - RedWCeMM (see MR, 1/2x flatmaptopair + 1/2x join + --> wcemm
maptopair + reduce)
+ - MapWuMM (see MR, mappartitions) --> wumm
+ - RedWuMM (see MR, 1/2x flatmaptopair + --> wumm
+ 1/2x join + maptopair)
C) CORE MATRIX MULT PRIMITIVES LibMatrixMult (incl related script patterns)
@@ -112,9 +118,11 @@ C) CORE MATRIX MULT PRIMITIVES LibMatrixMult (incl related script patterns)
- sequential / multi-threaded (same block ops, par over rows in X)
- all dense, sparse-dense factors, sparse/dense-* x 7 patterns
-* 8) wcemm (sum(X*log(U%*%t(V))))
+* 8) wcemm (sum(X*log(U%*%t(V))))
- sequential / multi-threaded (same block ops, par over rows in X)
- all dense, sparse-dense factors, sparse/dense-*, 1 pattern
-
-
\ No newline at end of file
+* 9) wumm ((a) X*uop(U%*%t(V)), (b) X/uop(U%*%t(V)))
+ - any unary operator, e.g., X*exp(U%*%t(V)) or X*(U%*%t(V))^2
+ - sequential / multi-threaded (same block ops, par over rows in X)
+ - all dense, sparse-dense factors, sparse/dense-*, 2 pattern
[31/47] incubator-systemml git commit: Generalization wdivmm
compiler/runtime (basic/-X patterns), for als_cg
Posted by du...@apache.org.
Generalization wdivmm compiler/runtime (basic/-X patterns), for als_cg
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/e9c85835
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/e9c85835
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/e9c85835
Branch: refs/heads/gh-pages
Commit: e9c858350fa2197899232eef77cd4969e7ba73ed
Parents: ec7b262
Author: Matthias Boehm <mb...@us.ibm.com>
Authored: Sat Sep 26 15:35:22 2015 -0700
Committer: Matthias Boehm <mb...@us.ibm.com>
Committed: Sat Sep 26 15:35:22 2015 -0700
----------------------------------------------------------------------
devdocs/MatrixMultiplicationOperators.txt | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/e9c85835/devdocs/MatrixMultiplicationOperators.txt
----------------------------------------------------------------------
diff --git a/devdocs/MatrixMultiplicationOperators.txt b/devdocs/MatrixMultiplicationOperators.txt
index 2fc831f..a7c2d25 100644
--- a/devdocs/MatrixMultiplicationOperators.txt
+++ b/devdocs/MatrixMultiplicationOperators.txt
@@ -1,6 +1,6 @@
#####################################################################
# TITLE: An Overview of Matrix Multiplication Operators in SystemML #
-# DATE MODIFIED: 09/25/2015 #
+# DATE MODIFIED: 09/26/2015 #
#####################################################################
In the following, we give an overview of backend-specific physical matrix multiplication operators in SystemML as well as their internally used matrix multiplication block operations.
@@ -104,10 +104,12 @@ C) CORE MATRIX MULT PRIMITIVES LibMatrixMult (incl related script patterns)
- sequential / multi-threaded (same block ops, par over rows in W)
- all dense, sparse-dense factors, sparse/dense-* x 4 patterns
-* 7) wdivmm ((a) t(t(U)%*%(X/(U%*%t(V)))), (b) (X/(U%*%t(V)))%*%V),
- (c) t(t(U)%*%(X*(U%*%t(V)))), (d) (X*(U%*%t(V)))%*%V)
+* 7) wdivmm ((a) t(t(U)%*%(W/(U%*%t(V)))), (b) (W/(U%*%t(V)))%*%V,
+ (c) t(t(U)%*%(W*(U%*%t(V)))), (d) (W*(U%*%t(V)))%*%V,
+ (e) W*(U%*%t(V)), (f) t(t(U)%*%(W*(U%*%t(V)-X))),
+ (g) (W*(U%*%t(V)-X)%*%V)
- sequential / multi-threaded (same block ops, par over rows in X)
- - all dense, sparse-dense factors, sparse/dense-* x 4 patterns
+ - all dense, sparse-dense factors, sparse/dense-* x 7 patterns
* 8) wcemm (sum(X*log(U%*%t(V))))
- sequential / multi-threaded (same block ops, par over rows in X)
[28/47] incubator-systemml git commit: Update
dml-language-reference.md
Posted by du...@apache.org.
Update dml-language-reference.md
Added "seed" description to rand().
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/0ad76f6c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/0ad76f6c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/0ad76f6c
Branch: refs/heads/gh-pages
Commit: 0ad76f6c050cd0a24b903e9bbd9c1acdacc52bb7
Parents: 442e83a
Author: Shirish Tatikonda <st...@us.ibm.com>
Authored: Wed Sep 23 12:32:35 2015 -0700
Committer: Shirish Tatikonda <st...@us.ibm.com>
Committed: Wed Sep 23 12:32:35 2015 -0700
----------------------------------------------------------------------
dml-language-reference.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/0ad76f6c/dml-language-reference.md
----------------------------------------------------------------------
diff --git a/dml-language-reference.md b/dml-language-reference.md
index c553a58..de73ab2 100644
--- a/dml-language-reference.md
+++ b/dml-language-reference.md
@@ -534,7 +534,7 @@ min() <br/> max() | Return the minimum/maximum cell value in matrix | Input: mat
min() <br/> max() | Return the minimum/maximum cell values of two matrices, matrix and scalar, or scalar value of two scalars. | Input: matrices or scalars <br/> Output: matrix or scalar | With x,y, z as scalars, and X, Y, Z as matrices: <br/> Z = min (X, Y) <br/> Z = min (X, y) <br/> z = min(x,y)
nrow(), <br/> ncol(), <br/> length() | Return the number of rows, number of columns, or number of cells in matrix respectively. | Input: matrix <br/> Output: scalar | nrow(X)
prod() | Return the product of all cells in matrix | Input: matrix <br/> Output: scalarj | prod(X)
-rand() | Generates a random matrix | Input: (rows=<value>, cols=<value>, min=<value>, max=<value>, sparsity=<value>, pdf=<string>) <br/> rows/cols: Number of rows/cols (expression) <br/> min/max: Min/max value for cells (either constant value, or variable that evaluates to constant value) <br/> sparsity: fraction of non-zero cells (constant value) <br/> pdf: “uniform” (min, max) distribution, or “normal” (0,1) distribution; or “poisson” (lambda=1) distribution. string; default value is “uniform”. Note that, for the Poisson distribution, users can provide the mean/lambda parameter as follows: <br/> rand(rows=1000,cols=1000, pdf=”poisson”, lambda=2.5). <br/> The default value for lambda is 1. <br/> Output: matrix | X = rand(rows=10, cols=20, min=0, max=1, pdf="uniform", sparsity=0.2) <br/> The example generates a 10 x 20 matrix, with cell values uniformly chosen at random between 0 and 1, and approximately 20% of cells will have
non-zero values.
+rand() | Generates a random matrix | Input: (rows=<value>, cols=<value>, min=<value>, max=<value>, sparsity=<value>, pdf=<string>, seed=<value>) <br/> rows/cols: Number of rows/cols (expression) <br/> min/max: Min/max value for cells (either constant value, or variable that evaluates to constant value) <br/> sparsity: fraction of non-zero cells (constant value) <br/> pdf: “uniform” (min, max) distribution, or “normal” (0,1) distribution; or “poisson” (lambda=1) distribution. string; default value is “uniform”. Note that, for the Poisson distribution, users can provide the mean/lambda parameter as follows: <br/> rand(rows=1000,cols=1000, pdf=”poisson”, lambda=2.5). <br/> The default value for lambda is 1. <br/> seed: Every invocation of rand() internally generates a random seed with which the cell values are generated. One can optionally provide a seed when repeatability is desired. <br/> Output: matrix | X = rand(rows=1
0, cols=20, min=0, max=1, pdf="uniform", sparsity=0.2) <br/> The example generates a 10 x 20 matrix, with cell values uniformly chosen at random between 0 and 1, and approximately 20% of cells will have non-zero values.
removeEmpty() | Removes all empty rows or columns from the input matrix target X according to the specified margin. | Input : (target= X <matrix>, margin="...") <br/> Output : <matrix> <br/> Valid values for margin are "rows" or "cols". | A = removeEmpty(target=X, margin="rows")
replace() | Creates a copy of input matrix X, where all values that are equal to the scalar pattern s1 are replaced with the scalar replacement s2. | Input : (target= X <matrix>, pattern=<scalar>, replacement=<scalar>) <br/> Output : <matrix> <br/> If s1 is NaN, then all NaN values of X are treated as equal and hence replaced with s2. Positive and negative infinity are treated as different values. | A = replace(target=X, pattern=s1, replacement=s2)
seq() | Creates a single column vector with values starting from <from>, to <to>, in increments of <increment> | Input: (<from>, <to>, <increment>) <br/> Output: <matrix> | S = seq (10, 200, 10)
[06/47] incubator-systemml git commit: Sparsity-aware spark/mr
mapmm/mapmmchain/pmm compilation,
(incl corrected estimates wrt partitioned broadcast inputs)
Posted by du...@apache.org.
Sparsity-aware spark/mr mapmm/mapmmchain/pmm compilation,
(incl corrected estimates wrt partitioned broadcast inputs)
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/9bb58c69
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/9bb58c69
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/9bb58c69
Branch: refs/heads/gh-pages
Commit: 9bb58c691c5af0d2501876cc81d934ba772dbef8
Parents: e378de0
Author: mboehm7 <mb...@us.ibm.com>
Authored: Thu Aug 20 02:51:52 2015 -0700
Committer: mboehm7 <mb...@us.ibm.com>
Committed: Thu Aug 20 02:51:52 2015 -0700
----------------------------------------------------------------------
october_todo_list.txt | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/9bb58c69/october_todo_list.txt
----------------------------------------------------------------------
diff --git a/october_todo_list.txt b/october_todo_list.txt
index 101663b..e8a32c3 100644
--- a/october_todo_list.txt
+++ b/october_todo_list.txt
@@ -27,7 +27,7 @@
3) Compiler features
* Code motion (cse across loops) 2pw
* Extended inter-procedural analysis 2pw (e.g., step_glm)
-* Sparsity-aware broadcast op selection 1pw
+* Sparsity-aware broadcast op selection 1pw .. done
* Function calls in expressions 2pw
3) Parfor