You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@singa.apache.org by bu...@apache.org on 2016/04/12 08:24:54 UTC
svn commit: r985457 [33/35] - in /websites/staging/singa/trunk/content: ./
community/ develop/ docs/ docs/jp/ docs/kr/ docs/zh/ releases/ v0.1.0/
v0.2.0/ v0.2.0/jp/ v0.2.0/kr/ v0.2.0/zh/
Added: websites/staging/singa/trunk/content/v0.2.0/zh/installation_source.html
==============================================================================
--- websites/staging/singa/trunk/content/v0.2.0/zh/installation_source.html (added)
+++ websites/staging/singa/trunk/content/v0.2.0/zh/installation_source.html Tue Apr 12 06:24:50 2016
@@ -0,0 +1,618 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2016-04-12
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20160412" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Apache SINGA – ä»æºç¨åºå®è£
SIGNA</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.4.min.js"></script>
+
+
+
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+
+
+<meta content="Apache SINGA" name="author"/>
+
+
+<meta content="SINGA, Deep Learning, Distributed training" name="keywords"/>
+
+ </head>
+ <body class="topBarEnabled">
+
+
+
+
+
+
+ <a href="https://github.com/apache/incubator-singa">
+ <img style="position: absolute; top: 0; right: 0; border: 0; z-index: 10000;"
+ src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"
+ alt="Fork me on GitHub">
+ </a>
+
+
+
+
+
+ <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+ <div class="navbar-inner">
+ <div class="container-fluid">
+ <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </a>
+
+ <ul class="nav">
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache SINGA <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../docs/overview.html" title="Introduction">Introduction</a>
+</li>
+
+ <li> <a href="../../docs/quick-start.html" title="Quick Start">Quick Start</a>
+</li>
+
+ <li> <a href="../../downloads.html" title="Downloads">Downloads</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li class="dropdown-submenu">
+ <a href="../../docs/index.html" title="v0.3.0">v0.3.0</a>
+ <ul class="dropdown-menu">
+ <li> <a href="../../docs/index.html" title="English">English</a>
+</li>
+ <li> <a href="../../docs/zh/index.html" title="ä¸æ">ä¸æ</a>
+</li>
+ <li> <a href="../../docs/jp/index.html" title="æ¥æ¬èª">æ¥æ¬èª</a>
+</li>
+ <li> <a href="../../docs/kr/index.html" title="íêµì´">íêµì´</a>
+</li>
+ </ul>
+ </li>
+
+ <li> <a href="../../v0.2.0/index.html" title="v0.2.0">v0.2.0</a>
+</li>
+
+ <li> <a href="../../v0.1.0/index.html" title="v0.1.0">v0.1.0</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Development <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../develop/schedule.html" title="Schedule">Schedule</a>
+</li>
+
+ <li class="dropdown-submenu">
+ <a href="../../develop/how-contribute.html" title="How to Contribute">How to Contribute</a>
+ <ul class="dropdown-menu">
+ <li> <a href="../../develop/contribute-code.html" title="Code">Code</a>
+</li>
+ <li> <a href="../../develop/contribute-docs.html" title="Documentation">Documentation</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../community/source-repository.html" title="Source Repository">Source Repository</a>
+</li>
+
+ <li> <a href="../../community/mail-lists.html" title="Mailing Lists">Mailing Lists</a>
+</li>
+
+ <li> <a href="../../community/issue-tracking.html" title="Issue Tracking">Issue Tracking</a>
+</li>
+
+ <li> <a href="../../community/team-list.html" title="SINGA Team">SINGA Team</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">External Links <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="http://www.apache.org/" title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+
+ <li> <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" title="NUS Site">NUS Site</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+
+ </div>
+
+ </div>
+ </div>
+ </div>
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="../../index.html" id="bannerLeft" title="Apache SINGA">
+ <img src="../../images/singa-logo.png" alt="Apache SINGA"/>
+ </a>
+ </div>
+ <div class="pull-right"> <div id="bannerRight">
+ <img src="../../images/singa-title.png" alt="Apache SINGA"/>
+ </div>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="../../index.html" title="Apache SINGA">
+ Apache SINGA</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active ">ä»æºç¨åºå®è£
SIGNA</li>
+
+
+
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">Apache SINGA</li>
+
+ <li>
+
+ <a href="../../docs/overview.html" title="Introduction">
+ <span class="none"></span>
+ Introduction</a>
+ </li>
+
+ <li>
+
+ <a href="../../docs/quick-start.html" title="Quick Start">
+ <span class="none"></span>
+ Quick Start</a>
+ </li>
+
+ <li>
+
+ <a href="../../downloads.html" title="Downloads">
+ <span class="none"></span>
+ Downloads</a>
+ </li>
+ <li class="nav-header">Documentaion</li>
+
+ <li>
+
+ <a href="../../docs/index.html" title="v0.3.0">
+ <span class="icon-chevron-right"></span>
+ v0.3.0</a>
+ </li>
+
+ <li>
+
+ <a href="../../v0.2.0/index.html" title="v0.2.0">
+ <span class="none"></span>
+ v0.2.0</a>
+ </li>
+
+ <li>
+
+ <a href="../../v0.1.0/index.html" title="v0.1.0">
+ <span class="none"></span>
+ v0.1.0</a>
+ </li>
+ <li class="nav-header">Development</li>
+
+ <li>
+
+ <a href="../../develop/schedule.html" title="Schedule">
+ <span class="none"></span>
+ Schedule</a>
+ </li>
+
+ <li>
+
+ <a href="../../develop/how-contribute.html" title="How to Contribute">
+ <span class="icon-chevron-right"></span>
+ How to Contribute</a>
+ </li>
+ <li class="nav-header">Community</li>
+
+ <li>
+
+ <a href="../../community/source-repository.html" title="Source Repository">
+ <span class="none"></span>
+ Source Repository</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/mail-lists.html" title="Mailing Lists">
+ <span class="none"></span>
+ Mailing Lists</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/issue-tracking.html" title="Issue Tracking">
+ <span class="none"></span>
+ Issue Tracking</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/team-list.html" title="SINGA Team">
+ <span class="none"></span>
+ SINGA Team</a>
+ </li>
+ <li class="nav-header">External Links</li>
+
+ <li>
+
+ <a href="http://www.apache.org/" class="externalLink" title="Apache Software Foundation">
+ <span class="none"></span>
+ Apache Software Foundation</a>
+ </li>
+
+ <li>
+
+ <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" class="externalLink" title="NUS Site">
+ <span class="none"></span>
+ NUS Site</a>
+ </li>
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://incubator.apache.org" title="apache-incubator" class="builtBy">
+ <img class="builtBy" alt="Apache Incubator" src="http://incubator.apache.org/images/egg-logo.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <h1>从源程序安装SIGNA</h1>
+<hr />
+<div class="section">
+<h2><a name="a"></a>依赖</h2>
+<p>SINGA 在Linux平台上开发与测试。安装SINGA需要下拉列依赖库：</p>
+
+<ul>
+
+<li>
+<p>glog version 0.3.3</p></li>
+
+<li>
+<p>google-protobuf version 2.6.0</p></li>
+
+<li>
+<p>openblas version >= 0.2.10</p></li>
+
+<li>
+<p>zeromq version >= 3.2</p></li>
+
+<li>
+<p>czmq version >= 3</p></li>
+
+<li>
+<p>zookeeper version 3.4.6</p></li>
+</ul>
+<p>可选依赖包括：</p>
+
+<ul>
+
+<li>lmdb version 0.9.10</li>
+</ul>
+<p>你可以使用下列命令将所有的依赖库安装到$PREFIX文件夹下：</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"># make sure you are in the thirdparty folder
+cd thirdparty
+./install.sh all $PREFIX
+</pre></div></div>
+<p>如果$PREFIX不是一个系统路径（如：/esr/local/），请在继续安装前使用下述命令导出相关变量：</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">export LD_LIBRARY_PATH=$PREFIX/lib:$LD_LIBRARY_PATH
+export CPLUS_INCLUDE_PATH=$PREFIX/include:$CPLUS_INCLUDE_PATH
+export LIBRARY_PATH=$PREFIX/lib:$LIBRARY_PATH
+export PATH=$PREFIX/bin:$PATH
+</pre></div></div>
+<p>关于使用这个脚本的细节后文会详细介绍。</p></div>
+<div class="section">
+<h2><a name="aSINGA"></a>从源程序安装SINGA</h2>
+<p>SINGA 使用 GNU autotools 构建，需要GCC (version >= 4.8)。 有两种方式安装SINGA。</p>
+
+<ul>
+
+<li>
+<p>如果你想使用最近的代码，请执行以下命令从 <a class="externalLink" href="https://github.com/apache/incubator-singa.git">Github</a> 上克隆：</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">$ git clone git@github.com:apache/incubator-singa.git
+$ cd incubator-singa
+$ ./autogen.sh
+$ ./configure
+$ make
+</pre></div></div></li>
+</ul>
+<p>注意: 由于我们的疏忽，在加入Apache Incubator项目后，<a class="externalLink" href="https://github.com/orgs/nusinga">nusinga</a> 帐号下的SINGA库（repo）并没有删除，但它早已没有更新，很抱歉给大家带来的不便。</p>
+
+<ul>
+
+<li>
+<p>如果你下载了发布包，请按以下命令安装：</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">$ tar xvf singa-xxx
+$ cd singa-xxx
+$ ./configure
+$ make
+</pre></div></div>
+<p>SINGA的部分特性依赖于外部库，这些特性可以使用<tt>--enable-<feature></tt>编译。 比如，按准跟支持lmdb的SINGA，可以运行下面的命令：</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">$ ./configure --enable-lmdb
+</pre></div></div></li>
+</ul>
+<!-- -
+Zhongle: please update the code to use the follow command
+
+ $ make test
+
+After compilation, you will find the binary file singatest. Just run it!
+More details about configure script can be found by running:
+
+ $ ./configure -h -->
+<p>SINGA编译成功后， <i>libsinga.so</i> 和可执行文件 <i>singa</i> 会生成在 <i>.libs/</i> 文件夹下。</p>
+<p>如果缺失（或没有检测到）某些依赖库，可使用下面的脚本下载和安装：</p>
+<!-- -
+to be updated after zhongle changes the code to use
+
+ ./install.sh libname \-\-prefix= -->
+
+<div class="source">
+<div class="source"><pre class="prettyprint"># must goto thirdparty folder
+$ cd thirdparty
+$ ./install.sh LIB_NAME PREFIX
+</pre></div></div>
+<p>如果没有指定安装路径，这些库会被安装在这些软件默认的安装路径下。比如，如果想在默认系统文件夹下安装<tt>zeromq</tt>，请执行以下命令：</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">$ ./install.sh zeromq
+</pre></div></div>
+<p>或者，如果想安装到其他目录：</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">$ ./install.sh zeromq PREFIX
+</pre></div></div>
+<p>也可以将所有的依赖库安装到 <i>/usr/local</i> 目录:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">$ ./install.sh all /usr/local
+</pre></div></div>
+<p>下表展示了各依赖库的第一个参数：</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">LIB_NAME LIBRARIE
+czmq* czmq lib
+glog glog lib
+lmdb lmdb lib
+OpenBLAS OpenBLAS lib
+protobuf Google protobuf
+zeromq zeromq lib
+zookeeper Apache zookeeper
+</pre></div></div>
+<p>*: 因为 <tt>czmq</tt> 依赖于 <tt>zeromq</tt>，下述脚本多提供一个参数，说明 <tt>zeromq</tt> 的位置。 <tt>czmq</tt> 的安装命令是：</p>
+<!-- -
+to be updated to
+
+ $./install.sh czmq \-\-prefix=/usr/local \-\-zeromq=/usr/local/zeromq -->
+
+<div class="source">
+<div class="source"><pre class="prettyprint">$./install.sh czmq /usr/local -f=/usr/local/zeromq
+</pre></div></div>
+<p>执行后，<tt>czmq</tt> 会被安装在 <i>/usr/local</i>，上述最后一个路径指明了 zeromq 的路径。</p>
+<div class="section">
+<h3><a name="a"></a>常见问题</h3>
+
+<ul>
+
+<li>Q1: 即使安装了 OpenBLAS，仍遇见 <tt>./configure --> cannot find blas_segmm() function</tt> 错误。</li>
+</ul>
+<p>A1: 该错误是指编译器找不着<tt>OpenBLAS</tt>，如果你安装在 $PREFIX (如, /opt/OpenBLAS)，你需要将路径导出，如下所示</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> $ export LIBRARY_PATH=$PREFIX/lib:$LIBRARY_PATH
+ # e.g.,
+ $ export LIBRARY_PATH=/opt/OpenBLAS/lib:$LIBRARY_PATH
+</pre></div></div>
+
+<ul>
+
+<li>Q2: 碰见错误<tt>cblas.h no such file or directory exists</tt>。</li>
+</ul>
+<p>Q2: 你需要将 cblas.h 所在文件夹包含到 CPLUS_INCLUDE_PATH 中，如：</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> $ export CPLUS_INCLUDE_PATH=$PREFIX/include:$CPLUS_INCLUDE_PATH
+ # e.g.,
+ $ export CPLUS_INCLUDE_PATH=/opt/OpenBLAS/include:$CPLUS_INCLUDE_PATH
+ # then reconfigure and make SINGA
+ $ ./configure
+ $ make
+</pre></div></div>
+
+<ul>
+
+<li>Q3: 编译SINGA时，碰见错误<tt>SSE2 instruction set not enabled</tt>。</li>
+</ul>
+<p>A3: 你可以尝试以下命令:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> $ make CFLAGS='-msse2' CXXFLAGS='-msse2'
+</pre></div></div>
+
+<ul>
+
+<li>Q4: 当我试着import .py文件时，从google.protobuf.internal 得到错误<tt>ImportError: cannot import name enum_type_wrapper</tt>。</li>
+</ul>
+<p>A4: 通过 <tt>make install</tt> 安装google protobuf后, 我们应该安装python运行时库。在protobuf源文件夹下运行：</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> $ cd /PROTOBUF/SOURCE/FOLDER
+ $ cd python
+ $ python setup.py build
+ $ python setup.py install
+</pre></div></div>
+<p>如果你要在系统文件夹中安装python的运行时库，可能要用<tt>sudo</tt>。</p>
+
+<ul>
+
+<li>Q5: 遇见由gflags导致的链接错误。</li>
+</ul>
+<p>A5: SINGA不依赖gflags，但你可能在安装glog时安装了gflags。这种情况下你需要用 <i>thirdparty/install.sh</i> 重新将glog安装到另一文件夹，并将该文件夹路径导出到LDFLAGS 和 CPPFLAGS 中。</p>
+
+<ul>
+
+<li>Q6: 在mac OS X上编译SINGA和安装 <tt>glog</tt> 时，遇到了致命错误 <tt>'ext/slist' file not found</tt></li>
+</ul>
+<p>A6: 请单独安装<tt>glog</tt>，再尝试以下命令:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> $ make CFLAGS='-stdlib=libstdc++' CXXFLAGS='stdlib=libstdc++'
+</pre></div></div>
+
+<ul>
+
+<li>Q7: 当我启动一个训练作业时，程序报错为 “ZOO_ERROR…zk retcode=-4…”。</li>
+</ul>
+<p>A7: 这是因为 zookeeper 没有启动，请启动 zookeeper 服务。</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> $ ./bin/zk-service start
+</pre></div></div>
+<p>如果仍有这个错误，可能是没有java，你可以用下述命令查看</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> $ java --version
+</pre></div></div>
+
+<ul>
+
+<li>Q8: 当我从源文件安装 OpenBLAS 时，被告知需要一个 fortran 编译器。</li>
+</ul>
+<p>A8: 按如下命令编译 OpenBLAS：</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> $ make ONLY_CBLAS=1
+</pre></div></div>
+<p>或者用apt-get安装</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> $ sudo apt-get install openblas-dev
+</pre></div></div>
+<p>或者</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> $ sudo yum install openblas-devel
+</pre></div></div>
+<p>后两个命令需要 root 权限，注意OpenBLAS安装后设置环境变量包含头文件和库的路径（参照 依赖 小节）</p>
+
+<ul>
+
+<li>Q9: 当我安装 protocol buffer 时，被告知 GLIBC++_3.4.20 not found in /usr/lib64/libstdc++.so.6.</li>
+</ul>
+<p>A9: 这说明链接器找到了 libstdc++.so.6，但是这个文件比用于编译和链接程序的GCC版本老。程序要求属于新版本GCC的libstdc++，所以必须告诉链接器怎么找到新版本的额libstdc++共享库。最简单的解决方法是找到正确的 libstdc++，并把它导出到 LD_LIBRARY_PATH 中。如, 如果GLIBC++_3.4.20 被下面的命令列出，</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> $ strings /usr/local/lib64/libstdc++.so.6|grep GLIBC++
+</pre></div></div>
+<p>你只需这样设置你的环境变量：</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> $ export LD_LIBRARY_PATH=/usr/local/lib64:$LD_LIBRARY_PATH
+</pre></div></div>
+
+<ul>
+
+<li>Q10: 当我在编译glog时，提示如下错误“src/logging_unittest.cc:83:20: error: ‘gflags’ is not a namespace-name”</li>
+</ul>
+<p>A10: 可能是你已经安装的gflags版本，其命名空间不是gflags，而是其他的，比如是’google’。因此glog不能找到 ‘gflags’ 命名空间。</p>
+<p>编译glog不需要gflags，你可以修改 configure.ac 文件，忽略 gflags。</p>
+
+<ol style="list-style-type: decimal">
+
+<li>cd to glog src directory</li>
+
+<li>修改 configure.ac 第125行，改为 “AC_CHECK_LIB(gflags, main, ac_cv_have_libgflags=0, ac_cv_have_libgflags=0)”</li>
+
+<li>autoreconf</li>
+</ol>
+<p>然后，请重新编译glog。</p></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+
+<p>Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>
Added: websites/staging/singa/trunk/content/v0.2.0/zh/mlp.html
==============================================================================
--- websites/staging/singa/trunk/content/v0.2.0/zh/mlp.html (added)
+++ websites/staging/singa/trunk/content/v0.2.0/zh/mlp.html Tue Apr 12 06:24:50 2016
@@ -0,0 +1,538 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2016-04-12
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20160412" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Apache SINGA – MLP Example</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.4.min.js"></script>
+
+
+
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+
+
+<meta content="Apache SINGA" name="author"/>
+
+
+<meta content="SINGA, Deep Learning, Distributed training" name="keywords"/>
+
+ </head>
+ <body class="topBarEnabled">
+
+
+
+
+
+
+ <a href="https://github.com/apache/incubator-singa">
+ <img style="position: absolute; top: 0; right: 0; border: 0; z-index: 10000;"
+ src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"
+ alt="Fork me on GitHub">
+ </a>
+
+
+
+
+
+ <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+ <div class="navbar-inner">
+ <div class="container-fluid">
+ <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </a>
+
+ <ul class="nav">
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache SINGA <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../docs/overview.html" title="Introduction">Introduction</a>
+</li>
+
+ <li> <a href="../../docs/quick-start.html" title="Quick Start">Quick Start</a>
+</li>
+
+ <li> <a href="../../downloads.html" title="Downloads">Downloads</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li class="dropdown-submenu">
+ <a href="../../docs/index.html" title="v0.3.0">v0.3.0</a>
+ <ul class="dropdown-menu">
+ <li> <a href="../../docs/index.html" title="English">English</a>
+</li>
+ <li> <a href="../../docs/zh/index.html" title="ä¸æ">ä¸æ</a>
+</li>
+ <li> <a href="../../docs/jp/index.html" title="æ¥æ¬èª">æ¥æ¬èª</a>
+</li>
+ <li> <a href="../../docs/kr/index.html" title="íêµì´">íêµì´</a>
+</li>
+ </ul>
+ </li>
+
+ <li> <a href="../../v0.2.0/index.html" title="v0.2.0">v0.2.0</a>
+</li>
+
+ <li> <a href="../../v0.1.0/index.html" title="v0.1.0">v0.1.0</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Development <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../develop/schedule.html" title="Schedule">Schedule</a>
+</li>
+
+ <li class="dropdown-submenu">
+ <a href="../../develop/how-contribute.html" title="How to Contribute">How to Contribute</a>
+ <ul class="dropdown-menu">
+ <li> <a href="../../develop/contribute-code.html" title="Code">Code</a>
+</li>
+ <li> <a href="../../develop/contribute-docs.html" title="Documentation">Documentation</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../community/source-repository.html" title="Source Repository">Source Repository</a>
+</li>
+
+ <li> <a href="../../community/mail-lists.html" title="Mailing Lists">Mailing Lists</a>
+</li>
+
+ <li> <a href="../../community/issue-tracking.html" title="Issue Tracking">Issue Tracking</a>
+</li>
+
+ <li> <a href="../../community/team-list.html" title="SINGA Team">SINGA Team</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">External Links <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="http://www.apache.org/" title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+
+ <li> <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" title="NUS Site">NUS Site</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+
+ </div>
+
+ </div>
+ </div>
+ </div>
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="../../index.html" id="bannerLeft" title="Apache SINGA">
+ <img src="../../images/singa-logo.png" alt="Apache SINGA"/>
+ </a>
+ </div>
+ <div class="pull-right"> <div id="bannerRight">
+ <img src="../../images/singa-title.png" alt="Apache SINGA"/>
+ </div>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="../../index.html" title="Apache SINGA">
+ Apache SINGA</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active ">MLP Example</li>
+
+
+
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">Apache SINGA</li>
+
+ <li>
+
+ <a href="../../docs/overview.html" title="Introduction">
+ <span class="none"></span>
+ Introduction</a>
+ </li>
+
+ <li>
+
+ <a href="../../docs/quick-start.html" title="Quick Start">
+ <span class="none"></span>
+ Quick Start</a>
+ </li>
+
+ <li>
+
+ <a href="../../downloads.html" title="Downloads">
+ <span class="none"></span>
+ Downloads</a>
+ </li>
+ <li class="nav-header">Documentaion</li>
+
+ <li>
+
+ <a href="../../docs/index.html" title="v0.3.0">
+ <span class="icon-chevron-right"></span>
+ v0.3.0</a>
+ </li>
+
+ <li>
+
+ <a href="../../v0.2.0/index.html" title="v0.2.0">
+ <span class="none"></span>
+ v0.2.0</a>
+ </li>
+
+ <li>
+
+ <a href="../../v0.1.0/index.html" title="v0.1.0">
+ <span class="none"></span>
+ v0.1.0</a>
+ </li>
+ <li class="nav-header">Development</li>
+
+ <li>
+
+ <a href="../../develop/schedule.html" title="Schedule">
+ <span class="none"></span>
+ Schedule</a>
+ </li>
+
+ <li>
+
+ <a href="../../develop/how-contribute.html" title="How to Contribute">
+ <span class="icon-chevron-right"></span>
+ How to Contribute</a>
+ </li>
+ <li class="nav-header">Community</li>
+
+ <li>
+
+ <a href="../../community/source-repository.html" title="Source Repository">
+ <span class="none"></span>
+ Source Repository</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/mail-lists.html" title="Mailing Lists">
+ <span class="none"></span>
+ Mailing Lists</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/issue-tracking.html" title="Issue Tracking">
+ <span class="none"></span>
+ Issue Tracking</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/team-list.html" title="SINGA Team">
+ <span class="none"></span>
+ SINGA Team</a>
+ </li>
+ <li class="nav-header">External Links</li>
+
+ <li>
+
+ <a href="http://www.apache.org/" class="externalLink" title="Apache Software Foundation">
+ <span class="none"></span>
+ Apache Software Foundation</a>
+ </li>
+
+ <li>
+
+ <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" class="externalLink" title="NUS Site">
+ <span class="none"></span>
+ NUS Site</a>
+ </li>
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://incubator.apache.org" title="apache-incubator" class="builtBy">
+ <img class="builtBy" alt="Apache Incubator" src="http://incubator.apache.org/images/egg-logo.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <h1>MLP Example</h1>
+<hr />
+<p>Multilayer perceptron (MLP) is a subclass of feed-forward neural networks. A MLP typically consists of multiple directly connected layers, with each layer fully connected to the next one. In this example, we will use SINGA to train a <a class="externalLink" href="http://arxiv.org/abs/1003.0358">simple MLP model proposed by Ciresan</a> for classifying handwritten digits from the <a class="externalLink" href="http://yann.lecun.com/exdb/mnist/">MNIST dataset</a>.</p>
+<div class="section">
+<h2><a name="Running_instructions"></a>Running instructions</h2>
+<p>Please refer to the <a href="installation.html">installation</a> page for instructions on building SINGA, and the <a href="quick-start.html">quick start</a> for instructions on starting zookeeper.</p>
+<p>We have provided scripts for preparing the training and test dataset in <i>examples/cifar10/</i>.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"># in examples/mnist
+$ cp Makefile.example Makefile
+$ make download
+$ make create
+</pre></div></div>
+<div class="section">
+<h3><a name="Training_on_CPU"></a>Training on CPU</h3>
+<p>After the datasets are prepared, we start the training by</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">./bin/singa-run.sh -conf examples/mnist/job.conf
+</pre></div></div>
+<p>After it is started, you should see output like</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">Record job information to /tmp/singa-log/job-info/job-1-20150817-055231
+Executing : ./singa -conf /xxx/incubator-singa/examples/mnist/job.conf -singa_conf /xxx/incubator-singa/conf/singa.conf -singa_job 1
+E0817 07:15:09.211885 34073 cluster.cc:51] proc #0 -> 192.168.5.128:49152 (pid = 34073)
+E0817 07:15:14.972231 34114 server.cc:36] Server (group = 0, id = 0) start
+E0817 07:15:14.972520 34115 worker.cc:134] Worker (group = 0, id = 0) start
+E0817 07:15:24.462602 34073 trainer.cc:373] Test step-0, loss : 2.341021, accuracy : 0.109100
+E0817 07:15:47.341076 34073 trainer.cc:373] Train step-0, loss : 2.357269, accuracy : 0.099000
+E0817 07:16:07.173364 34073 trainer.cc:373] Train step-10, loss : 2.222740, accuracy : 0.201800
+E0817 07:16:26.714855 34073 trainer.cc:373] Train step-20, loss : 2.091030, accuracy : 0.327200
+E0817 07:16:46.590946 34073 trainer.cc:373] Train step-30, loss : 1.969412, accuracy : 0.442100
+E0817 07:17:06.207080 34073 trainer.cc:373] Train step-40, loss : 1.865466, accuracy : 0.514800
+E0817 07:17:25.890033 34073 trainer.cc:373] Train step-50, loss : 1.773849, accuracy : 0.569100
+E0817 07:17:51.208935 34073 trainer.cc:373] Test step-60, loss : 1.613709, accuracy : 0.662100
+E0817 07:17:53.176766 34073 trainer.cc:373] Train step-60, loss : 1.659150, accuracy : 0.652600
+E0817 07:18:12.783370 34073 trainer.cc:373] Train step-70, loss : 1.574024, accuracy : 0.666000
+E0817 07:18:32.904942 34073 trainer.cc:373] Train step-80, loss : 1.529380, accuracy : 0.670500
+E0817 07:18:52.608111 34073 trainer.cc:373] Train step-90, loss : 1.443911, accuracy : 0.703500
+E0817 07:19:12.168465 34073 trainer.cc:373] Train step-100, loss : 1.387759, accuracy : 0.721000
+E0817 07:19:31.855865 34073 trainer.cc:373] Train step-110, loss : 1.335246, accuracy : 0.736500
+E0817 07:19:57.327133 34073 trainer.cc:373] Test step-120, loss : 1.216652, accuracy : 0.769900
+</pre></div></div>
+<p>After the training of some steps (depends on the setting) or the job is finished, SINGA will <a href="checkpoint.html">checkpoint</a> the model parameters.</p></div>
+<div class="section">
+<h3><a name="Training_on_GPU"></a>Training on GPU</h3>
+<p>To train this example model on GPU, just add a field in the configuration file for the GPU device,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"># job.conf
+gpu: 0
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Training_using_Python_script"></a>Training using Python script</h3>
+<p>The python helpers come with SINGA 0.2 make it easy to configure the job. For example the job.conf is replaced with a simple python script mnist_mlp.py which has about 30 lines of code following the <a class="externalLink" href="http://keras.io/">Keras API</a>.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">./bin/singa-run.sh -exec tool/python/examples/mnist_mlp.py
+</pre></div></div></div></div>
+<div class="section">
+<h2><a name="Details"></a>Details</h2>
+<p>To train a model in SINGA, you need to prepare the datasets, and a job configuration which specifies the neural net structure, training algorithm (BP or CD), SGD update algorithm (e.g. Adagrad), number of training/test steps, etc.</p>
+<div class="section">
+<h3><a name="Data_preparation"></a>Data preparation</h3>
+<p>Before using SINGA, you need to write a program to pre-process the dataset you use to a format that SINGA can read. Please refer to the <a href="data.html">Data Preparation</a> to get details about preparing this MNIST dataset.</p></div>
+<div class="section">
+<h3><a name="Neural_net"></a>Neural net</h3>
+
+<div style="text-align: center">
+<img src="../images/example-mlp.png" style="width: 230px" alt="" />
+<br /><b>Figure 1 - Net structure of the MLP example. </b></img>
+</div>
+<p>Figure 1 shows the structure of the simple MLP model, which is constructed following <a class="externalLink" href="http://arxiv.org/abs/1003.0358">Ciresan’s paper</a>. The dashed circle contains two layers which represent one feature transformation stage. There are 6 such stages in total. They sizes of the <a href="layer.html#innerproductlayer">InnerProductLayer</a>s in these circles decrease from 2500->2000->1500->1000->500->10.</p>
+<p>Next we follow the guide in <a href="neural-net.html">neural net page</a> and <a href="layer.html">layer page</a> to write the neural net configuration.</p>
+
+<ul>
+
+<li>
+<p>We configure an input layer to read the training/testing records from a disk file.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">layer {
+ name: "data"
+ type: kRecordInput
+ store_conf {
+ backend: "kvfile"
+ path: "examples/mnist/train_data.bin"
+ random_skip: 5000
+ batchsize: 64
+ shape: 784
+ std_value: 127.5
+ mean_value: 127.5
+ }
+ exclude: kTest
+ }
+
+layer {
+ name: "data"
+ type: kRecordInput
+ store_conf {
+ backend: "kvfile"
+ path: "examples/mnist/test_data.bin"
+ batchsize: 100
+ shape: 784
+ std_value: 127.5
+ mean_value: 127.5
+ }
+ exclude: kTrain
+ }
+</pre></div></div></li>
+</ul>
+
+<ul>
+
+<li>
+<p>All <a href="layer.html#innerproductlayer">InnerProductLayer</a>s are configured similarly as,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">layer{
+ name: "fc1"
+ type: kInnerProduct
+ srclayers:"data"
+ innerproduct_conf{
+ num_output: 2500
+ }
+ param{
+ name: "w1"
+ ...
+ }
+ param{
+ name: "b1"
+ ..
+ }
+}
+</pre></div></div>
+<p>with the <tt>num_output</tt> decreasing from 2500 to 10.</p></li>
+
+<li>
+<p>A <a href="layer.html#stanhlayer">STanhLayer</a> is connected to every InnerProductLayer except the last one. It transforms the feature via scaled tanh function.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">layer{
+ name: "tanh1"
+ type: kSTanh
+ srclayers:"fc1"
+}
+</pre></div></div></li>
+
+<li>
+<p>The final <a href="layer.html#softmaxloss">Softmax loss layer</a> connects to LabelLayer and the last STanhLayer.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">layer{
+ name: "loss"
+ type:kSoftmaxLoss
+ softmaxloss_conf{ topk:1 }
+ srclayers:"fc6"
+ srclayers:"data"
+}
+</pre></div></div></li>
+</ul></div>
+<div class="section">
+<h3><a name="Updater"></a>Updater</h3>
+<p>The <a href="updater.html#updater">normal SGD updater</a> is selected. The learning rate shrinks by 0.997 every 60 steps (i.e., one epoch).</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">updater{
+ type: kSGD
+ learning_rate{
+ base_lr: 0.001
+ type : kStep
+ step_conf{
+ change_freq: 60
+ gamma: 0.997
+ }
+ }
+}
+</pre></div></div></div>
+<div class="section">
+<h3><a name="TrainOneBatch_algorithm"></a>TrainOneBatch algorithm</h3>
+<p>The MLP model is a feed-forward model, hence <a href="train-one-batch#back-propagation">Back-propagation algorithm</a> is selected.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">train_one_batch {
+ alg: kBP
+}
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Cluster_setting"></a>Cluster setting</h3>
+<p>The following configuration set a single worker and server for training. <a href="frameworks.html">Training frameworks</a> page introduces configurations of a couple of distributed training frameworks.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">cluster {
+ nworker_groups: 1
+ nserver_groups: 1
+}
+</pre></div></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+
+<p>Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>
Added: websites/staging/singa/trunk/content/v0.2.0/zh/neural-net.html
==============================================================================
--- websites/staging/singa/trunk/content/v0.2.0/zh/neural-net.html (added)
+++ websites/staging/singa/trunk/content/v0.2.0/zh/neural-net.html Tue Apr 12 06:24:50 2016
@@ -0,0 +1,573 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2016-04-12
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20160412" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Apache SINGA – Neural Net</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.4.min.js"></script>
+
+
+
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+
+
+<meta content="Apache SINGA" name="author"/>
+
+
+<meta content="SINGA, Deep Learning, Distributed training" name="keywords"/>
+
+ </head>
+ <body class="topBarEnabled">
+
+
+
+
+
+
+ <a href="https://github.com/apache/incubator-singa">
+ <img style="position: absolute; top: 0; right: 0; border: 0; z-index: 10000;"
+ src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"
+ alt="Fork me on GitHub">
+ </a>
+
+
+
+
+
+ <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+ <div class="navbar-inner">
+ <div class="container-fluid">
+ <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </a>
+
+ <ul class="nav">
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache SINGA <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../docs/overview.html" title="Introduction">Introduction</a>
+</li>
+
+ <li> <a href="../../docs/quick-start.html" title="Quick Start">Quick Start</a>
+</li>
+
+ <li> <a href="../../downloads.html" title="Downloads">Downloads</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li class="dropdown-submenu">
+ <a href="../../docs/index.html" title="v0.3.0">v0.3.0</a>
+ <ul class="dropdown-menu">
+ <li> <a href="../../docs/index.html" title="English">English</a>
+</li>
+ <li> <a href="../../docs/zh/index.html" title="ä¸æ">ä¸æ</a>
+</li>
+ <li> <a href="../../docs/jp/index.html" title="æ¥æ¬èª">æ¥æ¬èª</a>
+</li>
+ <li> <a href="../../docs/kr/index.html" title="íêµì´">íêµì´</a>
+</li>
+ </ul>
+ </li>
+
+ <li> <a href="../../v0.2.0/index.html" title="v0.2.0">v0.2.0</a>
+</li>
+
+ <li> <a href="../../v0.1.0/index.html" title="v0.1.0">v0.1.0</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Development <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../develop/schedule.html" title="Schedule">Schedule</a>
+</li>
+
+ <li class="dropdown-submenu">
+ <a href="../../develop/how-contribute.html" title="How to Contribute">How to Contribute</a>
+ <ul class="dropdown-menu">
+ <li> <a href="../../develop/contribute-code.html" title="Code">Code</a>
+</li>
+ <li> <a href="../../develop/contribute-docs.html" title="Documentation">Documentation</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../community/source-repository.html" title="Source Repository">Source Repository</a>
+</li>
+
+ <li> <a href="../../community/mail-lists.html" title="Mailing Lists">Mailing Lists</a>
+</li>
+
+ <li> <a href="../../community/issue-tracking.html" title="Issue Tracking">Issue Tracking</a>
+</li>
+
+ <li> <a href="../../community/team-list.html" title="SINGA Team">SINGA Team</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">External Links <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="http://www.apache.org/" title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+
+ <li> <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" title="NUS Site">NUS Site</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+
+ </div>
+
+ </div>
+ </div>
+ </div>
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="../../index.html" id="bannerLeft" title="Apache SINGA">
+ <img src="../../images/singa-logo.png" alt="Apache SINGA"/>
+ </a>
+ </div>
+ <div class="pull-right"> <div id="bannerRight">
+ <img src="../../images/singa-title.png" alt="Apache SINGA"/>
+ </div>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="../../index.html" title="Apache SINGA">
+ Apache SINGA</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active ">Neural Net</li>
+
+
+
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">Apache SINGA</li>
+
+ <li>
+
+ <a href="../../docs/overview.html" title="Introduction">
+ <span class="none"></span>
+ Introduction</a>
+ </li>
+
+ <li>
+
+ <a href="../../docs/quick-start.html" title="Quick Start">
+ <span class="none"></span>
+ Quick Start</a>
+ </li>
+
+ <li>
+
+ <a href="../../downloads.html" title="Downloads">
+ <span class="none"></span>
+ Downloads</a>
+ </li>
+ <li class="nav-header">Documentaion</li>
+
+ <li>
+
+ <a href="../../docs/index.html" title="v0.3.0">
+ <span class="icon-chevron-right"></span>
+ v0.3.0</a>
+ </li>
+
+ <li>
+
+ <a href="../../v0.2.0/index.html" title="v0.2.0">
+ <span class="none"></span>
+ v0.2.0</a>
+ </li>
+
+ <li>
+
+ <a href="../../v0.1.0/index.html" title="v0.1.0">
+ <span class="none"></span>
+ v0.1.0</a>
+ </li>
+ <li class="nav-header">Development</li>
+
+ <li>
+
+ <a href="../../develop/schedule.html" title="Schedule">
+ <span class="none"></span>
+ Schedule</a>
+ </li>
+
+ <li>
+
+ <a href="../../develop/how-contribute.html" title="How to Contribute">
+ <span class="icon-chevron-right"></span>
+ How to Contribute</a>
+ </li>
+ <li class="nav-header">Community</li>
+
+ <li>
+
+ <a href="../../community/source-repository.html" title="Source Repository">
+ <span class="none"></span>
+ Source Repository</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/mail-lists.html" title="Mailing Lists">
+ <span class="none"></span>
+ Mailing Lists</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/issue-tracking.html" title="Issue Tracking">
+ <span class="none"></span>
+ Issue Tracking</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/team-list.html" title="SINGA Team">
+ <span class="none"></span>
+ SINGA Team</a>
+ </li>
+ <li class="nav-header">External Links</li>
+
+ <li>
+
+ <a href="http://www.apache.org/" class="externalLink" title="Apache Software Foundation">
+ <span class="none"></span>
+ Apache Software Foundation</a>
+ </li>
+
+ <li>
+
+ <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" class="externalLink" title="NUS Site">
+ <span class="none"></span>
+ NUS Site</a>
+ </li>
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://incubator.apache.org" title="apache-incubator" class="builtBy">
+ <img class="builtBy" alt="Apache Incubator" src="http://incubator.apache.org/images/egg-logo.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <h1>Neural Net</h1>
+<hr />
+<p><tt>NeuralNet</tt> in SINGA represents an instance of user’s neural net model. As the neural net typically consists of a set of layers, <tt>NeuralNet</tt> comprises a set of unidirectionally connected <a href="layer.html">Layer</a>s. This page describes how to convert an user’s neural net into the configuration of <tt>NeuralNet</tt>.</p>
+<p><img src="../images/model-category.png" align="center" width="200px" alt="" /> <span><b>Figure 1 - Categorization of popular deep learning models.</b></span></p>
+<div class="section">
+<h2><a name="Net_structure_configuration"></a>Net structure configuration</h2>
+<p>Users configure the <tt>NeuralNet</tt> by listing all layers of the neural net and specifying each layer’s source layer names. Popular deep learning models can be categorized as Figure 1. The subsequent sections give details for each category.</p>
+<div class="section">
+<h3><a name="Feed-forward_models"></a>Feed-forward models</h3>
+
+<div align="left">
+<img src="../images/mlp-net.png" align="center" width="200px" alt="" />
+<span><b>Figure 2 - Net structure of a MLP model.</b></span>
+</div>
+<p>Feed-forward models, e.g., CNN and MLP, can easily get configured as their layer connections are undirected without circles. The configuration for the MLP model shown in Figure 1 is as follows,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">net {
+ layer {
+ name : 'data"
+ type : kData
+ }
+ layer {
+ name : 'image"
+ type : kImage
+ srclayer: 'data'
+ }
+ layer {
+ name : 'label"
+ type : kLabel
+ srclayer: 'data'
+ }
+ layer {
+ name : 'hidden"
+ type : kHidden
+ srclayer: 'image'
+ }
+ layer {
+ name : 'softmax"
+ type : kSoftmaxLoss
+ srclayer: 'hidden'
+ srclayer: 'label'
+ }
+}
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Energy_models"></a>Energy models</h3>
+<p><img src="../images/rbm-rnn.png" align="center" width="500px" alt="" /> <span><b>Figure 3 - Convert connections in RBM and RNN.</b></span></p>
+<p>For energy models including RBM, DBM, etc., their connections are undirected (i.e., Category B). To represent these models using <tt>NeuralNet</tt>, users can simply replace each connection with two directed connections, as shown in Figure 3a. In other words, for each pair of connected layers, their source layer field should include each other’s name. The full <a href="rbm.html">RBM example</a> has detailed neural net configuration for a RBM model, which looks like</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">net {
+ layer {
+ name : "vis"
+ type : kVisLayer
+ param {
+ name : "w1"
+ }
+ srclayer: "hid"
+ }
+ layer {
+ name : "hid"
+ type : kHidLayer
+ param {
+ name : "w2"
+ share_from: "w1"
+ }
+ srclayer: "vis"
+ }
+}
+</pre></div></div></div>
+<div class="section">
+<h3><a name="RNN_models"></a>RNN models</h3>
+<p>For recurrent neural networks (RNN), users can remove the recurrent connections by unrolling the recurrent layer. For example, in Figure 3b, the original layer is unrolled into a new layer with 4 internal layers. In this way, the model is like a normal feed-forward model, thus can be configured similarly. The <a href="rnn.html">RNN example</a> has a full neural net configuration for a RNN model.</p></div></div>
+<div class="section">
+<h2><a name="Configuration_for_multiple_nets"></a>Configuration for multiple nets</h2>
+<p>Typically, a training job includes three neural nets for training, validation and test phase respectively. The three neural nets share most layers except the data layer, loss layer or output layer, etc.. To avoid redundant configurations for the shared layers, users can uses the <tt>exclude</tt> filed to filter a layer in the neural net, e.g., the following layer will be filtered when creating the testing <tt>NeuralNet</tt>.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">layer {
+ ...
+ exclude : kTest # filter this layer for creating test net
+}
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Neural_net_partitioning"></a>Neural net partitioning</h2>
+<p>A neural net can be partitioned in different ways to distribute the training over multiple workers.</p>
+<div class="section">
+<h3><a name="Batch_and_feature_dimension"></a>Batch and feature dimension</h3>
+<p><img src="../images/partition_fc.png" align="center" width="400px" alt="" /> <span><b>Figure 4 - Partitioning of a fully connected layer.</b></span></p>
+<p>Every layer’s feature blob is considered a matrix whose rows are feature vectors. Thus, one layer can be split on two dimensions. Partitioning on dimension 0 (also called batch dimension) slices the feature matrix by rows. For instance, if the mini-batch size is 256 and the layer is partitioned into 2 sub-layers, each sub-layer would have 128 feature vectors in its feature blob. Partitioning on this dimension has no effect on the parameters, as every <a href="param.html">Param</a> object is replicated in the sub-layers. Partitioning on dimension 1 (also called feature dimension) slices the feature matrix by columns. For example, suppose the original feature vector has 50 units, after partitioning into 2 sub-layers, each sub-layer would have 25 units. This partitioning may result in <a href="param.html">Param</a> object being split, as shown in Figure 4. Both the bias vector and weight matrix are partitioned into two sub-layers.</p></div>
+<div class="section">
+<h3><a name="Partitioning_configuration"></a>Partitioning configuration</h3>
+<p>There are 4 partitioning schemes, whose configurations are give below,</p>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Partitioning each singe layer into sub-layers on batch dimension (see below). It is enabled by configuring the partition dimension of the layer to 0, e.g.,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> # with other fields omitted
+ layer {
+ partition_dim: 0
+ }
+</pre></div></div></li>
+
+<li>
+<p>Partitioning each singe layer into sub-layers on feature dimension (see below). It is enabled by configuring the partition dimension of the layer to 1, e.g.,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> # with other fields omitted
+ layer {
+ partition_dim: 1
+ }
+</pre></div></div></li>
+
+<li>
+<p>Partitioning all layers into different subsets. It is enabled by configuring the location ID of a layer, e.g.,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> # with other fields omitted
+ layer {
+ location: 1
+ }
+ layer {
+ location: 0
+ }
+</pre></div></div></li>
+</ol>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Hybrid partitioning of strategy 1, 2 and 3. The hybrid partitioning is useful for large models. An example application is to implement the <a class="externalLink" href="http://arxiv.org/abs/1404.5997">idea proposed by Alex</a>. Hybrid partitioning is configured like,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> # with other fields omitted
+ layer {
+ location: 1
+ }
+ layer {
+ location: 0
+ }
+ layer {
+ partition_dim: 0
+ location: 0
+ }
+ layer {
+ partition_dim: 1
+ location: 0
+ }
+</pre></div></div></li>
+</ol>
+<p>Currently SINGA supports strategy-2 well. Other partitioning strategies are are under test and will be released in later version.</p></div></div>
+<div class="section">
+<h2><a name="Parameter_sharing"></a>Parameter sharing</h2>
+<p>Parameters can be shared in two cases,</p>
+
+<ul>
+
+<li>
+<p>sharing parameters among layers via user configuration. For example, the visible layer and hidden layer of a RBM shares the weight matrix, which is configured through the <tt>share_from</tt> field as shown in the above RBM configuration. The configurations must be the same (except name) for shared parameters.</p></li>
+
+<li>
+<p>due to neural net partitioning, some <tt>Param</tt> objects are replicated into different workers, e.g., partitioning one layer on batch dimension. These workers share parameter values. SINGA controls this kind of parameter sharing automatically, users do not need to do any configuration.</p></li>
+
+<li>
+<p>the <tt>NeuralNet</tt> for training and testing (and validation) share most layers , thus share <tt>Param</tt> values.</p></li>
+</ul>
+<p>If the shared <tt>Param</tt> instances resident in the same process (may in different threads), they use the same chunk of memory space for their values. But they would have different memory spaces for their gradients. In fact, their gradients will be averaged by the stub or server.</p></div>
+<div class="section">
+<h2><a name="Advanced_user_guide"></a>Advanced user guide</h2>
+<div class="section">
+<h3><a name="Creation"></a>Creation</h3>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">static NeuralNet* NeuralNet::Create(const NetProto& np, Phase phase, int num);
+</pre></div></div>
+<p>The above function creates a <tt>NeuralNet</tt> for a given phase, and returns a pointer to the <tt>NeuralNet</tt> instance. The phase is in {kTrain, kValidation, kTest}. <tt>num</tt> is used for net partitioning which indicates the number of partitions. Typically, a training job includes three neural nets for training, validation and test phase respectively. The three neural nets share most layers except the data layer, loss layer or output layer, etc.. The <tt>Create</tt> function takes in the full net configuration including layers for training, validation and test. It removes layers for phases other than the specified phase based on the <tt>exclude</tt> field in <a href="layer.html">layer configuration</a>:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">layer {
+ ...
+ exclude : kTest # filter this layer for creating test net
+}
+</pre></div></div>
+<p>The filtered net configuration is passed to the constructor of <tt>NeuralNet</tt>:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">NeuralNet::NeuralNet(NetProto netproto, int npartitions);
+</pre></div></div>
+<p>The constructor creates a graph representing the net structure firstly in</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">Graph* NeuralNet::CreateGraph(const NetProto& netproto, int npartitions);
+</pre></div></div>
+<p>Next, it creates a layer for each node and connects layers if their nodes are connected.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">void NeuralNet::CreateNetFromGraph(Graph* graph, int npartitions);
+</pre></div></div>
+<p>Since the <tt>NeuralNet</tt> instance may be shared among multiple workers, the <tt>Create</tt> function returns a pointer to the <tt>NeuralNet</tt> instance .</p></div>
+<div class="section">
+<h3><a name="Parameter_sharing"></a>Parameter sharing</h3>
+<p><tt>Param</tt> sharing is enabled by first sharing the Param configuration (in <tt>NeuralNet::Create</tt>) to create two similar (e.g., the same shape) Param objects, and then calling (in <tt>NeuralNet::CreateNetFromGraph</tt>),</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">void Param::ShareFrom(const Param& from);
+</pre></div></div>
+<p>It is also possible to share <tt>Param</tt>s of two nets, e.g., sharing parameters of the training net and the test net,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">void NeuralNet:ShareParamsFrom(NeuralNet* other);
+</pre></div></div>
+<p>It will call <tt>Param::ShareFrom</tt> for each Param object.</p></div>
+<div class="section">
+<h3><a name="Access_functions"></a>Access functions</h3>
+<p><tt>NeuralNet</tt> provides a couple of access function to get the layers and params of the net:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">const std::vector<Layer*>& layers() const;
+const std::vector<Param*>& params() const ;
+Layer* name2layer(string name) const;
+Param* paramid2param(int id) const;
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Partitioning"></a>Partitioning</h3>
+<div class="section">
+<h4><a name="Implementation"></a>Implementation</h4>
+<p>SINGA partitions the neural net in <tt>CreateGraph</tt> function, which creates one node for each (partitioned) layer. For example, if one layer’s partition dimension is 0 or 1, then it creates <tt>npartition</tt> nodes for it; if the partition dimension is -1, a single node is created, i.e., no partitioning. Each node is assigned a partition (or location) ID. If the original layer is configured with a location ID, then the ID is assigned to each newly created node. These nodes are connected according to the connections of the original layers. Some connection layers will be added automatically. For instance, if two connected sub-layers are located at two different workers, then a pair of bridge layers is inserted to transfer the feature (and gradient) blob between them. When two layers are partitioned on different dimensions, a concatenation layer which concatenates feature rows (or columns) and a slice layer which slices feature rows (or columns) would be inserted. These
connection layers help making the network communication and synchronization transparent to the users.</p></div>
+<div class="section">
+<h4><a name="Dispatching_partitions_to_workers"></a>Dispatching partitions to workers</h4>
+<p>Each (partitioned) layer is assigned a location ID, based on which it is dispatched to one worker. Particularly, the pointer to the <tt>NeuralNet</tt> instance is passed to every worker within the same group, but each worker only computes over the layers that have the same partition (or location) ID as the worker’s ID. When every worker computes the gradients of the entire model parameters (strategy-2), we refer to this process as data parallelism. When different workers compute the gradients of different parameters (strategy-3 or strategy-1), we call this process model parallelism. The hybrid partitioning leads to hybrid parallelism where some workers compute the gradients of the same subset of model parameters while other workers compute on different model parameters. For example, to implement the hybrid parallelism in for the <a class="externalLink" href="http://arxiv.org/abs/1404.5997">DCNN model</a>, we set <tt>partition_dim = 0</tt> for lower layers and <tt>partition
_dim = 1</tt> for higher layers.</p></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+
+<p>Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>