You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@singa.apache.org by bu...@apache.org on 2016/01/13 04:47:58 UTC
svn commit: r977575 [7/19] - in /websites/staging/singa/trunk/content: ./
community/ develop/ docs/ docs/jp/ docs/kr/ docs/zh/ releases/ v0.1.0/
Added: websites/staging/singa/trunk/content/docs/jp/neural-net.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/jp/neural-net.html (added)
+++ websites/staging/singa/trunk/content/docs/jp/neural-net.html Wed Jan 13 03:47:56 2016
@@ -0,0 +1,571 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2016-01-13
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20160113" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Apache SINGA – Neural Net</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.4.min.js"></script>
+
+
+
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+
+
+<meta content="Apache SINGA" name="author"/>
+
+
+<meta content="SINGA, Deep Learning, Distributed training" name="keywords"/>
+
+ </head>
+ <body class="topBarEnabled">
+
+
+
+
+
+
+ <a href="https://github.com/apache/incubator-singa">
+ <img style="position: absolute; top: 0; right: 0; border: 0; z-index: 10000;"
+ src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"
+ alt="Fork me on GitHub">
+ </a>
+
+
+
+
+
+ <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+ <div class="navbar-inner">
+ <div class="container-fluid">
+ <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </a>
+
+ <ul class="nav">
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache SINGA <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../docs/overview.html" title="Introduction">Introduction</a>
+</li>
+
+ <li> <a href="../../docs/quick-start.html" title="Quick Start">Quick Start</a>
+</li>
+
+ <li> <a href="../../downloads.html" title="Downloads">Downloads</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li class="dropdown-submenu">
+ <a href="../../docs/index.html" title="Latest">Latest</a>
+ <ul class="dropdown-menu">
+ <li> <a href="../../docs/installation.html" title="Installation">Installation</a>
+</li>
+ <li> <a href="../../docs/programmer-guide.html" title="Programmer Guide">Programmer Guide</a>
+</li>
+ <li> <a href="../../docs/distributed-training.html" title="Distributed Training">Distributed Training</a>
+</li>
+ <li> <a href="../../docs/data.html" title="Data Preparation">Data Preparation</a>
+</li>
+ <li> <a href="../../docs/checkpoint.html" title="Checkpoint">Checkpoint</a>
+</li>
+ <li> <a href="../../docs/test.html" title="Test">Test</a>
+</li>
+ <li> <a href="../../docs/examples.html" title="Examples">Examples</a>
+</li>
+ <li> <a href="../../docs/debug.html" title="Debug">Debug</a>
+</li>
+ </ul>
+ </li>
+
+ <li> <a href="../../v0.1.0/index.html" title="v0.1.0">v0.1.0</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Development <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../develop/schedule.html" title="Schedule">Schedule</a>
+</li>
+
+ <li class="dropdown-submenu">
+ <a href="../../develop/how-contribute.html" title="How to Contribute">How to Contribute</a>
+ <ul class="dropdown-menu">
+ <li> <a href="../../develop/contribute-code.html" title="Code">Code</a>
+</li>
+ <li> <a href="../../develop/contribute-docs.html" title="Documentation">Documentation</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../community/source-repository.html" title="Source Repository">Source Repository</a>
+</li>
+
+ <li> <a href="../../community/mail-lists.html" title="Mailing Lists">Mailing Lists</a>
+</li>
+
+ <li> <a href="../../community/issue-tracking.html" title="Issue Tracking">Issue Tracking</a>
+</li>
+
+ <li> <a href="../../community/team-list.html" title="SINGA Team">SINGA Team</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">External Links <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="http://www.apache.org/" title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+
+ <li> <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" title="NUS Site">NUS Site</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+
+ </div>
+
+ </div>
+ </div>
+ </div>
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="../../index.html" id="bannerLeft" title="Apache SINGA">
+ <img src="../../images/singa-logo.png" alt="Apache SINGA"/>
+ </a>
+ </div>
+ <div class="pull-right"> <div id="bannerRight">
+ <img src="../../images/singa-title.png" alt="Apache SINGA"/>
+ </div>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="../../index.html" title="Apache SINGA">
+ Apache SINGA</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active ">Neural Net</li>
+
+
+
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">Apache SINGA</li>
+
+ <li>
+
+ <a href="../../docs/overview.html" title="Introduction">
+ <span class="none"></span>
+ Introduction</a>
+ </li>
+
+ <li>
+
+ <a href="../../docs/quick-start.html" title="Quick Start">
+ <span class="none"></span>
+ Quick Start</a>
+ </li>
+
+ <li>
+
+ <a href="../../downloads.html" title="Downloads">
+ <span class="none"></span>
+ Downloads</a>
+ </li>
+ <li class="nav-header">Documentaion</li>
+
+ <li>
+
+ <a href="../../docs/index.html" title="Latest">
+ <span class="icon-chevron-right"></span>
+ Latest</a>
+ </li>
+
+ <li>
+
+ <a href="../../v0.1.0/index.html" title="v0.1.0">
+ <span class="none"></span>
+ v0.1.0</a>
+ </li>
+ <li class="nav-header">Development</li>
+
+ <li>
+
+ <a href="../../develop/schedule.html" title="Schedule">
+ <span class="none"></span>
+ Schedule</a>
+ </li>
+
+ <li>
+
+ <a href="../../develop/how-contribute.html" title="How to Contribute">
+ <span class="icon-chevron-right"></span>
+ How to Contribute</a>
+ </li>
+ <li class="nav-header">Community</li>
+
+ <li>
+
+ <a href="../../community/source-repository.html" title="Source Repository">
+ <span class="none"></span>
+ Source Repository</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/mail-lists.html" title="Mailing Lists">
+ <span class="none"></span>
+ Mailing Lists</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/issue-tracking.html" title="Issue Tracking">
+ <span class="none"></span>
+ Issue Tracking</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/team-list.html" title="SINGA Team">
+ <span class="none"></span>
+ SINGA Team</a>
+ </li>
+ <li class="nav-header">External Links</li>
+
+ <li>
+
+ <a href="http://www.apache.org/" class="externalLink" title="Apache Software Foundation">
+ <span class="none"></span>
+ Apache Software Foundation</a>
+ </li>
+
+ <li>
+
+ <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" class="externalLink" title="NUS Site">
+ <span class="none"></span>
+ NUS Site</a>
+ </li>
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://incubator.apache.org" title="apache-incubator" class="builtBy">
+ <img class="builtBy" alt="Apache Incubator" src="http://incubator.apache.org/images/egg-logo.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <h1>Neural Net</h1>
+<hr />
+<p><tt>NeuralNet</tt> in SINGA represents an instance of user’s neural net model. As the neural net typically consists of a set of layers, <tt>NeuralNet</tt> comprises a set of unidirectionally connected <a href="layer.html">Layer</a>s. This page describes how to convert an user’s neural net into the configuration of <tt>NeuralNet</tt>.</p>
+<p><img src="../images/model-category.png" align="center" width="200px" alt="" /> <span><b>Figure 1 - Categorization of popular deep learning models.</b></span></p>
+<div class="section">
+<h2><a name="Net_structure_configuration"></a>Net structure configuration</h2>
+<p>Users configure the <tt>NeuralNet</tt> by listing all layers of the neural net and specifying each layer’s source layer names. Popular deep learning models can be categorized as Figure 1. The subsequent sections give details for each category.</p>
+<div class="section">
+<h3><a name="Feed-forward_models"></a>Feed-forward models</h3>
+
+<div align="left">
+<img src="../images/mlp-net.png" align="center" width="200px" alt="" />
+<span><b>Figure 2 - Net structure of a MLP model.</b></span>
+</div>
+<p>Feed-forward models, e.g., CNN and MLP, can easily get configured as their layer connections are undirected without circles. The configuration for the MLP model shown in Figure 1 is as follows,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">net {
+ layer {
+ name : 'data"
+ type : kData
+ }
+ layer {
+ name : 'image"
+ type : kImage
+ srclayer: 'data'
+ }
+ layer {
+ name : 'label"
+ type : kLabel
+ srclayer: 'data'
+ }
+ layer {
+ name : 'hidden"
+ type : kHidden
+ srclayer: 'image'
+ }
+ layer {
+ name : 'softmax"
+ type : kSoftmaxLoss
+ srclayer: 'hidden'
+ srclayer: 'label'
+ }
+}
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Energy_models"></a>Energy models</h3>
+<p><img src="../images/rbm-rnn.png" align="center" width="500px" alt="" /> <span><b>Figure 3 - Convert connections in RBM and RNN.</b></span></p>
+<p>For energy models including RBM, DBM, etc., their connections are undirected (i.e., Category B). To represent these models using <tt>NeuralNet</tt>, users can simply replace each connection with two directed connections, as shown in Figure 3a. In other words, for each pair of connected layers, their source layer field should include each other’s name. The full <a href="rbm.html">RBM example</a> has detailed neural net configuration for a RBM model, which looks like</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">net {
+ layer {
+ name : "vis"
+ type : kVisLayer
+ param {
+ name : "w1"
+ }
+ srclayer: "hid"
+ }
+ layer {
+ name : "hid"
+ type : kHidLayer
+ param {
+ name : "w2"
+ share_from: "w1"
+ }
+ srclayer: "vis"
+ }
+}
+</pre></div></div></div>
+<div class="section">
+<h3><a name="RNN_models"></a>RNN models</h3>
+<p>For recurrent neural networks (RNN), users can remove the recurrent connections by unrolling the recurrent layer. For example, in Figure 3b, the original layer is unrolled into a new layer with 4 internal layers. In this way, the model is like a normal feed-forward model, thus can be configured similarly. The <a href="rnn.html">RNN example</a> has a full neural net configuration for a RNN model.</p></div></div>
+<div class="section">
+<h2><a name="Configuration_for_multiple_nets"></a>Configuration for multiple nets</h2>
+<p>Typically, a training job includes three neural nets for training, validation and test phase respectively. The three neural nets share most layers except the data layer, loss layer or output layer, etc.. To avoid redundant configurations for the shared layers, users can uses the <tt>exclude</tt> filed to filter a layer in the neural net, e.g., the following layer will be filtered when creating the testing <tt>NeuralNet</tt>.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">layer {
+ ...
+ exclude : kTest # filter this layer for creating test net
+}
+</pre></div></div></div>
+<div class="section">
+<h2><a name="Neural_net_partitioning"></a>Neural net partitioning</h2>
+<p>A neural net can be partitioned in different ways to distribute the training over multiple workers.</p>
+<div class="section">
+<h3><a name="Batch_and_feature_dimension"></a>Batch and feature dimension</h3>
+<p><img src="../images/partition_fc.png" align="center" width="400px" alt="" /> <span><b>Figure 4 - Partitioning of a fully connected layer.</b></span></p>
+<p>Every layer’s feature blob is considered a matrix whose rows are feature vectors. Thus, one layer can be split on two dimensions. Partitioning on dimension 0 (also called batch dimension) slices the feature matrix by rows. For instance, if the mini-batch size is 256 and the layer is partitioned into 2 sub-layers, each sub-layer would have 128 feature vectors in its feature blob. Partitioning on this dimension has no effect on the parameters, as every <a href="param.html">Param</a> object is replicated in the sub-layers. Partitioning on dimension 1 (also called feature dimension) slices the feature matrix by columns. For example, suppose the original feature vector has 50 units, after partitioning into 2 sub-layers, each sub-layer would have 25 units. This partitioning may result in <a href="param.html">Param</a> object being split, as shown in Figure 4. Both the bias vector and weight matrix are partitioned into two sub-layers.</p></div>
+<div class="section">
+<h3><a name="Partitioning_configuration"></a>Partitioning configuration</h3>
+<p>There are 4 partitioning schemes, whose configurations are give below,</p>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Partitioning each singe layer into sub-layers on batch dimension (see below). It is enabled by configuring the partition dimension of the layer to 0, e.g.,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> # with other fields omitted
+ layer {
+ partition_dim: 0
+ }
+</pre></div></div></li>
+
+<li>
+<p>Partitioning each singe layer into sub-layers on feature dimension (see below). It is enabled by configuring the partition dimension of the layer to 1, e.g.,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> # with other fields omitted
+ layer {
+ partition_dim: 1
+ }
+</pre></div></div></li>
+
+<li>
+<p>Partitioning all layers into different subsets. It is enabled by configuring the location ID of a layer, e.g.,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> # with other fields omitted
+ layer {
+ location: 1
+ }
+ layer {
+ location: 0
+ }
+</pre></div></div></li>
+</ol>
+
+<ol style="list-style-type: decimal">
+
+<li>
+<p>Hybrid partitioning of strategy 1, 2 and 3. The hybrid partitioning is useful for large models. An example application is to implement the <a class="externalLink" href="http://arxiv.org/abs/1404.5997">idea proposed by Alex</a>. Hybrid partitioning is configured like,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint"> # with other fields omitted
+ layer {
+ location: 1
+ }
+ layer {
+ location: 0
+ }
+ layer {
+ partition_dim: 0
+ location: 0
+ }
+ layer {
+ partition_dim: 1
+ location: 0
+ }
+</pre></div></div></li>
+</ol>
+<p>Currently SINGA supports strategy-2 well. Other partitioning strategies are are under test and will be released in later version.</p></div></div>
+<div class="section">
+<h2><a name="Parameter_sharing"></a>Parameter sharing</h2>
+<p>Parameters can be shared in two cases,</p>
+
+<ul>
+
+<li>
+<p>sharing parameters among layers via user configuration. For example, the visible layer and hidden layer of a RBM shares the weight matrix, which is configured through the <tt>share_from</tt> field as shown in the above RBM configuration. The configurations must be the same (except name) for shared parameters.</p></li>
+
+<li>
+<p>due to neural net partitioning, some <tt>Param</tt> objects are replicated into different workers, e.g., partitioning one layer on batch dimension. These workers share parameter values. SINGA controls this kind of parameter sharing automatically, users do not need to do any configuration.</p></li>
+
+<li>
+<p>the <tt>NeuralNet</tt> for training and testing (and validation) share most layers , thus share <tt>Param</tt> values.</p></li>
+</ul>
+<p>If the shared <tt>Param</tt> instances resident in the same process (may in different threads), they use the same chunk of memory space for their values. But they would have different memory spaces for their gradients. In fact, their gradients will be averaged by the stub or server.</p></div>
+<div class="section">
+<h2><a name="Advanced_user_guide"></a>Advanced user guide</h2>
+<div class="section">
+<h3><a name="Creation"></a>Creation</h3>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">static NeuralNet* NeuralNet::Create(const NetProto& np, Phase phase, int num);
+</pre></div></div>
+<p>The above function creates a <tt>NeuralNet</tt> for a given phase, and returns a pointer to the <tt>NeuralNet</tt> instance. The phase is in {kTrain, kValidation, kTest}. <tt>num</tt> is used for net partitioning which indicates the number of partitions. Typically, a training job includes three neural nets for training, validation and test phase respectively. The three neural nets share most layers except the data layer, loss layer or output layer, etc.. The <tt>Create</tt> function takes in the full net configuration including layers for training, validation and test. It removes layers for phases other than the specified phase based on the <tt>exclude</tt> field in <a href="layer.html">layer configuration</a>:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">layer {
+ ...
+ exclude : kTest # filter this layer for creating test net
+}
+</pre></div></div>
+<p>The filtered net configuration is passed to the constructor of <tt>NeuralNet</tt>:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">NeuralNet::NeuralNet(NetProto netproto, int npartitions);
+</pre></div></div>
+<p>The constructor creates a graph representing the net structure firstly in</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">Graph* NeuralNet::CreateGraph(const NetProto& netproto, int npartitions);
+</pre></div></div>
+<p>Next, it creates a layer for each node and connects layers if their nodes are connected.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">void NeuralNet::CreateNetFromGraph(Graph* graph, int npartitions);
+</pre></div></div>
+<p>Since the <tt>NeuralNet</tt> instance may be shared among multiple workers, the <tt>Create</tt> function returns a pointer to the <tt>NeuralNet</tt> instance .</p></div>
+<div class="section">
+<h3><a name="Parameter_sharing"></a>Parameter sharing</h3>
+<p><tt>Param</tt> sharing is enabled by first sharing the Param configuration (in <tt>NeuralNet::Create</tt>) to create two similar (e.g., the same shape) Param objects, and then calling (in <tt>NeuralNet::CreateNetFromGraph</tt>),</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">void Param::ShareFrom(const Param& from);
+</pre></div></div>
+<p>It is also possible to share <tt>Param</tt>s of two nets, e.g., sharing parameters of the training net and the test net,</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">void NeuralNet:ShareParamsFrom(NeuralNet* other);
+</pre></div></div>
+<p>It will call <tt>Param::ShareFrom</tt> for each Param object.</p></div>
+<div class="section">
+<h3><a name="Access_functions"></a>Access functions</h3>
+<p><tt>NeuralNet</tt> provides a couple of access function to get the layers and params of the net:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">const std::vector<Layer*>& layers() const;
+const std::vector<Param*>& params() const ;
+Layer* name2layer(string name) const;
+Param* paramid2param(int id) const;
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Partitioning"></a>Partitioning</h3>
+<div class="section">
+<h4><a name="Implementation"></a>Implementation</h4>
+<p>SINGA partitions the neural net in <tt>CreateGraph</tt> function, which creates one node for each (partitioned) layer. For example, if one layer’s partition dimension is 0 or 1, then it creates <tt>npartition</tt> nodes for it; if the partition dimension is -1, a single node is created, i.e., no partitioning. Each node is assigned a partition (or location) ID. If the original layer is configured with a location ID, then the ID is assigned to each newly created node. These nodes are connected according to the connections of the original layers. Some connection layers will be added automatically. For instance, if two connected sub-layers are located at two different workers, then a pair of bridge layers is inserted to transfer the feature (and gradient) blob between them. When two layers are partitioned on different dimensions, a concatenation layer which concatenates feature rows (or columns) and a slice layer which slices feature rows (or columns) would be inserted. These
connection layers help making the network communication and synchronization transparent to the users.</p></div>
+<div class="section">
+<h4><a name="Dispatching_partitions_to_workers"></a>Dispatching partitions to workers</h4>
+<p>Each (partitioned) layer is assigned a location ID, based on which it is dispatched to one worker. Particularly, the pointer to the <tt>NeuralNet</tt> instance is passed to every worker within the same group, but each worker only computes over the layers that have the same partition (or location) ID as the worker’s ID. When every worker computes the gradients of the entire model parameters (strategy-2), we refer to this process as data parallelism. When different workers compute the gradients of different parameters (strategy-3 or strategy-1), we call this process model parallelism. The hybrid partitioning leads to hybrid parallelism where some workers compute the gradients of the same subset of model parameters while other workers compute on different model parameters. For example, to implement the hybrid parallelism in for the <a class="externalLink" href="http://arxiv.org/abs/1404.5997">DCNN model</a>, we set <tt>partition_dim = 0</tt> for lower layers and <tt>partition
_dim = 1</tt> for higher layers.</p></div></div></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+
+<p>Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>
Added: websites/staging/singa/trunk/content/docs/jp/neuralnet-partition.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/jp/neuralnet-partition.html (added)
+++ websites/staging/singa/trunk/content/docs/jp/neuralnet-partition.html Wed Jan 13 03:47:56 2016
@@ -0,0 +1,348 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2016-01-13
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20160113" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Apache SINGA – Neural Net Partition</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.4.min.js"></script>
+
+
+
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+
+
+<meta content="Apache SINGA" name="author"/>
+
+
+<meta content="SINGA, Deep Learning, Distributed training" name="keywords"/>
+
+ </head>
+ <body class="topBarEnabled">
+
+
+
+
+
+
+ <a href="https://github.com/apache/incubator-singa">
+ <img style="position: absolute; top: 0; right: 0; border: 0; z-index: 10000;"
+ src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"
+ alt="Fork me on GitHub">
+ </a>
+
+
+
+
+
+ <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+ <div class="navbar-inner">
+ <div class="container-fluid">
+ <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </a>
+
+ <ul class="nav">
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache SINGA <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../docs/overview.html" title="Introduction">Introduction</a>
+</li>
+
+ <li> <a href="../../docs/quick-start.html" title="Quick Start">Quick Start</a>
+</li>
+
+ <li> <a href="../../downloads.html" title="Downloads">Downloads</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li class="dropdown-submenu">
+ <a href="../../docs/index.html" title="Latest">Latest</a>
+ <ul class="dropdown-menu">
+ <li> <a href="../../docs/installation.html" title="Installation">Installation</a>
+</li>
+ <li> <a href="../../docs/programmer-guide.html" title="Programmer Guide">Programmer Guide</a>
+</li>
+ <li> <a href="../../docs/distributed-training.html" title="Distributed Training">Distributed Training</a>
+</li>
+ <li> <a href="../../docs/data.html" title="Data Preparation">Data Preparation</a>
+</li>
+ <li> <a href="../../docs/checkpoint.html" title="Checkpoint">Checkpoint</a>
+</li>
+ <li> <a href="../../docs/test.html" title="Test">Test</a>
+</li>
+ <li> <a href="../../docs/examples.html" title="Examples">Examples</a>
+</li>
+ <li> <a href="../../docs/debug.html" title="Debug">Debug</a>
+</li>
+ </ul>
+ </li>
+
+ <li> <a href="../../v0.1.0/index.html" title="v0.1.0">v0.1.0</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Development <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../develop/schedule.html" title="Schedule">Schedule</a>
+</li>
+
+ <li class="dropdown-submenu">
+ <a href="../../develop/how-contribute.html" title="How to Contribute">How to Contribute</a>
+ <ul class="dropdown-menu">
+ <li> <a href="../../develop/contribute-code.html" title="Code">Code</a>
+</li>
+ <li> <a href="../../develop/contribute-docs.html" title="Documentation">Documentation</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../community/source-repository.html" title="Source Repository">Source Repository</a>
+</li>
+
+ <li> <a href="../../community/mail-lists.html" title="Mailing Lists">Mailing Lists</a>
+</li>
+
+ <li> <a href="../../community/issue-tracking.html" title="Issue Tracking">Issue Tracking</a>
+</li>
+
+ <li> <a href="../../community/team-list.html" title="SINGA Team">SINGA Team</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">External Links <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="http://www.apache.org/" title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+
+ <li> <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" title="NUS Site">NUS Site</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+
+ </div>
+
+ </div>
+ </div>
+ </div>
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="../../index.html" id="bannerLeft" title="Apache SINGA">
+ <img src="../../images/singa-logo.png" alt="Apache SINGA"/>
+ </a>
+ </div>
+ <div class="pull-right"> <div id="bannerRight">
+ <img src="../../images/singa-title.png" alt="Apache SINGA"/>
+ </div>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="../../index.html" title="Apache SINGA">
+ Apache SINGA</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active ">Neural Net Partition</li>
+
+
+
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">Apache SINGA</li>
+
+ <li>
+
+ <a href="../../docs/overview.html" title="Introduction">
+ <span class="none"></span>
+ Introduction</a>
+ </li>
+
+ <li>
+
+ <a href="../../docs/quick-start.html" title="Quick Start">
+ <span class="none"></span>
+ Quick Start</a>
+ </li>
+
+ <li>
+
+ <a href="../../downloads.html" title="Downloads">
+ <span class="none"></span>
+ Downloads</a>
+ </li>
+ <li class="nav-header">Documentaion</li>
+
+ <li>
+
+ <a href="../../docs/index.html" title="Latest">
+ <span class="icon-chevron-right"></span>
+ Latest</a>
+ </li>
+
+ <li>
+
+ <a href="../../v0.1.0/index.html" title="v0.1.0">
+ <span class="none"></span>
+ v0.1.0</a>
+ </li>
+ <li class="nav-header">Development</li>
+
+ <li>
+
+ <a href="../../develop/schedule.html" title="Schedule">
+ <span class="none"></span>
+ Schedule</a>
+ </li>
+
+ <li>
+
+ <a href="../../develop/how-contribute.html" title="How to Contribute">
+ <span class="icon-chevron-right"></span>
+ How to Contribute</a>
+ </li>
+ <li class="nav-header">Community</li>
+
+ <li>
+
+ <a href="../../community/source-repository.html" title="Source Repository">
+ <span class="none"></span>
+ Source Repository</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/mail-lists.html" title="Mailing Lists">
+ <span class="none"></span>
+ Mailing Lists</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/issue-tracking.html" title="Issue Tracking">
+ <span class="none"></span>
+ Issue Tracking</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/team-list.html" title="SINGA Team">
+ <span class="none"></span>
+ SINGA Team</a>
+ </li>
+ <li class="nav-header">External Links</li>
+
+ <li>
+
+ <a href="http://www.apache.org/" class="externalLink" title="Apache Software Foundation">
+ <span class="none"></span>
+ Apache Software Foundation</a>
+ </li>
+
+ <li>
+
+ <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" class="externalLink" title="NUS Site">
+ <span class="none"></span>
+ NUS Site</a>
+ </li>
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://incubator.apache.org" title="apache-incubator" class="builtBy">
+ <img class="builtBy" alt="Apache Incubator" src="http://incubator.apache.org/images/egg-logo.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <h1>Neural Net Partition</h1>
+<hr />
+<p>The purposes of partitioning neural network is to distribute the partitions onto different working units (e.g., threads or nodes, called workers in this article) and parallelize the processing. Another reason for partition is to handle large neural network which cannot be hold in a single node. For instance, to train models against images with high resolution we need large neural networks (in terms of training parameters).</p>
+<p>Since <i>Layer</i> is the first class citizen in SIGNA, we do the partition against layers. Specifically, we support partitions at two levels. First, users can configure the location (i.e., worker ID) of each layer. In this way, users assign one worker for each layer. Secondly, for one layer, we can partition its neurons or partition the instances (e.g, images). They are called layer partition and data partition respectively. We illustrate the two types of partitions using an simple convolutional neural network.</p>
+<p><img src="../images/conv-mnist.png" style="width: 220px" alt="" /></p>
+<p>The above figure shows a convolutional neural network without any partition. It has 8 layers in total (one rectangular represents one layer). The first layer is DataLayer (data) which reads data from local disk files/databases (or HDFS). The second layer is a MnistLayer which parses the records from MNIST data to get the pixels of a batch of 8 images (each image is of size 28x28). The LabelLayer (label) parses the records to get the label of each image in the batch. The ConvolutionalLayer (conv1) transforms the input image to the shape of 8x27x27. The ReLULayer (relu1) conducts elementwise transformations. The PoolingLayer (pool1) sub-samples the images. The fc1 layer is fully connected with pool1 layer. It mulitplies each image with a weight matrix to generate a 10 dimension hidden feature which is then normalized by a SoftmaxLossLayer to get the prediction.</p>
+<p><img src="../images/conv-mnist-datap.png" style="width: 1000px" alt="" /></p>
+<p>The above figure shows the convolutional neural network after partitioning all layers except the DataLayer and ParserLayers, into 3 partitions using data partition. The read layers process 4 images of the batch, the black and blue layers process 2 images respectively. Some helper layers, i.e., SliceLayer, ConcateLayer, BridgeSrcLayer, BridgeDstLayer and SplitLayer, are added automatically by our partition algorithm. Layers of the same color resident in the same worker. There would be data transferring across different workers at the boundary layers (i.e., BridgeSrcLayer and BridgeDstLayer), e.g., between s-slice-mnist-conv1 and d-slice-mnist-conv1.</p>
+<p><img src="../images/conv-mnist-layerp.png" style="width: 1000px" alt="" /></p>
+<p>The above figure shows the convolutional neural network after partitioning all layers except the DataLayer and ParserLayers, into 2 partitions using layer partition. We can see that each layer processes all 8 images from the batch. But different partitions process different part of one image. For instance, the layer conv1-00 process only 4 channels. The other 4 channels are processed by conv1-01 which residents in another worker.</p>
+<p>Since the partition is done at the layer level, we can apply different partitions for different layers to get a hybrid partition for the whole neural network. Moreover, we can also specify the layer locations to locate different layers to different workers.</p>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+
+<p>Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>
Added: websites/staging/singa/trunk/content/docs/jp/overview.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/jp/overview.html (added)
+++ websites/staging/singa/trunk/content/docs/jp/overview.html Wed Jan 13 03:47:56 2016
@@ -0,0 +1,386 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia at 2016-01-13
+ | Rendered using Apache Maven Fluido Skin 1.4
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20160113" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Apache SINGA – ã¤ã³ãããã¯ã·ã§ã³</title>
+ <link rel="stylesheet" href="../../css/apache-maven-fluido-1.4.min.css" />
+ <link rel="stylesheet" href="../../css/site.css" />
+ <link rel="stylesheet" href="../../css/print.css" media="print" />
+
+
+
+
+
+ <script type="text/javascript" src="../../js/apache-maven-fluido-1.4.min.js"></script>
+
+
+
+<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
+
+
+<script type="text/x-mathjax-config">MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});</script>
+
+
+<meta content="Apache SINGA" name="author"/>
+
+
+<meta content="SINGA, Deep Learning, Distributed training" name="keywords"/>
+
+ </head>
+ <body class="topBarEnabled">
+
+
+
+
+
+
+ <a href="https://github.com/apache/incubator-singa">
+ <img style="position: absolute; top: 0; right: 0; border: 0; z-index: 10000;"
+ src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png"
+ alt="Fork me on GitHub">
+ </a>
+
+
+
+
+
+ <div id="topbar" class="navbar navbar-fixed-top navbar-inverse">
+ <div class="navbar-inner">
+ <div class="container-fluid">
+ <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </a>
+
+ <ul class="nav">
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache SINGA <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../docs/overview.html" title="Introduction">Introduction</a>
+</li>
+
+ <li> <a href="../../docs/quick-start.html" title="Quick Start">Quick Start</a>
+</li>
+
+ <li> <a href="../../downloads.html" title="Downloads">Downloads</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentaion <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li class="dropdown-submenu">
+ <a href="../../docs/index.html" title="Latest">Latest</a>
+ <ul class="dropdown-menu">
+ <li> <a href="../../docs/installation.html" title="Installation">Installation</a>
+</li>
+ <li> <a href="../../docs/programmer-guide.html" title="Programmer Guide">Programmer Guide</a>
+</li>
+ <li> <a href="../../docs/distributed-training.html" title="Distributed Training">Distributed Training</a>
+</li>
+ <li> <a href="../../docs/data.html" title="Data Preparation">Data Preparation</a>
+</li>
+ <li> <a href="../../docs/checkpoint.html" title="Checkpoint">Checkpoint</a>
+</li>
+ <li> <a href="../../docs/test.html" title="Test">Test</a>
+</li>
+ <li> <a href="../../docs/examples.html" title="Examples">Examples</a>
+</li>
+ <li> <a href="../../docs/debug.html" title="Debug">Debug</a>
+</li>
+ </ul>
+ </li>
+
+ <li> <a href="../../v0.1.0/index.html" title="v0.1.0">v0.1.0</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Development <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../develop/schedule.html" title="Schedule">Schedule</a>
+</li>
+
+ <li class="dropdown-submenu">
+ <a href="../../develop/how-contribute.html" title="How to Contribute">How to Contribute</a>
+ <ul class="dropdown-menu">
+ <li> <a href="../../develop/contribute-code.html" title="Code">Code</a>
+</li>
+ <li> <a href="../../develop/contribute-docs.html" title="Documentation">Documentation</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="../../community/source-repository.html" title="Source Repository">Source Repository</a>
+</li>
+
+ <li> <a href="../../community/mail-lists.html" title="Mailing Lists">Mailing Lists</a>
+</li>
+
+ <li> <a href="../../community/issue-tracking.html" title="Issue Tracking">Issue Tracking</a>
+</li>
+
+ <li> <a href="../../community/team-list.html" title="SINGA Team">SINGA Team</a>
+</li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">External Links <b class="caret"></b></a>
+ <ul class="dropdown-menu">
+
+ <li> <a href="http://www.apache.org/" title="Apache Software Foundation">Apache Software Foundation</a>
+</li>
+
+ <li> <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" title="NUS Site">NUS Site</a>
+</li>
+ </ul>
+ </li>
+ </ul>
+
+
+
+
+ </div>
+
+ </div>
+ </div>
+ </div>
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="../../index.html" id="bannerLeft" title="Apache SINGA">
+ <img src="../../images/singa-logo.png" alt="Apache SINGA"/>
+ </a>
+ </div>
+ <div class="pull-right"> <div id="bannerRight">
+ <img src="../../images/singa-title.png" alt="Apache SINGA"/>
+ </div>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li class="">
+ <a href="../../index.html" title="Apache SINGA">
+ Apache SINGA</a>
+ <span class="divider">/</span>
+ </li>
+ <li class="active ">ã¤ã³ãããã¯ã·ã§ã³</li>
+
+
+
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span2">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">Apache SINGA</li>
+
+ <li>
+
+ <a href="../../docs/overview.html" title="Introduction">
+ <span class="none"></span>
+ Introduction</a>
+ </li>
+
+ <li>
+
+ <a href="../../docs/quick-start.html" title="Quick Start">
+ <span class="none"></span>
+ Quick Start</a>
+ </li>
+
+ <li>
+
+ <a href="../../downloads.html" title="Downloads">
+ <span class="none"></span>
+ Downloads</a>
+ </li>
+ <li class="nav-header">Documentaion</li>
+
+ <li>
+
+ <a href="../../docs/index.html" title="Latest">
+ <span class="icon-chevron-right"></span>
+ Latest</a>
+ </li>
+
+ <li>
+
+ <a href="../../v0.1.0/index.html" title="v0.1.0">
+ <span class="none"></span>
+ v0.1.0</a>
+ </li>
+ <li class="nav-header">Development</li>
+
+ <li>
+
+ <a href="../../develop/schedule.html" title="Schedule">
+ <span class="none"></span>
+ Schedule</a>
+ </li>
+
+ <li>
+
+ <a href="../../develop/how-contribute.html" title="How to Contribute">
+ <span class="icon-chevron-right"></span>
+ How to Contribute</a>
+ </li>
+ <li class="nav-header">Community</li>
+
+ <li>
+
+ <a href="../../community/source-repository.html" title="Source Repository">
+ <span class="none"></span>
+ Source Repository</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/mail-lists.html" title="Mailing Lists">
+ <span class="none"></span>
+ Mailing Lists</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/issue-tracking.html" title="Issue Tracking">
+ <span class="none"></span>
+ Issue Tracking</a>
+ </li>
+
+ <li>
+
+ <a href="../../community/team-list.html" title="SINGA Team">
+ <span class="none"></span>
+ SINGA Team</a>
+ </li>
+ <li class="nav-header">External Links</li>
+
+ <li>
+
+ <a href="http://www.apache.org/" class="externalLink" title="Apache Software Foundation">
+ <span class="none"></span>
+ Apache Software Foundation</a>
+ </li>
+
+ <li>
+
+ <a href="http://www.comp.nus.edu.sg/~dbsystem/singa/" class="externalLink" title="NUS Site">
+ <span class="none"></span>
+ NUS Site</a>
+ </li>
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://incubator.apache.org" title="apache-incubator" class="builtBy">
+ <img class="builtBy" alt="Apache Incubator" src="http://incubator.apache.org/images/egg-logo.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span10" >
+
+ <h1>イントロダクション</h1>
+<hr />
+<p>SINGAは、大規模なデータ分析の為のディープラーニングモデルのトレーニングを目的とした「分散ディープラーニング・プラットフォーム」です。 モデルとなるネットワークの「レイヤー」概念に基づき、直感的にプログラミングできるようデザインされています。</p>
+
+<ul>
+
+<li>
+<p>Convolutional Neural Network (畳み込みニューラルネットワーク) のようなフィードフォワードネットワークや、Restricted Boltzmann Machine (制限ボルツマンマシン) のようなエネルギーモデル、Recurrent Neural Network (再帰型ニューラルネットワーク) モデル等、多様なモデルをサポートします。</p></li>
+
+<li>
+<p>様々な「レイヤー」が、Built-in Layer として実装されています。</p></li>
+
+<li>
+<p>SINGAのアーキテクチャーは、synchronous (同期)、asynchronous (非同期)、そして hybrid (ハイブリッド) トレーニングを実行できるよう設計されています。</p></li>
+
+<li>
+<p>大規模なモデルのトレーニングを並列化するための、ニューラルネットワーク分割スキーム (バッチや特徴の分割) をサポートしています。</p></li>
+</ul>
+<div class="section">
+<h2><a name="a"></a>目的</h2>
+<p>拡張性：分散システムとして、より多くのリソースを利用し、特定の精度に達するまでのトレーニングスピードを向上させる。</p>
+<p>使いやすさ：大規模な分散モデルの効率的なトレーニングに必要となる、データやモデルの分割、ネットワーク通信等、プログラマーにとって時間のかかる作業を簡略化し、ディープで複雑なモデルやアルゴリズムの実装を容易にする。</p></div>
+<div class="section">
+<h2><a name="a"></a>設計理念</h2>
+<p>スケーラビリティは、分散ディープラーニングにおいて重要な研究課題です。 SINGAは、様々なトレーニングフレームワークのスケーラビリティーを保てるようよう設計されています。 * Synchronous (同期)：トレーニングの１ステップで得られる効果を高めます。 * Asynchronous (非同期)：トレーニングの&
#x53ce;束率を高めます。 * Hybrid (ハイブリッド)：コストやリソース(クラスターサイズ等)に合った効果と収束率のバランスをとり、スケーラビリティーを高めます。</p>
+<p>SINGAは、ディープラーニングモデルのネットワークの「レイヤー」概念に基づき、直感的にプログラミング出来るようデザインされています。様々なモデルを実装しトレーニングできます。</p></div>
+<div class="section">
+<h2><a name="a"></a>システム概要</h2>
+<p><img src="../../images/sgd.png" align="center" width="400px" alt="" /> <span><b>Figure 1 - SGD フロー</b></span></p>
+<p>「ディープラーニングモデルをトレーニングする」とは、 特定のタスク(分類、予測等)を達成するために使われる特徴量を生成する変換関数の最適なパラメターを探すことです。 パラメターの良し悪しは、<a class="externalLink" href="https://en.wikipedia.org/wiki/Cross_entropy">Cross-Entropy Loss</a> 等の loss function (損失関数)で判断します。この関数は&#x
4e00;般的に非線形、または非凸関数なので、閉解を探すのが難しいです。</p>
+<p>そこで、Stochastic Gradient Descent (確率的勾配降下法) を利用します。 Figure 1 のように、ランダムに初期化されたパラメーターの値を、損失関数が小さくなるよう繰り返しアップデートしていきます。</p>
+<p><img src="../../images/overview.png" align="center" width="400px" alt="" /> <span><b>Figure 2 - SINGA 概要</b></span></p>
+<p>トレーニングに要するワークロードは、workers と servers に分散されます。Figure 2 のように、ループ毎に workers は <i>TrainOneBatch</i> 関数を呼び、パラメーター勾配を計算します。 <i>TrainOneBatch</i> は、ニューラルネットワークの構造が記述された <i>NeuralNet</i> オブジェクトに基づいて、「レイヤー」を順に見て回ります。 計算さ
れた勾配はローカルノードの stub に送られ、集計された後、対応する servers に送られます。Servers は、アップデートされたパラメーターを workers に送り返し、次のループを実行します。</p></div>
+<div class="section">
+<h2><a name="a"></a>ジョブ</h2>
+<p>SINGAで「ジョブ」とは、ニューラルネットワークモデルとデータ、トレーニング方法やクラスタートポロジー等が記述された、「Job Configuration」を指します。 job configurationは、Figure 2に描かれた次の4つのコンポネントを特定します。</p>
+
+<ul>
+
+<li><a href="neural-net.html">NeuralNet</a>：ニューラルネットワークの構造と、各「レイヤー」の設定を記述します。</li>
+
+<li><a href="train-one-batch.html">TrainOneBatch</a>：異なるモデルカテゴリに適したアルゴリズムを記述します。</li>
+
+<li><a href="updater.html">Updater</a>： serverでの、パラメーターのアップデート方法を記述します。</li>
+
+<li><a href="distributed-training.html">Cluster Topology</a>：workers と servers の分散トポロジーを記述します。</li>
+</ul>
+<p><a href="programming-guide.html">main 関数</a>のSINGAドライバにジョブを渡します。</p>
+<p>このプロセスは、Hadoopでのジョブサブミッションに似ています。 ユーザーが main関数内でジョブの設定をします。 Hadoopユーザーは、自身の mapper や reducer を設定しますが、SINGAユーザーは、自身の「レイヤー」やUpdater等を設定します。</p></div>
+ </div>
+ </div>
+ </div>
+
+ <hr/>
+
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+
+<p>Copyright © 2015 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p>
+ </div>
+
+
+ </div>
+ </footer>
+ </body>
+</html>