You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@singa.apache.org by wa...@apache.org on 2018/05/09 15:25:27 UTC
svn commit: r1831260 [4/8] - in /incubator/singa/site/trunk: en/
en/_sources/docs/ en/_static/css/ en/_static/js/ en/community/ en/develop/
en/docs/ en/docs/model_zoo/ en/docs/model_zoo/caffe/
en/docs/model_zoo/char-rnn/ en/docs/model_zoo/cifar10/ en/d...
Modified: incubator/singa/site/trunk/en/docs/metric.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/metric.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/metric.html (original)
+++ incubator/singa/site/trunk/en/docs/metric.html Wed May 9 15:25:26 2018
@@ -188,8 +188,187 @@
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
- <div class="section" id="metric">
-<h1>Metric<a class="headerlink" href="#metric" title="Permalink to this headline">¶</a></h1>
+ <div class="section" id="module-singa.metric">
+<span id="metric"></span><h1>Metric<a class="headerlink" href="#module-singa.metric" title="Permalink to this headline">¶</a></h1>
+<p>This module includes a set of metric classes for evaluating the modelâs
+performance. The specific metric classes could be converted from C++
+implmentation or implemented directly using Python.</p>
+<p>Example usage:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">singa</span> <span class="k">import</span> <span class="n">tensor</span>
+<span class="kn">from</span> <span class="nn">singa</span> <span class="k">import</span> <span class="n">metric</span>
+
+<span class="n">x</span> <span class="o">=</span> <span class="n">tensor</span><span class="o">.</span><span class="n">Tensor</span><span class="p">((</span><span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">))</span>
+<span class="n">x</span><span class="o">.</span><span class="n">uniform</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> <span class="c1"># randomly genearte the prediction activation</span>
+<span class="n">x</span> <span class="o">=</span> <span class="n">tensor</span><span class="o">.</span><span class="n">SoftMax</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="c1"># normalize the prediction into probabilities</span>
+<span class="n">y</span> <span class="o">=</span> <span class="n">tensor</span><span class="o">.</span><span class="n">from_numpy</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="n">dtype</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">int</span><span class="p">))</span> <span class="c1"># set the truth</span>
+
+<span class="n">f</span> <span class="o">=</span> <span class="n">metric</span><span class="o">.</span><span class="n">Accuracy</span><span class="p">()</span>
+<span class="n">acc</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">evaluate</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span> <span class="c1"># averaged accuracy over all 3 samples in x</span>
+</pre></div>
+</div>
+<dl class="class">
+<dt id="singa.metric.Metric">
+<em class="property">class </em><code class="descclassname">singa.metric.</code><code class="descname">Metric</code><a class="headerlink" href="#singa.metric.Metric" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
+<p>Base metric class.</p>
+<p>Subclasses that wrap the C++ loss classes can use the inherited foward,
+and evaluate functions of this base class. Other subclasses need
+to override these functions. Users need to feed in the <strong>predictions</strong> and
+ground truth to get the metric values.</p>
+<dl class="method">
+<dt id="singa.metric.Metric.forward">
+<code class="descname">forward</code><span class="sig-paren">(</span><em>x</em>, <em>y</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.metric.Metric.forward" title="Permalink to this definition">¶</a></dt>
+<dd><p>Compute the metric for each sample.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>x</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â predictions, one row per sample</li>
+<li><strong>y</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â ground truth values, one row per sample</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">a tensor of floats, one per sample</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.metric.Metric.evaluate">
+<code class="descname">evaluate</code><span class="sig-paren">(</span><em>x</em>, <em>y</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.metric.Metric.evaluate" title="Permalink to this definition">¶</a></dt>
+<dd><p>Compute the averaged metric over all samples.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>x</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â predictions, one row per sample</li>
+<li><strong>y</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â ground truth values, one row per sample</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">a float value for the averaged metric</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.metric.Accuracy">
+<em class="property">class </em><code class="descclassname">singa.metric.</code><code class="descname">Accuracy</code><a class="headerlink" href="#singa.metric.Accuracy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <a class="reference internal" href="#singa.metric.Metric" title="singa.metric.Metric"><code class="xref py py-class docutils literal notranslate"><span class="pre">singa.metric.Metric</span></code></a></p>
+<p>Compute the top one accuracy for single label prediction tasks.</p>
+<p>It calls the C++ functions to do the calculation.</p>
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.metric.Precision">
+<em class="property">class </em><code class="descclassname">singa.metric.</code><code class="descname">Precision</code><span class="sig-paren">(</span><em>top_k</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.metric.Precision" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <a class="reference internal" href="#singa.metric.Metric" title="singa.metric.Metric"><code class="xref py py-class docutils literal notranslate"><span class="pre">singa.metric.Metric</span></code></a></p>
+<p>Make the top-k labels of max probability as the prediction</p>
+<p>Compute the precision against the groundtruth labels</p>
+<dl class="method">
+<dt id="singa.metric.Precision.forward">
+<code class="descname">forward</code><span class="sig-paren">(</span><em>x</em>, <em>y</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.metric.Precision.forward" title="Permalink to this definition">¶</a></dt>
+<dd><p>Compute the precision for each sample.</p>
+<p>Convert tensor to numpy for computation</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>x</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â predictions, one row per sample</li>
+<li><strong>y</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â ground truth labels, one row per sample</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">a tensor of floats, one per sample</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.metric.Precision.evaluate">
+<code class="descname">evaluate</code><span class="sig-paren">(</span><em>x</em>, <em>y</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.metric.Precision.evaluate" title="Permalink to this definition">¶</a></dt>
+<dd><p>Compute the averaged precision over all samples.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>x</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â predictions, one row per sample</li>
+<li><strong>y</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â ground truth values, one row per sample</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">a float value for the averaged metric</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.metric.Recall">
+<em class="property">class </em><code class="descclassname">singa.metric.</code><code class="descname">Recall</code><span class="sig-paren">(</span><em>top_k</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.metric.Recall" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <a class="reference internal" href="#singa.metric.Metric" title="singa.metric.Metric"><code class="xref py py-class docutils literal notranslate"><span class="pre">singa.metric.Metric</span></code></a></p>
+<p>Make the top-k labels of max probability as the prediction</p>
+<p>Compute the recall against the groundtruth labels</p>
+<dl class="method">
+<dt id="singa.metric.Recall.forward">
+<code class="descname">forward</code><span class="sig-paren">(</span><em>x</em>, <em>y</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.metric.Recall.forward" title="Permalink to this definition">¶</a></dt>
+<dd><p>Compute the recall for each sample.</p>
+<p>Convert tensor to numpy for computation</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>x</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â predictions, one row per sample</li>
+<li><strong>y</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â ground truth labels, one row per sample</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">a tensor of floats, one per sample</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.metric.Recall.evaluate">
+<code class="descname">evaluate</code><span class="sig-paren">(</span><em>x</em>, <em>y</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.metric.Recall.evaluate" title="Permalink to this definition">¶</a></dt>
+<dd><p>Compute the averaged precision over all samples.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>x</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â predictions, one row per sample</li>
+<li><strong>y</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â ground truth values, one row per sample</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">a float value for the averaged metric</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</dd></dl>
+
</div>
@@ -255,9 +434,7 @@
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/model_zoo/caffe/README.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/model_zoo/caffe/README.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/model_zoo/caffe/README.html (original)
+++ incubator/singa/site/trunk/en/docs/model_zoo/caffe/README.html Wed May 9 15:25:26 2018
@@ -259,9 +259,7 @@ You can start the prediction program by
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/model_zoo/char-rnn/README.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/model_zoo/char-rnn/README.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/model_zoo/char-rnn/README.html (original)
+++ incubator/singa/site/trunk/en/docs/model_zoo/char-rnn/README.html Wed May 9 15:25:26 2018
@@ -299,9 +299,7 @@ Other plain text files can also be used.
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/model_zoo/cifar10/README.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/model_zoo/cifar10/README.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/model_zoo/cifar10/README.html (original)
+++ incubator/singa/site/trunk/en/docs/model_zoo/cifar10/README.html Wed May 9 15:25:26 2018
@@ -344,9 +344,7 @@ The âmodel.binâ file generated
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/model_zoo/imagenet/alexnet/README.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/model_zoo/imagenet/alexnet/README.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/model_zoo/imagenet/alexnet/README.html (original)
+++ incubator/singa/site/trunk/en/docs/model_zoo/imagenet/alexnet/README.html Wed May 9 15:25:26 2018
@@ -339,9 +339,7 @@ folder in data preprocessing step;</li>
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/model_zoo/imagenet/googlenet/README.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/model_zoo/imagenet/googlenet/README.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/model_zoo/imagenet/googlenet/README.html (original)
+++ incubator/singa/site/trunk/en/docs/model_zoo/imagenet/googlenet/README.html Wed May 9 15:25:26 2018
@@ -334,9 +334,7 @@ Refer to <a class="reference external" h
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/model_zoo/imagenet/inception/README.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/model_zoo/imagenet/inception/README.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/model_zoo/imagenet/inception/README.html (original)
+++ incubator/singa/site/trunk/en/docs/model_zoo/imagenet/inception/README.html Wed May 9 15:25:26 2018
@@ -270,9 +270,7 @@ After downloading and decompressing the
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/model_zoo/imagenet/resnet/README.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/model_zoo/imagenet/resnet/README.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/model_zoo/imagenet/resnet/README.html (original)
+++ incubator/singa/site/trunk/en/docs/model_zoo/imagenet/resnet/README.html Wed May 9 15:25:26 2018
@@ -281,9 +281,7 @@ the convert.py program.</p>
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/model_zoo/imagenet/vgg/README.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/model_zoo/imagenet/vgg/README.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/model_zoo/imagenet/vgg/README.html (original)
+++ incubator/singa/site/trunk/en/docs/model_zoo/imagenet/vgg/README.html Wed May 9 15:25:26 2018
@@ -278,9 +278,7 @@ to SINGA for image classification.</p>
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/model_zoo/index.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/model_zoo/index.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/model_zoo/index.html (original)
+++ incubator/singa/site/trunk/en/docs/model_zoo/index.html Wed May 9 15:25:26 2018
@@ -294,9 +294,7 @@
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/model_zoo/mnist/README.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/model_zoo/mnist/README.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/model_zoo/mnist/README.html (original)
+++ incubator/singa/site/trunk/en/docs/model_zoo/mnist/README.html Wed May 9 15:25:26 2018
@@ -244,9 +244,7 @@ the program with an additional argument<
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/net.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/net.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/net.html (original)
+++ incubator/singa/site/trunk/en/docs/net.html Wed May 9 15:25:26 2018
@@ -188,8 +188,298 @@
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
- <div class="section" id="feedforward-net">
-<h1>FeedForward Net<a class="headerlink" href="#feedforward-net" title="Permalink to this headline">¶</a></h1>
+ <div class="section" id="module-singa.net">
+<span id="feedforward-net"></span><h1>FeedForward Net<a class="headerlink" href="#module-singa.net" title="Permalink to this headline">¶</a></h1>
+<p>Nerual net class for constructing the nets using layers and providing access
+functions for net info, e.g., parameters.</p>
+<p>Example usages:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">singa</span> <span class="k">import</span> <span class="n">net</span> <span class="k">as</span> <span class="n">ffnet</span>
+<span class="kn">from</span> <span class="nn">singa</span> <span class="k">import</span> <span class="n">metric</span>
+<span class="kn">from</span> <span class="nn">singa</span> <span class="k">import</span> <span class="n">loss</span>
+<span class="kn">from</span> <span class="nn">singa</span> <span class="k">import</span> <span class="n">layer</span>
+<span class="kn">from</span> <span class="nn">singa</span> <span class="k">import</span> <span class="n">device</span>
+
+<span class="c1"># create net and add layers</span>
+<span class="n">net</span> <span class="o">=</span> <span class="n">ffnet</span><span class="o">.</span><span class="n">FeedForwardNet</span><span class="p">(</span><span class="n">loss</span><span class="o">.</span><span class="n">SoftmaxCrossEntropy</span><span class="p">(),</span> <span class="n">metric</span><span class="o">.</span><span class="n">Accuracy</span><span class="p">())</span>
+<span class="n">net</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">layer</span><span class="o">.</span><span class="n">Conv2D</span><span class="p">(</span><span class="s1">'conv1'</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">input_sample_shape</span><span class="o">=</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span><span class="mi">32</span><span class="p">,</span><span class="mi">32</span><span class="p">,)))</span>
+<span class="n">net</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">layer</span><span class="o">.</span><span class="n">Activation</span><span class="p">(</span><span class="s1">'relu1'</span><span class="p">))</span>
+<span class="n">net</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">layer</span><span class="o">.</span><span class="n">MaxPooling2D</span><span class="p">(</span><span class="s1">'pool1'</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span>
+<span class="n">net</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">layer</span><span class="o">.</span><span class="n">Flatten</span><span class="p">(</span><span class="s1">'flat'</span><span class="p">))</span>
+<span class="n">net</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">layer</span><span class="o">.</span><span class="n">Dense</span><span class="p">(</span><span class="s1">'dense'</span><span class="p">,</span> <span class="mi">10</span><span class="p">))</span>
+
+<span class="c1"># init parameters</span>
+<span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">net</span><span class="o">.</span><span class="n">param_values</span><span class="p">():</span>
+ <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">shape</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
+ <span class="n">p</span><span class="o">.</span><span class="n">set_value</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
+ <span class="k">else</span><span class="p">:</span>
+ <span class="n">p</span><span class="o">.</span><span class="n">gaussian</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mf">0.01</span><span class="p">)</span>
+
+<span class="c1"># move net onto gpu</span>
+<span class="n">dev</span> <span class="o">=</span> <span class="n">device</span><span class="o">.</span><span class="n">create_cuda_gpu</span><span class="p">()</span>
+<span class="n">net</span><span class="o">.</span><span class="n">to_device</span><span class="p">(</span><span class="n">dev</span><span class="p">)</span>
+
+<span class="c1"># training (skipped)</span>
+
+<span class="c1"># do prediction after training</span>
+<span class="n">x</span> <span class="o">=</span> <span class="n">tensor</span><span class="o">.</span><span class="n">Tensor</span><span class="p">((</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="mi">32</span><span class="p">),</span> <span class="n">dev</span><span class="p">)</span>
+<span class="n">x</span><span class="o">.</span><span class="n">uniform</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
+<span class="n">y</span> <span class="o">=</span> <span class="n">net</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
+<span class="nb">print</span> <span class="n">tensor</span><span class="o">.</span><span class="n">to_numpy</span><span class="p">(</span><span class="n">y</span><span class="p">)</span>
+</pre></div>
+</div>
+<dl class="class">
+<dt id="singa.net.FeedForwardNet">
+<em class="property">class </em><code class="descclassname">singa.net.</code><code class="descname">FeedForwardNet</code><span class="sig-paren">(</span><em>loss=None</em>, <em>metric=None</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.to_device">
+<code class="descname">to_device</code><span class="sig-paren">(</span><em>dev</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.to_device" title="Permalink to this definition">¶</a></dt>
+<dd><p>Move the net onto the given device, including
+all parameters and intermediate data.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.add">
+<code class="descname">add</code><span class="sig-paren">(</span><em>lyr</em>, <em>src=None</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.add" title="Permalink to this definition">¶</a></dt>
+<dd><p>Append a layer into the layer list.</p>
+<p>This function will get the sample shape from the src layers to setup the
+newly added layer. For the first layer, it is setup outside. The calling
+function should ensure the correctness of the layer order. If src is
+None, the last layer is the src layer. If there are multiple src layers,
+the src is a list of the src layers.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
+<li><strong>lyr</strong> (<a class="reference internal" href="layer.html#singa.layer.Layer" title="singa.layer.Layer"><em>Layer</em></a>) â the layer to be added</li>
+<li><strong>src</strong> (<a class="reference internal" href="layer.html#singa.layer.Layer" title="singa.layer.Layer"><em>Layer</em></a>) â the source layer of lyr</li>
+</ul>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.param_values">
+<code class="descname">param_values</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.param_values" title="Permalink to this definition">¶</a></dt>
+<dd><p>Return a list of tensors for all parameters</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.param_specs">
+<code class="descname">param_specs</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.param_specs" title="Permalink to this definition">¶</a></dt>
+<dd><p>Return a list of ParamSpec for all parameters</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.param_names">
+<code class="descname">param_names</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.param_names" title="Permalink to this definition">¶</a></dt>
+<dd><p>Return a list for the names of all params</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.train">
+<code class="descname">train</code><span class="sig-paren">(</span><em>x</em>, <em>y</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.train" title="Permalink to this definition">¶</a></dt>
+<dd><p>Run BP for one iteration.
+This method is deprecated. It is only kept for backward compatibility.
+The name of this method is confusing since it does not update parameters.
+Please use backprob() instead.
+The back progagation algorithm computes gradients but it does not train.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.backprob">
+<code class="descname">backprob</code><span class="sig-paren">(</span><em>x</em>, <em>y</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.backprob" title="Permalink to this definition">¶</a></dt>
+<dd><p>Run BP for one iteration.</p>
+<p>Currently only support nets with a single output layer, and a single
+loss objective and metric.
+For multiple outputs (with multiple loss/metric), please manually
+call forward, compute loss/metric and call backward. backward() is also
+more memory efficient than this function.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>x</strong> â input data, a single input Tensor or a dict: layer name -> Tensor</li>
+<li><strong>y</strong> â label data, a single input Tensor.</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">gradients of parameters and the loss and metric values.</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.evaluate">
+<code class="descname">evaluate</code><span class="sig-paren">(</span><em>x</em>, <em>y</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.evaluate" title="Permalink to this definition">¶</a></dt>
+<dd><p>Evaluate the loss and metric of the given data.</p>
+<p>Currently only support nets with a single output layer, and a single
+loss objective and metric.
+TODO(wangwei) consider multiple loss objectives and metrics.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
+<li><strong>x</strong> â input data, a single input Tensor or a dict: layer name -> Tensor</li>
+<li><strong>y</strong> â label data, a single input Tensor.</li>
+</ul>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.predict">
+<code class="descname">predict</code><span class="sig-paren">(</span><em>x</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.predict" title="Permalink to this definition">¶</a></dt>
+<dd><p>Forward the input data through each layer to get the values of the
+output layers.</p>
+<p>Currently only support nets with a single output layer</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>x</strong> â input data, a single input Tensor or a dict: layer name -> Tensor</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body">a single output tensor as the prediction result.</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.topo_sort">
+<code class="descname">topo_sort</code><span class="sig-paren">(</span><em>layers</em>, <em>src_of_layer</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.topo_sort" title="Permalink to this definition">¶</a></dt>
+<dd><p>Topology sort of layers.</p>
+<p>It would try to preserve the orders of the input layers.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>layers</strong> â a list of layers; the layers from the output of the same
+layer (e.g., slice layer) should be added by users in correct
+order; This function would not change their order.</li>
+<li><strong>src_of_layer</strong> â a dictionary: src layer name -> a list of src layers</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">A list of ordered layer</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.forward">
+<code class="descname">forward</code><span class="sig-paren">(</span><em>flag</em>, <em>x</em>, <em>output=[]</em>, <em>freeze=None</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.forward" title="Permalink to this definition">¶</a></dt>
+<dd><p>Forward the input(s) through every layer.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>flag</strong> â True for training; False for evaluation; could also be
+model_pb2.kTrain or model_pb2.kEval, or other values for future
+use.</li>
+<li><strong>x</strong> â a single SINGA tensor if there is a single input; otherwise, a
+dictionary: layer name-> singa tensor, for each layer accepting
+input data. Do not associate a layer with input tensor if it is
+connected from another layer. For such case, use a Dummy() layer
+to accept the input data and connect the dummy layer to this
+layer.</li>
+<li><strong>output</strong> (<em>list</em>) â a list of layer names whose output would be returned
+in addition to the default output.</li>
+<li><strong>freeze</strong> (<em>str</em>) â layer name, freeze all layers before this layer; flag
+is set to false for these layers.</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last"><dl class="docutils">
+<dt>if there is only one output layer and output arg is empty, return</dt>
+<dd><p class="first last">the result from the single output layer; otherwise, return a
+dictionary: layer name -> output tensor(s)</p>
+</dd>
+</dl>
+</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.backward">
+<code class="descname">backward</code><span class="sig-paren">(</span><em>dy</em>, <em>output=[]</em>, <em>freeze=None</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.backward" title="Permalink to this definition">¶</a></dt>
+<dd><p>Run back-propagation after forward-propagation.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>dy</strong> â a single tensor if there is a single loss function; otherwise,
+a dictionary maps the name of the layer connecting to the loss
+function -> gradient from the loss function. Do not associate a
+layer with gradient tensor if it is connecting to another layer.
+For such case, connect this layer to a Dummy() layer and use the
+dummy layer to accept the gradient.</li>
+<li><strong>output</strong> (<em>list</em>) â a list of layer names whose output gradient would be
+returned in addition to the param gradient</li>
+<li><strong>freeze</strong> (<em>str</em>) â layer name, stop backward after this layer.</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">a geneartor iterator that generates
+(param_names, param_values, param_grads, layer_grads) after
+processing each layer h, where the first three lists are for h
+and the last item is a dictionary which maps
+layer name -> its output gradient tensor(s). At the end of this
+function, the key set includes all layers in the output arg.</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>f</em>, <em>buffer_size=10</em>, <em>use_pickle=False</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save model parameters using io/snapshot.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
+<li><strong>f</strong> â file name</li>
+<li><strong>buffer_size</strong> â size (MB) of the IO, default setting is 10MB; Please
+make sure it is larger than any single parameter object.</li>
+<li><strong>use_pickle</strong> (<em>Boolean</em>) â if true, it would use pickle for dumping;
+otherwise, it would use protobuf for serialization, which uses
+less space.</li>
+</ul>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.net.FeedForwardNet.load">
+<code class="descname">load</code><span class="sig-paren">(</span><em>f</em>, <em>buffer_size=10</em>, <em>use_pickle=False</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.net.FeedForwardNet.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Load model parameters using io/snapshot.</p>
+<p>Please refer to the argument description in save().</p>
+</dd></dl>
+
+</dd></dl>
+
</div>
@@ -255,9 +545,7 @@
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/neural-net.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/neural-net.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/neural-net.html (original)
+++ incubator/singa/site/trunk/en/docs/neural-net.html Wed May 9 15:25:26 2018
@@ -544,9 +544,7 @@ lower layers and <code class="docutils l
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/notebook/README.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/notebook/README.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/notebook/README.html (original)
+++ incubator/singa/site/trunk/en/docs/notebook/README.html Wed May 9 15:25:26 2018
@@ -223,9 +223,7 @@
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/optimizer.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/optimizer.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/optimizer.html (original)
+++ incubator/singa/site/trunk/en/docs/optimizer.html Wed May 9 15:25:26 2018
@@ -188,8 +188,462 @@
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
- <div class="section" id="optimizer">
-<h1>Optimizer<a class="headerlink" href="#optimizer" title="Permalink to this headline">¶</a></h1>
+ <div class="section" id="module-singa.optimizer">
+<span id="optimizer"></span><h1>Optimizer<a class="headerlink" href="#module-singa.optimizer" title="Permalink to this headline">¶</a></h1>
+<p>This module includes a set of optimizers for updating model parameters.</p>
+<p>Example usage:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">singa</span> <span class="k">import</span> <span class="n">optimizer</span>
+<span class="kn">from</span> <span class="nn">singa</span> <span class="k">import</span> <span class="n">tensor</span>
+
+<span class="n">sgd</span> <span class="o">=</span> <span class="n">optimizer</span><span class="o">.</span><span class="n">SGD</span><span class="p">(</span><span class="n">lr</span><span class="o">=</span><span class="mf">0.01</span><span class="p">,</span> <span class="n">momentum</span><span class="o">=</span><span class="mf">0.9</span><span class="p">,</span> <span class="n">weight_decay</span><span class="o">=</span><span class="mf">1e-4</span><span class="p">)</span>
+<span class="n">p</span> <span class="o">=</span> <span class="n">tensor</span><span class="o">.</span><span class="n">Tensor</span><span class="p">((</span><span class="mi">3</span><span class="p">,</span><span class="mi">5</span><span class="p">))</span>
+<span class="n">p</span><span class="o">.</span><span class="n">uniform</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
+<span class="n">g</span> <span class="o">=</span> <span class="n">tensor</span><span class="o">.</span><span class="n">Tensor</span><span class="p">((</span><span class="mi">3</span><span class="p">,</span><span class="mi">5</span><span class="p">))</span>
+<span class="n">g</span><span class="o">.</span><span class="n">gaussian</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mf">0.01</span><span class="p">)</span>
+
+<span class="n">sgd</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">g</span><span class="p">,</span> <span class="n">p</span><span class="p">,</span> <span class="s1">'param'</span><span class="p">)</span> <span class="c1"># use the global lr=0.1 for epoch 1</span>
+<span class="n">sgd</span><span class="o">.</span><span class="n">apply_with_lr</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mf">0.03</span><span class="p">,</span> <span class="n">g</span><span class="p">,</span> <span class="n">p</span><span class="p">,</span> <span class="s1">'param'</span><span class="p">)</span> <span class="c1"># use lr=0.03 for epoch 2</span>
+</pre></div>
+</div>
+<dl class="class">
+<dt id="singa.optimizer.Optimizer">
+<em class="property">class </em><code class="descclassname">singa.optimizer.</code><code class="descname">Optimizer</code><span class="sig-paren">(</span><em>lr=None</em>, <em>momentum=None</em>, <em>weight_decay=None</em>, <em>regularizer=None</em>, <em>constraint=None</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.Optimizer" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
+<p>The base python optimizer class.</p>
+<p>Typically, an optimizer is used as follows:</p>
+<ol class="arabic simple">
+<li>construct the optimizer</li>
+<li>(optional) register each parameter with its specs.</li>
+<li>use the optimizer to update parameter values given parameter gradients
+and other optional info</li>
+</ol>
+<p>The subclasses should override the apply_with_lr function to do the real
+parameter udpate.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
+<li><strong>lr</strong> (<em>float</em>) â a constant value for the learning rate</li>
+<li><strong>momentum</strong> (<em>float</em>) â a constant value for the momentum value</li>
+<li><strong>weight_decay</strong> (<em>float</em>) â the coefficent for L2 regularizer, which is
+mutually exclusive with âregularizerâ.</li>
+<li><strong>regularizer</strong> â an instance of Regularizer or RegularizerConf; If set,
+regularization would be applied in apply_with_lr().
+Users can also do regularization outside.</li>
+<li><strong>constraint</strong> â an instance of Constraint or ConstraintConf; If set,
+constraint would be applied inside apply_with_lr(). Users can
+also apply constraint outside.</li>
+</ul>
+</td>
+</tr>
+</tbody>
+</table>
+<dl class="method">
+<dt id="singa.optimizer.Optimizer.register">
+<code class="descname">register</code><span class="sig-paren">(</span><em>name</em>, <em>specs</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.Optimizer.register" title="Permalink to this definition">¶</a></dt>
+<dd><p>Register the param specs, including creating regularizer and
+constraint per param object. Param specific regularizer and constraint
+have higher priority than the global ones. If all parameters share the
+same setting for learning rate, regularizer and constraint, then there
+is no need to call this function.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
+<li><strong>name</strong> (<em>str</em>) â parameter name</li>
+<li><strong>specs</strong> (<em>ParamSpec</em>) â protobuf obj, including regularizer and
+constraint, multipliers for learning rate and weight decay.</li>
+</ul>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.optimizer.Optimizer.apply_regularizer_constraint">
+<code class="descname">apply_regularizer_constraint</code><span class="sig-paren">(</span><em>epoch</em>, <em>value</em>, <em>grad</em>, <em>name=None</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.Optimizer.apply_regularizer_constraint" title="Permalink to this definition">¶</a></dt>
+<dd><p>Apply regularization and constraint if available.</p>
+<p>If there are both global regularizer (constraint) and param specific
+regularizer (constraint), it would use the param specific one.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>epoch</strong> (<em>int</em>) â training epoch ID</li>
+<li><strong>value</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â parameter value Tensor</li>
+<li><strong>grad</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â parameter gradient Tensor</li>
+<li><strong>name</strong> (<em>string</em>) â to get parameter specific regularizer or constraint</li>
+<li><strong>step</strong> (<em>int</em>) â iteration ID within one epoch</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">the updated gradient Tensor</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.optimizer.Optimizer.apply_with_lr">
+<code class="descname">apply_with_lr</code><span class="sig-paren">(</span><em>epoch</em>, <em>lr</em>, <em>grad</em>, <em>value</em>, <em>name=None</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.Optimizer.apply_with_lr" title="Permalink to this definition">¶</a></dt>
+<dd><p>Do update of parameters with given learning rate if the grad is not
+empty.</p>
+<p>The subclass optimizer must override this function.
+This function do nothing if the grad is empty.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>epoch</strong> (<em>int</em>) â training epoch ID</li>
+<li><strong>lr</strong> (<em>float</em>) â learning rate</li>
+<li><strong>grad</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â parameter gradient</li>
+<li><strong>value</strong> (<em>Tesnor</em>) â parameter value</li>
+<li><strong>name</strong> (<em>string</em>) â paramter name to index parameter specific
+updating rules (including regularizer and constraint)</li>
+<li><strong>step</strong> (<em>int</em>) â iteration ID within one epoch</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">updated parameter value</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.optimizer.Optimizer.apply">
+<code class="descname">apply</code><span class="sig-paren">(</span><em>epoch</em>, <em>grad</em>, <em>value</em>, <em>name=None</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.Optimizer.apply" title="Permalink to this definition">¶</a></dt>
+<dd><p>Do update assuming the learning rate generator is set.</p>
+<p>The subclass optimizer does not need to override this function.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>epoch</strong> (<em>int</em>) â training epoch ID</li>
+<li><strong>grad</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â parameter gradient</li>
+<li><strong>value</strong> (<em>Tesnor</em>) â parameter value</li>
+<li><strong>name</strong> (<em>string</em>) â paramter name to retrieval parameter specific
+updating rules (including regularizer and constraint)</li>
+<li><strong>step</strong> (<em>int</em>) â training iteration ID within one epoch</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">updated parameter value</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.optimizer.SGD">
+<em class="property">class </em><code class="descclassname">singa.optimizer.</code><code class="descname">SGD</code><span class="sig-paren">(</span><em>lr=None</em>, <em>momentum=None</em>, <em>weight_decay=None</em>, <em>regularizer=None</em>, <em>constraint=None</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.SGD" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <a class="reference internal" href="#singa.optimizer.Optimizer" title="singa.optimizer.Optimizer"><code class="xref py py-class docutils literal notranslate"><span class="pre">singa.optimizer.Optimizer</span></code></a></p>
+<p>The vallina Stochasitc Gradient Descent algorithm with momentum.</p>
+<p>See the base Optimizer for all arguments.</p>
+<dl class="method">
+<dt id="singa.optimizer.SGD.apply_with_lr">
+<code class="descname">apply_with_lr</code><span class="sig-paren">(</span><em>epoch</em>, <em>lr</em>, <em>grad</em>, <em>value</em>, <em>name</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.SGD.apply_with_lr" title="Permalink to this definition">¶</a></dt>
+<dd><p>Do update of parameters with given learning rate if the grad is not
+empty.</p>
+<p>The subclass optimizer must override this function.
+This function do nothing if the grad is empty.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>epoch</strong> (<em>int</em>) â training epoch ID</li>
+<li><strong>lr</strong> (<em>float</em>) â learning rate</li>
+<li><strong>grad</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â parameter gradient</li>
+<li><strong>value</strong> (<em>Tesnor</em>) â parameter value</li>
+<li><strong>name</strong> (<em>string</em>) â paramter name to index parameter specific
+updating rules (including regularizer and constraint)</li>
+<li><strong>step</strong> (<em>int</em>) â iteration ID within one epoch</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">updated parameter value</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.optimizer.Nesterov">
+<em class="property">class </em><code class="descclassname">singa.optimizer.</code><code class="descname">Nesterov</code><span class="sig-paren">(</span><em>lr=None</em>, <em>momentum=0.9</em>, <em>weight_decay=None</em>, <em>regularizer=None</em>, <em>constraint=None</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.Nesterov" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <a class="reference internal" href="#singa.optimizer.Optimizer" title="singa.optimizer.Optimizer"><code class="xref py py-class docutils literal notranslate"><span class="pre">singa.optimizer.Optimizer</span></code></a></p>
+<p>The SGD with Nesterov momentum.</p>
+<p>See the base Optimizer for all arguments.</p>
+<dl class="method">
+<dt id="singa.optimizer.Nesterov.apply_with_lr">
+<code class="descname">apply_with_lr</code><span class="sig-paren">(</span><em>epoch</em>, <em>lr</em>, <em>grad</em>, <em>value</em>, <em>name</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.Nesterov.apply_with_lr" title="Permalink to this definition">¶</a></dt>
+<dd><p>Do update of parameters with given learning rate if the grad is not
+empty.</p>
+<p>The subclass optimizer must override this function.
+This function do nothing if the grad is empty.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>epoch</strong> (<em>int</em>) â training epoch ID</li>
+<li><strong>lr</strong> (<em>float</em>) â learning rate</li>
+<li><strong>grad</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â parameter gradient</li>
+<li><strong>value</strong> (<em>Tesnor</em>) â parameter value</li>
+<li><strong>name</strong> (<em>string</em>) â paramter name to index parameter specific
+updating rules (including regularizer and constraint)</li>
+<li><strong>step</strong> (<em>int</em>) â iteration ID within one epoch</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">updated parameter value</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.optimizer.RMSProp">
+<em class="property">class </em><code class="descclassname">singa.optimizer.</code><code class="descname">RMSProp</code><span class="sig-paren">(</span><em>rho=0.9</em>, <em>epsilon=1e-08</em>, <em>lr=None</em>, <em>weight_decay=None</em>, <em>regularizer=None</em>, <em>constraint=None</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.RMSProp" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <a class="reference internal" href="#singa.optimizer.Optimizer" title="singa.optimizer.Optimizer"><code class="xref py py-class docutils literal notranslate"><span class="pre">singa.optimizer.Optimizer</span></code></a></p>
+<p>RMSProp optimizer.</p>
+<p>See the base Optimizer for all constructor args.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
+<li><strong>rho</strong> (<em>float</em>) â float within [0, 1]</li>
+<li><strong>epsilon</strong> (<em>float</em>) â small value for preventing numeric error</li>
+</ul>
+</td>
+</tr>
+</tbody>
+</table>
+<dl class="method">
+<dt id="singa.optimizer.RMSProp.apply_with_lr">
+<code class="descname">apply_with_lr</code><span class="sig-paren">(</span><em>epoch</em>, <em>lr</em>, <em>grad</em>, <em>value</em>, <em>name</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.RMSProp.apply_with_lr" title="Permalink to this definition">¶</a></dt>
+<dd><p>Do update of parameters with given learning rate if the grad is not
+empty.</p>
+<p>The subclass optimizer must override this function.
+This function do nothing if the grad is empty.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>epoch</strong> (<em>int</em>) â training epoch ID</li>
+<li><strong>lr</strong> (<em>float</em>) â learning rate</li>
+<li><strong>grad</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â parameter gradient</li>
+<li><strong>value</strong> (<em>Tesnor</em>) â parameter value</li>
+<li><strong>name</strong> (<em>string</em>) â paramter name to index parameter specific
+updating rules (including regularizer and constraint)</li>
+<li><strong>step</strong> (<em>int</em>) â iteration ID within one epoch</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">updated parameter value</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.optimizer.AdaGrad">
+<em class="property">class </em><code class="descclassname">singa.optimizer.</code><code class="descname">AdaGrad</code><span class="sig-paren">(</span><em>epsilon=1e-08</em>, <em>lr=None</em>, <em>weight_decay=None</em>, <em>lr_gen=None</em>, <em>regularizer=None</em>, <em>constraint=None</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.AdaGrad" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <a class="reference internal" href="#singa.optimizer.Optimizer" title="singa.optimizer.Optimizer"><code class="xref py py-class docutils literal notranslate"><span class="pre">singa.optimizer.Optimizer</span></code></a></p>
+<p>AdaGrad optimizer.</p>
+<p>See the base Optimizer for all constructor args.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>epsilon</strong> (<em>float</em>) â small number for preventing numeric error.</td>
+</tr>
+</tbody>
+</table>
+<dl class="method">
+<dt id="singa.optimizer.AdaGrad.apply_with_lr">
+<code class="descname">apply_with_lr</code><span class="sig-paren">(</span><em>epoch</em>, <em>lr</em>, <em>grad</em>, <em>value</em>, <em>name</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.AdaGrad.apply_with_lr" title="Permalink to this definition">¶</a></dt>
+<dd><p>Do update of parameters with given learning rate if the grad is not
+empty.</p>
+<p>The subclass optimizer must override this function.
+This function do nothing if the grad is empty.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
+<li><strong>epoch</strong> (<em>int</em>) â training epoch ID</li>
+<li><strong>lr</strong> (<em>float</em>) â learning rate</li>
+<li><strong>grad</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â parameter gradient</li>
+<li><strong>value</strong> (<em>Tesnor</em>) â parameter value</li>
+<li><strong>name</strong> (<em>string</em>) â paramter name to index parameter specific
+updating rules (including regularizer and constraint)</li>
+<li><strong>step</strong> (<em>int</em>) â iteration ID within one epoch</li>
+</ul>
+</td>
+</tr>
+<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">updated parameter value</p>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.optimizer.Adam">
+<em class="property">class </em><code class="descclassname">singa.optimizer.</code><code class="descname">Adam</code><span class="sig-paren">(</span><em>beta_1=0.9</em>, <em>beta_2=0.999</em>, <em>epsilon=1e-08</em>, <em>lr=None</em>, <em>weight_decay=None</em>, <em>regularizer=None</em>, <em>constraint=None</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.Adam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <a class="reference internal" href="#singa.optimizer.Optimizer" title="singa.optimizer.Optimizer"><code class="xref py py-class docutils literal notranslate"><span class="pre">singa.optimizer.Optimizer</span></code></a></p>
+<p>Adam optimizer.</p>
+<p>See the base Optimizer for all constructor args.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
+<li><strong>beta_1</strong> (<em>float</em>) â coefficient of momentum</li>
+<li><strong>beta_2</strong> (<em>float</em>) â coefficient of aggregated squared gradient</li>
+<li><strong>epsilon</strong> (<em>float</em>) â small value for preventing numeric error</li>
+</ul>
+</td>
+</tr>
+</tbody>
+</table>
+<dl class="method">
+<dt id="singa.optimizer.Adam.apply_with_lr">
+<code class="descname">apply_with_lr</code><span class="sig-paren">(</span><em>epoch</em>, <em>lr</em>, <em>grad</em>, <em>value</em>, <em>name</em>, <em>step</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.Adam.apply_with_lr" title="Permalink to this definition">¶</a></dt>
+<dd><p>Update one parameter object.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>step</strong> (<em>int</em>) â the accumulated training iterations, not the iteration ID</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.optimizer.Regularizer">
+<em class="property">class </em><code class="descclassname">singa.optimizer.</code><code class="descname">Regularizer</code><a class="headerlink" href="#singa.optimizer.Regularizer" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
+<p>Base Python regularizer for parameter gradients.</p>
+<dl class="method">
+<dt id="singa.optimizer.Regularizer.apply">
+<code class="descname">apply</code><span class="sig-paren">(</span><em>epoch</em>, <em>value</em>, <em>grad</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.Regularizer.apply" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.optimizer.CppRegularizer">
+<em class="property">class </em><code class="descclassname">singa.optimizer.</code><code class="descname">CppRegularizer</code><span class="sig-paren">(</span><em>conf</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.CppRegularizer" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <a class="reference internal" href="#singa.optimizer.Regularizer" title="singa.optimizer.Regularizer"><code class="xref py py-class docutils literal notranslate"><span class="pre">singa.optimizer.Regularizer</span></code></a></p>
+<p>Wrapper for regularizer implemented using C++.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>conf</strong> (<em>RegularizerConf</em>) â protobuf message for the configuration.</td>
+</tr>
+</tbody>
+</table>
+<dl class="method">
+<dt id="singa.optimizer.CppRegularizer.apply">
+<code class="descname">apply</code><span class="sig-paren">(</span><em>epoch</em>, <em>value</em>, <em>grad</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.CppRegularizer.apply" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.optimizer.L2Regularizer">
+<em class="property">class </em><code class="descclassname">singa.optimizer.</code><code class="descname">L2Regularizer</code><span class="sig-paren">(</span><em>coefficient</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.L2Regularizer" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <a class="reference internal" href="#singa.optimizer.Regularizer" title="singa.optimizer.Regularizer"><code class="xref py py-class docutils literal notranslate"><span class="pre">singa.optimizer.Regularizer</span></code></a></p>
+<p>L2 regularization</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>coefficient</strong> (<em>float</em>) â regularization coefficient.</td>
+</tr>
+</tbody>
+</table>
+<dl class="method">
+<dt id="singa.optimizer.L2Regularizer.apply">
+<code class="descname">apply</code><span class="sig-paren">(</span><em>epoch</em>, <em>value</em>, <em>grad</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.L2Regularizer.apply" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.optimizer.Constraint">
+<em class="property">class </em><code class="descclassname">singa.optimizer.</code><code class="descname">Constraint</code><a class="headerlink" href="#singa.optimizer.Constraint" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
+<p>Base Python constraint class for paramter gradients</p>
+<dl class="method">
+<dt id="singa.optimizer.Constraint.apply">
+<code class="descname">apply</code><span class="sig-paren">(</span><em>epoch</em>, <em>value</em>, <em>grad</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.Constraint.apply" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.optimizer.CppConstraint">
+<em class="property">class </em><code class="descclassname">singa.optimizer.</code><code class="descname">CppConstraint</code><span class="sig-paren">(</span><em>conf</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.CppConstraint" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <a class="reference internal" href="#singa.optimizer.Constraint" title="singa.optimizer.Constraint"><code class="xref py py-class docutils literal notranslate"><span class="pre">singa.optimizer.Constraint</span></code></a></p>
+<p>Wrapper for constraints implemented using C++.</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>conf</strong> (<em>ConstraintConf</em>) â protobuf message for the configuration.</td>
+</tr>
+</tbody>
+</table>
+<dl class="method">
+<dt id="singa.optimizer.CppConstraint.apply">
+<code class="descname">apply</code><span class="sig-paren">(</span><em>epoch</em>, <em>value</em>, <em>grad</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.CppConstraint.apply" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="singa.optimizer.L2Constraint">
+<em class="property">class </em><code class="descclassname">singa.optimizer.</code><code class="descname">L2Constraint</code><span class="sig-paren">(</span><em>threshold=None</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.L2Constraint" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <a class="reference internal" href="#singa.optimizer.Constraint" title="singa.optimizer.Constraint"><code class="xref py py-class docutils literal notranslate"><span class="pre">singa.optimizer.Constraint</span></code></a></p>
+<p>Rescale the gradient to make the L2 norm <= a given threshold</p>
+<dl class="method">
+<dt id="singa.optimizer.L2Constraint.apply">
+<code class="descname">apply</code><span class="sig-paren">(</span><em>epoch</em>, <em>value</em>, <em>grad</em>, <em>step=-1</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.optimizer.L2Constraint.apply" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+</dd></dl>
+
</div>
@@ -255,9 +709,7 @@
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/snapshot.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/snapshot.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/snapshot.html (original)
+++ incubator/singa/site/trunk/en/docs/snapshot.html Wed May 9 15:25:26 2018
@@ -188,8 +188,58 @@
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
- <div class="section" id="snapshot">
-<h1>Snapshot<a class="headerlink" href="#snapshot" title="Permalink to this headline">¶</a></h1>
+ <div class="section" id="module-singa.snapshot">
+<span id="snapshot"></span><h1>Snapshot<a class="headerlink" href="#module-singa.snapshot" title="Permalink to this headline">¶</a></h1>
+<p>This script includes io::snapshot class and its methods.</p>
+<p>Example usages:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">singa</span> <span class="k">import</span> <span class="n">snapshot</span>
+
+<span class="n">sn1</span> <span class="o">=</span> <span class="n">snapshot</span><span class="o">.</span><span class="n">Snapshot</span><span class="p">(</span><span class="s1">'param'</span><span class="p">,</span> <span class="kc">False</span><span class="p">)</span>
+<span class="n">params</span> <span class="o">=</span> <span class="n">sn1</span><span class="o">.</span><span class="n">read</span><span class="p">()</span> <span class="c1"># read all params as a dictionary</span>
+
+<span class="n">sn2</span> <span class="o">=</span> <span class="n">snapshot</span><span class="o">.</span><span class="n">Snapshot</span><span class="p">(</span><span class="s1">'param_new'</span><span class="p">,</span> <span class="kc">False</span><span class="p">)</span>
+<span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">params</span><span class="o">.</span><span class="n">iteritems</span><span class="p">():</span>
+ <span class="n">sn2</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">)</span>
+</pre></div>
+</div>
+<dl class="class">
+<dt id="singa.snapshot.Snapshot">
+<em class="property">class </em><code class="descclassname">singa.snapshot.</code><code class="descname">Snapshot</code><span class="sig-paren">(</span><em>f</em>, <em>mode</em>, <em>buffer_size=10</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.snapshot.Snapshot" title="Permalink to this definition">¶</a></dt>
+<dd><p>Class and member functions for singa::Snapshot.</p>
+<dl class="method">
+<dt id="singa.snapshot.Snapshot.read">
+<code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#singa.snapshot.Snapshot.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Call read method to load all (param_name, param_val)</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Returns:</th><td class="field-body">a dict of (parameter name, parameter Tensor)</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+<dl class="method">
+<dt id="singa.snapshot.Snapshot.write">
+<code class="descname">write</code><span class="sig-paren">(</span><em>param_name</em>, <em>param_val</em><span class="sig-paren">)</span><a class="headerlink" href="#singa.snapshot.Snapshot.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Call Write method to write a parameter</p>
+<table class="docutils field-list" frame="void" rules="none">
+<col class="field-name" />
+<col class="field-body" />
+<tbody valign="top">
+<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
+<li><strong>param_name</strong> (<em>string</em>) â name of the parameter</li>
+<li><strong>param_val</strong> (<a class="reference internal" href="tensor.html#singa.tensor.Tensor" title="singa.tensor.Tensor"><em>Tensor</em></a>) â value tensor of the parameter</li>
+</ul>
+</td>
+</tr>
+</tbody>
+</table>
+</dd></dl>
+
+</dd></dl>
+
</div>
@@ -255,9 +305,7 @@
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>
Modified: incubator/singa/site/trunk/en/docs/software_stack.html
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/en/docs/software_stack.html?rev=1831260&r1=1831259&r2=1831260&view=diff
==============================================================================
--- incubator/singa/site/trunk/en/docs/software_stack.html (original)
+++ incubator/singa/site/trunk/en/docs/software_stack.html Wed May 9 15:25:26 2018
@@ -346,9 +346,7 @@ model parameter values using parameter g
<script type="text/javascript">
jQuery(function () {
-
- SphinxRtdTheme.Navigation.enableSticky();
-
+ SphinxRtdTheme.Navigation.enable(true);
});
</script>