Singa restricted Boltzmann machine implementation

[Lewis John Mcgibbney <le...@gmail.com>]

Hi Folks,
Can someone please detail if the RBM implementation(s) in Singa has
detailed documentation I can find over and above [0][1].
I am particularly interested trying to utilize the Singa RBM implementation
for unsupervised, online (WWW) learning.
Apologies if I am way off with some understanding but some initial guidance
would be very much appreciated.
Thanks
Lewis

[0]  https://singa.incubator.apache.org/docs/rbm.html
[1] http://www.cs.toronto.edu/~hinton/science.pdf

-- 
*Lewis*

Re: Singa restricted Boltzmann machine implementation

[Wang Wei <wa...@comp.nus.edu.sg>]

Hi Lewis,

SINGA uses Contrastive Divergence (CD) algorithm to compute gradients of
parameters, and SGD algorithm for parameter update.
We do not provide documentation on the CD algorithm, which is explained in
detail in Hinton's paper and some other online tutorials.

Generally, we convert the undirected connection between the hidden layer
and the visible layer into two directed connections as shown here
https://singa.incubator.apache.org/docs/neural-net.html.
Then the CDWorker's TrainOneBatchFunction runs the CD algorithm:

1. forward data to the visible layer
2. run positive phase and negative phase by calling ComputeFeature() of
each layer.
3. compute gradients by calling ComputeGradient function of each layer.

We will update the documentation for the RBM example after doing some code
cleaning.
If you can share with us some details of your application (on online
learning), we can give suggestions on how to implement it in SINGA.

The links to the RBM implementation files:
https://github.com/apache/incubator-singa/blob/master/include/singa/neuralnet/neuron_layer/rbm.h
https://github.com/apache/incubator-singa/blob/master/src/neuralnet/neuron_layer/rbm.cc

Best,
Wei & SINGA Team

On Fri, Oct 16, 2015 at 4:47 AM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:

> Hi Folks,
> Can someone please detail if the RBM implementation(s) in Singa has
> detailed documentation I can find over and above [0][1].
> I am particularly interested trying to utilize the Singa RBM implementation
> for unsupervised, online (WWW) learning.
> Apologies if I am way off with some understanding but some initial guidance
> would be very much appreciated.
> Thanks
> Lewis
>
> [0]  https://singa.incubator.apache.org/docs/rbm.html
> [1] http://www.cs.toronto.edu/~hinton/science.pdf
>
> --
> *Lewis*
>